Itachi was way too nice to Sasuke in those memories for there not to be something else going on with the clan and his real motives for leaving