I used to be a swimmer, and I had to eat 4000+ calories to maintain a bodyweight of 150lbs at 5'11. And that was heavily loaded towards carbs. Otherwise I couldn't perform to the degree required of me.
In college my girlfriend was on our D1 swim team and she ate like absolute **** just to maintain herself at a reasonable weight. I kid you not, the guys on her team drank like fish, ate whatever they could force down their mouths, and didn't care what it was as long as it was food. And the majority of them were lean and cut. Because they were swimming two a day practices 3 times a week, and the other 3 days per week that the practiced was one pool workout and one "dry land" workout (calisthenics or weight training). They could consume upwards of 6-7k calories per day without batting an eyelash.
Swimming is an intense sport that requires a large caloric expenditure, hence their ability to eat large quantities of food and stay lean.
Now don't get me wrong, I'm not saying it's a healthy diet for them in any way. Because when they stop swimming they have issues adjusting to everyday life (food-wise). For instance, my ex-gf who was the D1 swimmer, put on a sizeable amount of weight after she graduated due to her eating habits that were perfectly ok when she was swimming incredible amounts.
The disbelief that he requires that many calories is ignorant, due to most of you not realizing the caloric expenditure that comes from swimming. Phelps swims 4800 meters in the morning to "warm up", thats 3 miles gents. I'm not surprised he has to eat 10k+ calories to maintain his body DURING the Olympics. I don't believe he eats like this every day. I think this is geared towards keeping him flush with energy during the multitude of events he's swimming in such a short period of time.