You could Up the fat...it's only 10% of your calories...by adding some fish oil or evoo. Add 10g fishoil and 2 tblsp of evoo will give you back ~300 calories.
At 180lb you are eating at 16x bw. Dropping the 500 calories (800 total carb calories) will put you at about 13+ x bw. Additionally try to keep your carbs around training times and less at the end of the day.