As long as you are providing the body with other source of fuel, such as fat, it is perfectly fine to workout without carbs. I swim 5 thousand meters (5.468 yards) twice a week on 90 grams of mixture of nuts (60 g peanuts + 30g Brazil nuts).
You just need a little longer for digestion instead of 45-60m prior exercice I eat them 90m before.