I am a hardgainer like yourself. I think a lot of the time, hardgainers just don't have the same appetite as the naturally bigger guys. Consequently, it is good to shoot for foods that have a higher calorie density. That way you don't fill up so easily. You still have to plan your meals, just include the higher denisty foods.
Low fat foods don't fall into this category. Red meat, natural peanut butter, and avocado are examples of foods with good fat and higher calories. Starchy carbs like rice, oats, and potato (reg, sweet, yams), are higher calorie density carb sources. Of course don't forget the veggies but don't overdo it with them.