Short answer; there won't be a huge difference, no, but you will probably want to reach the reload cap anyway and by not delaying your gunners you haven't made it as hard as you think.

Explanation;

The armament sailor class, the basic one that most people delay at, adds a total of +7 reload to reload growth and adds 6 rookies every level.

The AA gunner class lowers reload growth by -5, this is the class you should choose when you are done delaying, and it also adds 6 rookies every level. This class makes it harder to reach the reload cap by a bit (-5 is not the worst reload reduction among the AA classes).

The next class is the DP gunner class. It doesn't add or remove and reload, and it also adds 6 rookies every level. So, this particular class does nothing to make reaching the reload cap easier or harder.

The last class is the Chief DP gunner class. It also doesn't add or remove any reload, however this class adds 7 rookies every level, 1 more than the rest. This extra gain in rookies allows you to add more experts, so this class actually helps you reach the reload cap.

Overall, if you class on time, the only modifiers to your gunner are that you gain 1 more rookie per level but your growth is -5 less when compared to a delayed gunner. Effectively, instead of having a sailor with fewer men each with 'high' reload ability, you will have a sailor with more men, each with less reload ability. The only downsides are that having more men makes a sailor a little heavier, and you need a few more experts. On the plus side, you get access to your stronger AA guns earlier than someone who has delayed their gunners. (Although, some people quite like the 3.9" guns).