When I first came to this farm, the bull was a Charollais and a lot of the calves were dopey about getting on sooking. Now we have Angus bulls and don't have the problem. Hereford AI calves have been great, too.
There can be a correlation about birth size and vigour - I guess a large calf has a more arduous birth, which takes it out of them. Generally bull calves are bigger than heifers, which could account for a perceived gender difference?
And actually, birth size could also be related to breed differences. We like wee newborns that grow like stink; the continental breeds have a tendency to be born larger, on the whole.