Technically, worming ruminants that have fasted before treatment means their rumens are emptier, which means the wormer is floating in a lot less stuff, so will be absorbed quicker.
Generally, if you are making sure you have the correct dose of wormer (weighing or estimating the heaviest animals so as not to underdose) then it should not reduce the efficacy, but like you say, it may work quicker on an empty stomach.
It can be tough to fast animals, as if they are out, they will be grazing, and if in, they can get upset, or even upset stomachs from not having hay to munch on, so it is a bit of a balance.
I tend not to starve mine beforehand, but as a busy vet, this is often because I am never exactly sure when I will be treating, it is often, "right, I'm not at work, the weather's good, it's gonna be now!"
Hope that helps