Except for Eprizero no wormers are licensed for dairy goats. The vet can prescribe (or if yo have sheep it is easy to buy wormers in agri store, just don't tell them its for goats) under the cascade system, for most of the sheep wormers the goat dose is twice the sheep dose, for the yellow ones it is 1.5 times sheep dose. Drenches are recommended over injectables, but Eprinex/Eprizero is a pour-on.
For wormers not licensed for goats the usual milk withdrawal time (in the absence of a period given for cows) is 7 days, if you are organic then double that. Most wormers will say not to be used for animals producing milk for human consumption, as these have never been tested for that - too expensive for the drug development company to out medicines through the regulatory system for goats, but you have no choice but use them. I would also do a FEC ten days/two weeks after worming to test efficacy. It will probably take you a couple of years to get to really low worm counts, as long as you keep a closed flock.
Just out of interest - what wormer did you use and at what dose?