The short answer is yes you can use pig muck in your veg garden.
This is what we do, we gather all the muck and dirty straw bedding in a heap and let it rot over at least six month before spreading on the veg garden. Pig muck is quite acidic and can burn if put on neat! So for you gather it all in heap over the summer whilst you have the pigs and then put it on your veg garden over winter and let the worms and nature do its stuff.
As we have a larger permanant herd we bring our pigs in over winter and all the winter muck, bedding etc goes in one heap, at the start of summer this heap is turned over with the JCB and set to one side, then we bring the previous years heap down which is now a crumbly lovlinness and we spread some in the poly tunnel some on the veg garden and the rest is used to grow pumpkins on, when the pumpkins have finished it goes on remains of the heap go on the veg garden to rot down over winter and the cycle begins again.
HTH
mandy