If you are ultimately selling the produce as food (in its basic form), then the VAT should be a one-way street for you. I agree there is a degree of paperwork involved, but it really does not take long if you are reasonable with excel/figures/accounting software.
It is surprising how much stuff which is required to keep animals attracts VAT. Fencing, gates, wire, quads, trailers, tractors, contractors, feeders, feed bins, tools, equipment etc etc. The list goes on and on and on.
If you arlready have most of the kit required and your fields/fencing etc are already all in good order then you may not actually reclaim that much.
You do need to be a bona-fide business though, not a glorified hobby, otherwise the VAT man will not be a happy bunny.