Obviously, as you said Tiggs, you're looking for an easy practical solution that you can apply now, with the legal route as the last resort.
I have had a slightly similar problem whereby I agreed for someone to graze some horses for a couple of weeks and then there was always some excuse as to why they didn't come off. There are always ways and means.
How you go about things really depends on whether you want to still remain on good terms with your neighbour or not.
So - you could give him warning that you are going to spray the field with Roundup or pig slurry prior to to reseeding. And be prepared to do it.
Or you could pull down down the already poor fencing, while making it look like it's just collapsed. Obviously tell him as soon as you "discover" this to avoid the animals wandering too far and maybe causing an accident.
You could threaten to report him to the British Cattle Movement Service for not informing them of movement of cattle. I'm assuming here that your field is not in any way recorded as being part of his holding. This threat could induce him to move them soonest as otherwise he may face an inspection by the RPA (alias the gestapo) which no one wants. Similarly you could report him to your local animal health office, again for unofficial movement of cattle.