Welcome to the forum!!
My fields have dry stone walls round, which are ideal for goats to stand on and get over!! When I first got Cain and Abel, they were always jumping the wall, they are Anglo Nubian and really lanky. We put fence posts up next to the wall, and then put rails along the top of the wall. They tried and failed to get out, and have not bothered since.
Paddy and Jasmine arrived, and they find it easy to hop over the walls when they fancy. For the other goats, which are pygmy, we have sheep netting with a strand of wire on the top. I cannot believe your goat can fit through the stock fence - although I have had them go under it.
I have not tried electric fencing for the goats, maybe that is an alternative.