Here's my opinion for what it's worth.
The main thing is not to give up and settle for a compromise.
If you do that it will most likely never happen for you in this lifetime, and chances are you will not be fit for it anyway. The spirit of the times is against smallfarming, as it so often has been throughout history.
The System wants consumers, TV watchers, 'stuff' buying workers, not self sufficient, frugal, thrifty types who don't want to join in and fuel the consumer economy.
I notice this in Canadian farming. The govt has no time for small players. Farmers are encouraged to think of themselves as Commodity Producers, to get bigger or get out. To farm more and more land in order to justify purchasing bigger and more expensive machinery.
Let's be honest, self sufficiency is selfish. A farmer who is feeding 800 people a year with his wheat crop is far more valuable to the system and humanity? than a smallholder who is just feeding himself and being smug when the power goes off!
In my own case I bought a 4 acre smallholding in Ireland for very little money, about the cost of a new car at the time, cash only, no one would have given a mortgage on it, at a time when being English was not a cool thing to be in that location. Worked it up, made a business selling herbs and vegetables, rode the Celtic Tiger, and sold it for 5 times what I paid for it.
Wishing to own more land I moved to Canada, did exactly the same thing again. Bought 22 acres in BC from a bank, it was a bankruptcy sale and I had to go to court to buy it. Again, after working and building, sold it for 3 times what I paid for it and moved to the prairies, just before prices started going crazy here too.
The Chinese are buying up Saskatchewan farmland for gawds sakes! What does that tell us?
So now I own outright about 200 acres, no debt. As a smallholder out of debt is out of danger, debt just sucks you back into the system. This is considered a very small holding here, just shows the meaning of relativity!
Several times along the way I was tempted to give up and get a job. This might have been a wise thing to do in hindsight, who knows? I have worked part time jobs along the way, but only in order to fuel my main project. But as I say, you have to fix your mind on your goal and just keep on.
Not easy, but then it never was...