I have been following this thread with interest and I have found the replies very thoughtful and reflecting peoples experiences.
One thing no-one has touched on is your children. You will still have to spend a lot of time rearing them and particularly as they grow they may love or hate where you end up. You will need to think about schooling, both primary and secondary, and getting them to school. Once they become teenagers their surroundings will be influential on their futures - this can be in either direction, for good or bad. Being in a very rural area could hold them back from what they want to do with their lives, or they could share your dream and a smallholding could be everything they want. There's no telling, but they need to come into your equations.
Our own experiences lead me to mention health - if your whole life depends on your being fit, you can come acropper if you are hit by a major illness - and when it happens to both of you it can mean the end of your dreams. In our case we have adapted, and our smallholding has adapted with us, but when you are starting out it is as well to check that there is flexibilty in where you choose to settle.
In our case, when we bought our smallholding nearly 17 years ago, we bought the smallholding before we had sold our house, which was quite a risk, but we didn't want to miss the smallholding and we were fairly sure the house would sell quickly - it did, within a week, which was great, but it was a risk and we could have ended up with bridging loans and even more stress.
You have the opportunity to choose carefully where to go, and there seem to be plenty of potential smallholdings out there so you can be picky. My main advice would be to know just what you are seeing when you look at the land - dig holes and assess the soil carefully. The house and buildings can be changed but you are stuck with the land, and where it is.