It's so easy to take on too much in your initial enthusiasm, before you realise just how much work is involved in each project. You can get so caught up in the endless slog that you lose your perspective.
Take a step back and look at the whole venture, what you both enjoy, what brings in profit (do you need to make a profit?), and what is vital to your chosen lifestyle. Decide if you want to be a farmer or a smallholder (think about it, they're different)
I would ditch the pigs for now, easy to replace them later. I agree about getting rid of most of the sheep, just keep a small breeding core, then when the children are older and you have become more efficient at what you do, (and they eat more) you can build your numbers back up. Keep your core flock well away from any stock on rented out fields, for biosecurity.
There is a limit to how much fruit and veg two people can eat, and your children are too small to eat much yet, so don't waste time growing more than you can use. Only the compost heap benefits from that.
We are currently going through something similar. We are at the opposite end of the age scale, and sometimes (mostly) we are just worn out. We realised the fun was going out of what we were doing. So, this is our last year breeding our Hebs, although we'll keep a dozen old ladies and a couple of tups. We now have just a handful of hens, which produce enough eggs for the house but none to sell on. We did get the geese and we are in two minds about them.
A big change has been in the veg garden. We have grassed over at least half of the outside veg area and concentrate on growing the foods we most like. We used to sell a surplus at the gate, but now we don't produce any surplus as it's just a waste of our hard work. I have to be realistic about what will be eaten, what will be processed for storage, and how much of that will actually be eaten.
We used to produce enough for our own children and their families, but we've realised they have far more energy than us so they can grow their own.
There are various ways you can make your veg growing less hard work, such as adopting a 'no dig' approach. We are currently experimenting with dividing the whole outdoor veg area in two. One half is covered with very strawy manure, then covered with a tarp and left for a whole year. meanwhile all the crops are grown in the other half. When the tarp is taken off, the soil should be soft, friable, weed free and ready to plant strait in to, with just a few thistle roots to take out.
We also use two rotavators, one for heavy work, the other a narrow machine for weeding between the rows.
It's all about efficiency, so you get the best return for your input.
Many couples do burn out for the very reasons you say, so it's good you've realised in time to do something to prevent it. This lifestyle choice (loaded expression) has to be fun and enjoyable or you might as well just go back to the rat race (no - don't do that
)