With only 4-6 acres, it could be tricky to manage the two separately if they need it - and Castlemilks can be a bit, shall we say, assertive, with other sheep. I'd always be cautious mixing polled and horned ewes, so personally I think I wouldn't start out with those two breeds.
Castlemilks can be delightful sheep, but I think I wouldn't advise anyone to get them who doesn't love them to begin with. Some of the things we have to do with sheep, and some of the problems we get trying, are so very much harder and more stressful for all concerned if you don't really like the sheep you're working with.
So my advice would be to get some boring little white store sheep for your first year, to learn the basics of sheepkeeping - there may even be a local farmer who can supply pet lambs for you to rear, but make sure your other local smallholders can vouch for his/her lambs, that they'll have been given a proper start.
Then, once you know a little, buy the sheep you really like.
Just two other things I'll say - if either of you spins

or fancies spinning, then that might push you away from the Balwens. You can spin Balwen, and it's great for hard-wearing items such as bags and mats, but on the basis of the batt I had, it's not something you would make your own jumpers out of, and personally I didn't enjoy spinning it very much. (And I'm not a person who thinks all fibre must be fine and soft, by any means, but this was a bit close to Brillo for my taste!)
Having said which, Castlemilk isn't a novice spin, and not all Castlemilks have great fleece for spinning - but the good ones are divine.
My final comment is that, if you do decide to go two breeds, one option for breeding would be to have a Balwen tup at home and send the CM girls to your neighbour's tup, if your neighbour was agreeable. You could always buy tups between you, on the understanding that your girls go to your neighbour's spot for tupping. Just a thought.