We managed for 20 years, travelling 8 miiles to our land. Yes, it is a problem in the bad snow etc. and when an animal is ill, lambing or whatever. I have been known to travel back at midnight to check on a sick animal, so thats the sore point of not living there. But its a start on the smallholding ladder, and I am sure thats what you want, so I would do it. I know lots of people who travel to their land, even farmers have outlying land from their house.
As to security, I never had a problem, except the neighbour, who wanted money for drugs so helped himself to two lots of diesel. I never lock my sheds, or feed stores, and nothing has ever gone, although I appreciate thats not the case nowadays sadly.