Just to clear up a couple of points,, Permitted Development rights only apply to holdings between 0.4 hectares up to 5.0 hectares and farm units of that size only benefit from Class B rights. Over 5.0 hectares (approx 12.5 acres) and you gain Class A rights. Class A rights are more comprehensive, the biggest gain being the right to create new agricultural buildings.
Those rights have to be confirmed with your local planning authority and to seek there agreement. It is not as easy as folk imagine as all the conditions within the regulations have to be complied with. Fail one and the planners will reject your development rights.
As for getting a dwelling on your farm land, that is never an easy task. It is slightly complicated just at this moment due to the new guidelines and policy (NPPF National Planning Policy Framework) being introduced, the wording of which seems simpler but has left a lot of grey areas which need to be determined.
As for the disability issue, I think that would begin to raise some interesting planning issues and might challenge some of the usual thinking, especially when one brings in to play, items like the Human Rights Convention and equality rights.
If you are wishing to pursue the idea further, I write a lot on the Field to Farm forum. Please don't take that as a plug or spam, it's just where I happen to be most of the time with like minded individuals.dealing with agricultural planning issues.