If it's some horrible, so-called weatherproof paint you've got on there, try and scrape it off first. Or just live with it for a while longer and let it weather away. I tried jetting some paint off my old natural stone walls (Dorset) but ended up pitting the soft sandstone. Also watch your step with 'Breathable'. The term is bandied about almost to the point of meaninglessness. Find a simple, natural lime wash and stick with that.
I'm guessing, but there must be tons of lime wash paint suppliers in Scotland you could choose from given the prevalence of natural building materials.