As a non-resident, but married to a Stornowegian with parents and family still there, I can tell you that my own experience is that the culture shock is the biggest issue. The weather, apart from the terrible storms in late autumn/winter, isn't dramatically different from the mainland, in fact the winters tend to be a bit milder. But that wind...
What I find difficult getting used to when I'm back on the island, are the limitations in various aspects of life: there's not the same freedom of movement on a Sunday, or indeed at any time unless you've got plenty of money: the price of fuel is astronomical. Things in Stornoway itself seem to be changing, slowly, but the Church still has a pretty firm grip on life in the islands, so there's no popping out for some shopping or to the cinema on a Sunday. We've gone out for drives on Sundays, and driving through some communities is... interesting... with unconcealed disapproving stares from the people heading to/from church
I had a funny experience one Sunday in going to the petrol station, run by a Norwegian, and it was pretty busy, but few folk spoke to each other, even those who knew each other, everyone trying to pretend they weren't actually there on the Sabbath
The island itself is beautiful, particularly the beaches on the west, and of course the mountainous isle of Harris is stunning too. We tend to travel up via Uig on Skye, landing at Tarbert a couple of hours later, and driving the 40 mins up the road to Stornoway - it's a better, and cheaper, journey than the 3 and a half hour ferry ride from Ullapool.
It's a great place, I don't know if I could live there myself, and I know it was incredibly difficult for my husband and friends of his generation to grow up in, but if I wanted to remove myself to somewhere totally different, peaceful and harkening back to a simpler and less commercial existence, then I'd have no hesitation in going to live there.