I would give them some nice soft hay, yes. Little and often, so it's always fresh and clean. They will probably just mess around with it at first.
Views differ about letting lambs have access to grass while still having milk. The key thing to keep in mind is that milk overflowing the milk stomach and getting into the rumen can be very serious; bottle lambs haven't learned when to stop feeding; and grass fills the rumen, meaning there is less room for milk in the milk stomach, so they may not have room for the same size bottle if they have been eating grass. Or indeed, if they've just eaten a bellyful of creep or hay.
If you have a patch of poor grass, and can keep them penned on a small patch so can limit how much they can access, then you could start letting them have a bit of time each day on that grass. As a rule of thumb, give them an hour or two between milk and grass, and between grass and milk, and don't just let them empty their bottles, also keep an eye on their sides. If they start to get "love handles" as they drink (ie., bulges behind the rib cage as you look down on them), take the bottle away; they're full. Too much milk will do far more harm than too little.
At five weeks they can't really digest much forage yet, so they need very little, it's just about learning to eat it and helping the rumen develop, and grow an appropriate population of bacteria.
By 6 weeks the rumen is functional but not yet really able to meet all their needs. By 8 weeks it is fully functional, so if at that stage they are eating enough forage and creep, you can stop the milk.
Having said which, there are folks on here wean at 6 weeks, have done it for years, and rear perfectly healthy lambs every year.
The other point which may not be clear from the above is that ruminants need the right gut bacteria to digest their food, whatever it is. So introduce new things slowly, and make changes slowly.
