They really need milk until at least 6 weeks, preferably 8 (although I know others on here disagree with me on that, and wean at 5 and 6 weeks.). The rumen is not fully developed until 8 weeks, and isn’t really developed enough to sustain life and health until around 6 weeks.
So it would be worth persevering if you can. I had two lambs passed to me at just over 4 weeks of age. One of them in particular was very anti a bottle. I penned them with hurdles on straw, with hay but not much creep, and offered a bottle twice a day, persevering each time until I got at least some into both. They had other sheep nearby / next to them but not in the pen with them. After maybe three days, one lamb was drinking well but the other took over a week. It did come round though, although that one never actually ran to me for its bottle, I always had to catch it.
For the period where I was keeping them both penned in order to get milk into the awkward one, I was never sure if I was doing the best thing. But seeing them at 4 months, both strapping great healthy things, I knew I’d made the right decisions and given them the best start.
Sometimes it feels mad to put so much effort in for so many weeks, into an animal which will be going to slaughter at about six months old, but when they are strapping great healthy lambs, enjoying life out at grass for the summer, not pot-bellied poor things that often don’t come through the winter, then it seems worth it to me