Compost is perfect for the root crops which don't like manure - it gives them nutrients without too much of whatever makes them twist and turn with manure. It will also help to lighten the clay soil. For me compost is almost more valuable than manure, maybe because I have loads of manure but not much compost.
I would be more specific with where you apply the manure. Potatoes and brassicas are heavy croppers so need plenty of manure dug in, whereas crops such as lettuce don't need manure. Root crops can follow after brassicas in your rotation so will get the leftover nutrients from previous years (after compost in the first year). Peas and beans can use a little manure but don't need a heavy application. Tomatoes, squashes and so on get plenty of manure but I dig it into a 50cm wide and deep hole just where they are going to grow. If you apply manure across the whole bed you will encourage weeds in places where no veg are using the nutrients when you are growing plants such as these which are widely spaced.
There will be lots of worms in the manure and compost - tiger worms - and the lumbricids (the big earth worms) will come in once they sense all that manure and tilled soil.