Here is what I do:
1. Provisional cast on, across the shoulders, for the back. Knit down to bottom of armholes. Stick stitches on waste yarn.
2. Pick up shoulders and knit down the front, doing whatever neck you want, til you get to the bottom of the armholes.
3. Put all stitches onto one circular, knit body to length required, shaping where needed.
4. Pick up round one armhole, knit to bottom of sleeve, shaping as you go. Do the same for the other sleeve.
The huge benefit is this, that apart from whatever you want to do with the neckline, is the jumper done. No sewing
For a cardi, just knit back and forth instead of round and round.
I found the book 'Custom knits' by Wendy Bernard really useful. It's all top down and positively encourages 'make it up as you go along'.