This seems like a lot of work and it is but it's not horrible.
If you're using Word, you're going to make 4 separate sections.
You'll have front matter and a contents page. More on that in a moment.
You'll have a section for each book.
This is not page break.
Look under page layout and you'll see section break. Use that.
This way you keep the TOC for each book separated.
In the front matter content, use hyperlinks and direct it to the bookmarked title.
Don't make another TOC.
Don't give Word reason to get confused.
And that's really it. You copy and paste to 1 long document, separating each with a section break each time. Clean up where you need to.
So yes, it's work but it's not horrible.
How you do the cover is up to you. I know someone who did it in Paint and that's about as basic a program as you can find. I tried a couple different ways and then paid for a photoshop action that automates the process beautifully. It is possible to do it manually and it will help if you're good at geometry which I'm not.