I found this book to be extremely insightful on the process and pitfalls. Communication in writing and being specific about grading specs for various finishes was one of my big takeaways. There's a lot more to it though. Having an architect act as an independent PM may be a good idea too if you forgoe a GC.
That said, I didn't follow through yet and haven't built anything myself. Even if you do use a GC it's still a great read.
Edit: I did get as far as talking to my go-to mortgage banker about construction financing, and learned that self-GC'd projects are pretty much not finance-able through traditional paths. Many of these projects end up unfinished, with the GC-owner running out of money. And with no certificate of occupancy they can't be easily sold or refinanced.
Having an architect can be helpful but it can also be significantly more expensive and if there is a problem, then you are likely to be pulled into the argument anyway.
Definitely communication, good diagrams, developing a good rappour with the contractor and not being overly tight or strict with them otherwise you can build resentment. For example, don't withhold all of their expected money because of a single piece of dodgy drywall, give them most or all of the money and make sure they make-good first thing next week.
That said, I didn't follow through yet and haven't built anything myself. Even if you do use a GC it's still a great read.
https://www.buildwise.org/review-what-your-contractor-cant-t...
Edit: I did get as far as talking to my go-to mortgage banker about construction financing, and learned that self-GC'd projects are pretty much not finance-able through traditional paths. Many of these projects end up unfinished, with the GC-owner running out of money. And with no certificate of occupancy they can't be easily sold or refinanced.