Most upvoted comment
High-level programmer wants to learn about manipulating binary files, bytes, and hex. Where should I start?(r/compsci)
I know exactly the book for you.
It’s a well written book that will guide you through the bits and bytes, the logic gates, the assembly instructions, and the hardware that implements it. It’s a nice casual reading style, not like many more dry textbooks. Get it now.
If you want something more in-depth and textbookey, with excercises etc, check out Patterson and Hennessy’s Computer Organization and Design which is a good book, but perhaps a bit more than what you want to dive into right now.