Reading the book from Bill Gates on climate change right now.
Most of the CO2 we produce (and I think this applies to energy too) is actually indirect, in the manufacturing processes for making stuff. Any stuff from industry and agriculture, specifically concrete, steel, and meat. I used to think this is because manufacturing takes energy (chemical reactions, for example to cook cement, or melting mineral to make steel and aluminium, occur at high temperatures), so if we finally have clean energy we solve the issue for good. But unfortunately not so. Even if we had 100% clean energy, beyond the high temperatures required, it's really the chemical reactions themselves, used by industry and agriculture, that release greenhouse gases. So even switching to 100% clean energy we would still be emitting CO2.
The book is very good, I recommend it.