You will eventually need them all for the advanced buildings. They may seem useless now, but they all have a purpose. You dont need to level them all up now, just focus on the ones that will bemefit you.
Forges are converted to advanced buildings much later in the game. But yes they will seem excessive now.
All four houses are needed to reach the adequate amount of recruit storage to train in said advanced buildings.
Having four TCs gives you more options.
If you are a cheap player, you will likely run 2x TC20, and having the option to then build both TC2 and a TC11 will be valuable.
This is how I started, and then later converted to running TC2, TC20 and 2x TC11.