Yes, the more I’ve experimented and looked at other people’s results, the more convinced I am that it’s almost always better to ignore the color restrictions.
The added survivability benefit is just too modest to matter much.
Where it can be more relevant is when the removal of a color fundamentally alters the nature of a particular hero, or makes a particular skill set unavailable or limited — e.g. 3* Tournaments without purple lack a way to clear status ailments, and Ursena’s extra holy damage and reflection aren’t relevant in a Tournament like this one.
It can also augment the strength of a hero that would already be a strong selection, for instance making Black Knight an even more effective tank when blues are disallowed; or making Mitsuko even more compelling when another color is limited, so an attacker feels pressure to consider choosing non-blues too, effectively further limiting the choices.
I do, however, think it’s worthwhile to consider the Special Rule, which has a much more material impact on battles.
For Rush Attack, Slow AoE or powerful buffs/effects become much stronger, as does healing — and, in turn, you should expect that an attacker with anything but a terrible board is going to have an opportunity to make use of some powerful Special Skills. I find Rush Attack battles are often either very short, or very drawn out because of that — you either die quickly, or there can be a long volley of damaging and healing back-and-forth.