No, because…
…they do.
Defense stats affect damage taken from direct-damage Special Skills (though not DoT).
And in addition to that, as @Kilted noted, there’s a random component to damage, even apart from the Attack and Defense stats at play, which is the θ in this equation:
As you can see in that formula, damage is a function of the ratio of Attack and Defense, so it’s going to be different for each enemy, based on their Defense amount, including application of Troops and status effects.