I don’t have Jabberwock to test this on, but since the formation still uses the same 1-5 input as the normal ‘V’ shape, I assume it still goes by position.
So for a front 3 - back 2, you set up your raid defense with the front three being the wings and tank, then your flanks are the back 2. So I would assume Jabber would hit the 2 far-edge front heroes since they are still technically the closest to the edges.
For a front 2 back 3, Jabber would likely hit the back 2 edges, since they would be positions 1 and 5 in your lineup.
Just my guess.