I was thinking, the reason I don't like bossy people is that I lack the confidence to be as bossy myself.
That would mean that I am jealous of their confidence.
I realize that it's not always confidence that leads to a bossy, loud, obnoxious, annoying people. It can just as well be a lack of confidence, a cloud of self-denial and so on.
So there is really no point in being jealous. But I still find them annoying.