Some people do disappear within a single year, if that's any comfort. The majority of stays are relatively short.

[Relatively, because he's been here long enough for anything short of a decade to seem short.]

The Elite Four and the Champion are, officially, part of the league; thus, twenty-one. But if you'd rather not face that challenge or if you only care about the badges themselves, you'll only have sixteen leaders to deal with, yes.