Glancing at a script for the episode, Kirk brings up the subject of "the penalty" for transporting harmful animals, and Spock informs him that "the penalty" is 20 years in a rehabilitation colony. Both Kirk and Spock seem to be in agreement that it's a penalty, not medical/psychological help.
Which suggests that it's not so much a rehab colony as it is a penal colony. Or perhaps it's a rehab colony that has the ability to double as a penal colony. I suspect that it's a jail that has a strong focus on rehab, and if you get rehabilitated, then you're still in jail for the rest of your sentence but at least you're rehabilitated now and unlikely to return, and if you don't get rehabilitated, then at least they'll let you out when your time is served instead of keeping you until you break. Also, it's probably socially-helpful for an ex-con to say they just got out of rehab, rather than saying they just got out of jail. Regardless, it seems that they have a fixed-sentence punishment, which Kirk could choose or choose not to inflict on Cyrano Jones, if Kirk (somewhat arbitrarily) decides that the punishment should apply to him.
Spock calculates that the cleanup would take 17.9 years, which Jones is pressured to "choose voluntarily" instead, as both Kirk and Spock knew he would, because apparently cleaning up Tribbles for 18 years is more pleasant than the "rehabilitation colony" experience, either that or 2.1 years of freedom really means a lot to these people.
(Also, the "rehabilitation colony" is probably just jail, but Gene Roddenberry didn't like the idea of jail, so he quickly renamed it to something better, without deeply and fundamentally changing it. He probably didn't even have a clear idea as to how a rehab colony would work better in society than a penal colony, but he wishes it could, and it's his fiction, so in his world we can assume that it exists and it works.)
