I wasn't thinking of a PS5 handheld, but something on par with whatever the Switch 2 is going to be.
I'm very well aware of the limitations of current portable tech at the moment, but if a portable that could potentially run PS4 level games is possible, I could see there being a market for it.
The real concern would be software, which is why it being able to run PS4 games natively could be a major selling point; I personally have a combined PS4/5 library of over 900 games, with the majority being PS4 games or cross gen ready games.
I wouldn't necessarily worry about how much support the new handheld was getting if I had access to all that library, many of which are games I've yet to install on my console, let alone play.
In either case, I wouldn't put too much stock in these rumors, but like I said, I can see how, in current state of the industry, and how much the Switch dominates Japan as a primary gaming console, Sony would explore ways in which they could compete without resulting in another Vita (man I loved that thing, and was so sad it didn't even reach PSP numbers 😞).
It doesn't mean they'll actually execute on the idea, only that I see the merit in exploring those ideas.