Pokémon in the overworld could be just as much of a surprise as random encounters if they were programmed well with AI and intractable in world/hiding in tall grass, etc. Sure, I wouldn't trust Gamefreak to do this and instead they'd just reuse let's go's way of handling it, but... I do think it's the future and it's what a lot of people want. Just a matter of when.
I'll be upfront and say I prefer random encounters.
With the said, until Game Freak can deliver Pokémon in the overworld (which implies much more then AI) I don't want to see it in a new gen game. Why?
In Let's Go go to Viridian Forest. You will see a Pidgey or Pidgeotto chilling right next to a Weedle or Caterpie. What wrong with this? In the Poké Dex and 1st season of the anime the birds eats the bug. Just knowing that broke my immersion of Let's Go. There are other examples like this.
The World needs to be bigger for it to feel natural. No not open world like Breath of the Wild but maybe something like XBCX2 where it's still linear but have enough space in it where Pokémon can act and behave like Pokémon. Imagine a group of Mankey's Being highly territorial over a tree or a flock of Spearal's flying about looking for lunch.
I couldn't stand the way it was done in Let's Go and wouldn't mind if Game Freak kept it exclusive to that series as I won't support those games again.
Again, it's a opinion. I already had some people get offended by my opinion like it had a chance to be reality in this thread.