The biggest reason to upgrade is Dolby Vision. The remote overhaul was needed as well, but lack of DV was what was holding Shield TV back in 2019 - especially as Apple TV released a software upgrade to allow real controllers to be paired with it and thus allowing streaming apps like Steam Link to achieve their full potential.
The performance boost being modest isn't a big deal - Shield Experience 8.0 garbage performance notwithstanding (giving the benefit of the doubt that the latest software playback issues are due to bugs rather than being too much for the hardware). Native Android TV games just haven't gained much traction, so there's not much to play natively on the device. This is a box that is great at UI speed, 4K playback smoothness (again, the latest software release hitches notwithstanding), and streaming games either locally or via GeForce Now. It's also the best Kodi box on the market.
Apple TV and Shield TV are really neck and neck now. If you want to use Kodi to maximum effect and prefer Gamestream over Steam Link, Shield TV is the better option now. If you want to use Apple Arcade and have a slightly more cohesive software experience in general, Apple TV is the better option. But that's really getting into the grain of it - both boxes cover all the necessities now, which is what's important.