Yeah, I completely agree. Steam Deck is really great in a lot of ways, and probably the closest thing you can get to a "Console" style experience with PC gaming, I feel. But there are definitely a lot of things that can be improved. Shader pre-caching should be an option you can toggle in Game Mode, and really nothing related to the Deck's basic functions should require going into Desktop.
Also, while playing Docked, I didn't have any issues with audio, but I did with video quality. It looked absolutely horrific out of the box, with dull colors and crushed blacks. It turns out that the Deck basically just can't tell your display that it's displaying Full RGB, so if you have RGB settings set to Auto (which is what virtually all TV's will default to), you'll have to go into your TV settings to change it. Many TV's don't label these settings as RGB either (on my old set, it was Standard and Nonstandard). I had to go Googling because I'd never run into anything like this before. Even a basic $300 laptop I bought from Wal-Mart awhile back didn't have this issue. Steam Deck is really a fantastic platform overall, but there are definitely a few bizarre oversights and issues.