or that damn light gets brighter.
No. CPU cache has more to do with feeding instructions to your CPU. If they are using less cache, its because they believe their system doesn't need as much cache to run games as you will typically find in a PC whose CPUs generally are shuffling way more workloads than a console would. It could also indicate that their OS is lean and less robust compared to whatever MS uses. An argument can be made that your SSD performance is tied to your CPU and how quickly it can call the drive, but I doubt reducing the cache size will impact that kinda performance.
NVMe SSDs have their own RAM, the type of RAM they use can impact their speed. To what degree I am not sure.