I mean... I can't say I did a lot of wandering around in FFXV and story is super important to me. And the game is in my top 3 FFs. (I really don't care about sidequests though and skip them in most games.) I genuinely liked the story a lot, and I've chatted on other sites with people who did too. So I wouldn't assume that everyone who enjoys the game just doesn't care about story/lore/etc. It's cool that you feel a certain way about the game but I don't think it's a good idea to assume other people did too and they just didn't value those things in the game.
I mean, it IS lacking. There's a reason the game is getting free story patches. It's because the original presentation was unsatisfactory. There are plenty of aspects of the story that are good and worth discussing, but you can't pretend that there isn't huge chunks of world lore left unexplained and characters that are just dropped. Major characters that only appear once or twice. Vital events that happen off screen and are just briefly told about. Side quests in most FF also really help flesh out the world lore and build character development too, so I can't see how you'd usually skip them if story is really all that important to you.