The funny thing is, to truly defeat this argument, all you have to do is cite a few works that they don't like for whatever reason.
Before this thread popped up earlier today, I had a conversation about this on discord, and I pointed out that people who defend sexual objectification and exploitation use the same "IT'S MY ARTISTIC VISION, DON'T CENSOR". To which the person I was arguing with said that wasn't the same thing as real artistic visions like Sekiro's difficulty being used to instill feelings of accomplishment. So then I pointed out that this argument is also technically valid if a racist bigot wanted to create a dehumanizing depiction of minorities, since he would also have a vision he would be trying to instill into an audience, but then that also didn't count because that person is malicious, and apparently malicious people are somehow incapable of artistic visions. So I decided to use an innocuous examples of Jar Jar Binks, who an usual failure of artistic intent, as that character actually did what he was designed to do (be entertaining to little kids). But we all know the story of Jar Jar's critical reception, even though he was an explicit part of Lucas' vision and even succeed in it's intended role.
There are endless examples are nearly universally panned games, movies, books or individual design decisions that are clearly stupid and bad and should not have been made. Tommy Wiseau's the Room. Kojima and his idiotic "breathes through her skin" narrative. The shitty, toxic garbage like Rapelay on Steam. But you don't even need to go to these extremes. What is the Last Jedi if not a meticulously crafted Auteur project, yet there is no shortage of people who rebel at the entire movie because they disagree with it's artistic vision. And yet, even as frustrating and tiresome as discussions around the Last Jedi have become, no one that I can recall disputed Rian Johnson's artistic vision in terms of him being allowed to make it, as an artist, because the debate is never about whether Rian Johnson should be allowed to write star wars how he wants, just that the way he did write it was questionable in their eyes, then then they bring the reasons why they feel that way. As crappy as the discussion around TLJ, it's atleast consistently about the quality of the writing decisions that were made, not the artistic freedom to make them.
Honestly, if scrutinized across all art, I think you'd find people backing off the "artistic vision" worship pretty fast. Artistic vision is only a valid defense when they like the particular art being discussed. It crumbles pretty fast when you bring in either Jar Jar or TLJ into the mix (depending what kind of fan they are). One you do, you'll generally get them to admit "Yeah, X was a really poor decision to make, they should have instead-" and whoops, suddenly artistic visions don't count for as much anymore.