The X-Men movies that failed, failed because they didn't focus on the X-Men as an actual team. Look at all the mainline X-Men movies that people generally hate (X3, Apocalypse, Dark Phoenix, sometimes First Class) and you'll see that those movies have a focus on one or two characters. X3 was Wolverine/Jean focused, Dark Phoenix was all about Jean, First Class was all about Magneto/Prof X, and Apocalypse was just all over the place.
Now, if you look at the other ones that did great, like DoFP, then you'll see there's more of a broad focus on mutants in general and their combined struggle. It's plainly just better writing.
Marvel has gotten this down to a science, so hopefully they can do a better job bringing the mutants to the big screen.