A jaded Luke who has come to believe the galaxy would be better off without him is way more interesting that Super Saiyan Luke. Luke didn't just run off out of fear. He ran off because he caused all this, and he legit thought that the galaxy would be better off without him. But, in the end, he does live up to his legend. TLJ succeeds because it shows you Luke, the Jedi Master who saved the galaxy, but also shows you Luke, the emotional, flawed human who made a huge mistake.
I wonder if Rian Johnson was inspired by True Grit at all. In that movie, John Wayne's Rooster Cogburn is a US Marshal with a reputation for "True Grit" aka being a badass Marshal who always gets his man. He turns out to be a drunk, cynical asshole. When push comes to shove at the end, though, Rooster takes on four men at once to save the girl who hired him, and shows that his reputation doesn't come from nowhere. Luke's arc in TLJ reminded me a lot of that film.
To me, TLJ was a movie of high highs and low lows. Canto Bight felt like something out of the prequels. But Luke's arc, along with his taking the Jedi as an organization to task (which has been long overdue), are so good that it makes the movie as a whole rival Empire as my favorite Star Wars movie.
For all the talk of how Hamill didn't like TLJ (which I don't think is particularly true), he gave easily the strongest performance of the film. The dude put in insane work.