-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
ResetEra Games of the Year Awards 2017
- Thread starter SweetNicole
- Start date
- Status
I saw some people asking for weighted votes and tagging me so, since the poll creators kindly made their full data available, here's the Top 20 on a Bayesian Average, like we do at the RPG Codex:
Anyway, kudos to the poll organizers, I know well how much work these polls demand. o7
- The Legend of Zelda: Breath of the Wild 3,10
- Super Mario Odyssey 2,62
- Nier: Automata 2,58
- Persona 5 2,53
- Horizon: Zero Dawn 2,50
- Divinity: Original Sin II 2,33
- Yakuza 0 2,31
- Hollow Knight 2,27
- Playerunknown's Battlegrounds 2,19
- Xenoblade Chronicles 2 2,18
- Prey 2,08
- The Legend of Heroes: Trails in the Sky The 3rd 2,06
- Assassins Creed: Origins 2,02
- Nioh 1,99
- Resident Evil VII 1,96
- Night in the woods 1,94
- Cuphead 1,93
- XCOM 2: War of the Chosen 1,93
- What Remains of Edith Finch 1,92
- Danganronpa v3: Killing Harmony 1,92
Anyway, kudos to the poll organizers, I know well how much work these polls demand. o7
Last edited:
I disagree, if the game is less played that means that it fails to pull public beyond its niche and that for me means it doesn't deserve to be rewarded as high as games that are are able to pull more public.
Like for example your list totally dismiss Splatoon 2.
Last edited:
Thanks for the data.
I couldn't disagree with this more if I tried.
Not everyone has access to every platform or hears about every game. If a niche is extremely passionate about a game, maybe it deserves a second look from others as well - more than a game that a lot of people played but felt lukewarn about it.
Splatoon 2 is #12 on the original poll, but only 4 of its 347 voters rated it as the #1 game of the year. Meanwhile 37 of the 175 people who voted for Hollow Knight ranked it #1. That means 21% of people who played it absolutely loved it, while in Splatoon 2 that number is 1%.
my problem with this method is that using a bayesian average on a relative ranking like you're doing makes no sense. What if someone loves every game 1-5 on their list near equally, but thinks number 6 is much lower? The difference between 1 and 2 on a list could be anything because it's a relative ranking of favorites, not an absolute score. Hell, one person's number one game could be an 8/10 for them if they didn't play many games they loved that year, while another might give 1-4 all 10/10s.
I think we are arguing a deeper question here. Why do we do GotY voting?
To promote underappreciated games? If that's the case we should just ban Zelda and Mario from the vote, because those games will sweep floor no matter how you change the way you vote and they are in no way underappreciated.
Or are we celebrating and rewarding the games we enjoy the most of the year? In that case the current system is good enough. People don't feel "lukewarm" towards Splatoon 2, they care enough to put it in their top 10. The vote shows more people enjoyed Splatoon 2 this year than Hollow Knight, it's simple as that.
Popular voting by default is a bad way to promote hidden gems. And trying to represent hidden gem in popular voting is self-contradictory. If you really want to recommend a game, you should make a thread about it instead of burying it among a thousand other votes. That's why media GotY lists usually do the job of digging hidden gems better than community votes.
Personally I have a lot of other sources for hidden gems: there gotta be a hundred media GotY lists out there, and Hollow Knight is well represented in many of them. But REF GotY vote is special to me because as a part of this community, I want to know the general consensus, the most comprehensive representation of our own taste in videogame. I want to know what games most people in this community enjoy, not what a small niche is passionate about.
Agreed, that's why I use a 1-4 scale at the Codex. But how's what I done any different from the Era's current poll? I'm just working with the data I have.
Lol, I am sorry but what? How you say that, the vote is ranking based not score based, you can't say only 1% absolutely loved splatoon.
I think I agree with your opinion even more than the one I gave before myself.
Point is, you can have both. Zelda and Mario still won when using weighted votes.
And I disagree: more people PLAYED Splatoon 2 than they played Hollow Knight - 347 vs 175 - but clearly Hollow Knight left a stronger impression. This is quite easy to compare:
Look at how many people ranked Hollow Knight as their #1 or #2, while few did that with Splatoon 2. It was a good game, but definetly not the best they played.
You are totally ignoring that people can play a game a not put into in their top 10. This argument makes no sense.Point is, you can have both. Zelda and Mario still won when using weighted votes.
And I disagree: more people PLAYED Splatoon 2 than they played Hollow Knight - 347 vs 175 - but clearly Hollow Knight left a stronger impression. This is quite easy to compare:
Look at how many people ranked Hollow Knight as their #1 or #2, while few did that with Splatoon 2. It was a good game, but definetly not the best they played.
Point is, you can have both. Zelda and Mario still won when using weighted votes.
And I disagree: more people PLAYED Splatoon 2 than they played Hollow Knight - 347 vs 175 - but clearly Hollow Knight left a stronger impression. This is quite easy to compare:
Look at how many people ranked Hollow Knight as their #1 or #2, while few did that with Splatoon 2. It was a good game, but definetly not the best they played.
The information that more people played Splatoon 2 than Hollow Knight should be represented in the final result. So what if Hollow Knight is more loved by a niche? It clearly has lower accessibility. I do agree that the gap in total points should be lower if REF isn't so skewed towards console game. But hey, this is exactly what I want: a faithful representation of the community. Our community is skewed towards console and our GotY list should reflect that.
There are a thousand of them if you haven't noticed. You'd better off read Polygon's top 50 games of the year.
BTW I believe the Polygon list has games that none of us vote for. Further proof that popular voting by default is bad for digging hidden gems.
Final thing I'll do then. Here's a visual representation of Era's vote:
Draw your own conclusions.
Draw your own conclusions.
Again we're left with the problem that people can play games and not vote for them in this system. I played a bunch of games I didn't vote for because they wouldn't get points regardless and I didn't feel like bothering with honorable mentionsFinal thing I'll do then. Here's a visual representation of Era's vote:
Draw your own conclusions.
Again, you can't equal voting as playing.Final thing I'll do then. Here's a visual representation of Era's vote:
Draw your own conclusions.
So what? If you didn't list it among your best 10 games of the year, then clearly you don't consider it all that great.
If you just want to find which games are the most played, go after sales numbers or something like that.
If you just want to find which games are the most played, go after sales numbers or something like that.
There were a bunch of games I played this year that I really enjoyed but didn't feel earned a vote in my top 10. Hell, in one case Stardew Valley only fell outside my top 10 because despite qualifying and this being my first time playing it, it didn't feel right to give points to a game that was a 100% straight port
Again, if it's not even among the 10 ten games you played this year, why the hell should it matter in a GOTY voting?
Final thing I'll do then. Here's a visual representation of Era's vote:
Draw your own conclusions.
This is pretty interesting.
Thanks for sharing it.
I'm not saying it should, I'm saying that equating the number of people who voted for a game to the number of people who played it and also voted is fundamentally flawed regardless.
You are missing the point. You are using data that iis meant for a ranking system and using it like a scoring system.
This is all semantics if you ask him. The data is useful since it gives you an idea of how much people who voted for a game care enough to show how much they enjoyed certain titles. In the old site they had a category for that "Game with more number 1 votes" or something like that. In the end, this is just a tool. I really don't understand why people (I don't mean you btw) have so much against certain games or certain types of games, that they'd do anything to try to downvote/punish a game in a thread about celebrating gaming.
I mean, I'd argue what he's doing is no different. Honestly, there are games I absolutely adore that would benefit from his voting system (Trails in the Sky the third being the prime example in that it jumps to the top 20 from below the top 100 in this system). At the end of the day, what he's arguing is essentially to put more weight onto the placement in the list rather than the number of votes. But it's not like the system we're using didn't already weight a number 1 vote differently to a number 10 vote, so it's basically saying my way of weighting is more correct because it gets the games I personally prefer higher on the list.
It is the same data tho, just with a different presentation.
Actually, "you can't equal voting as playing" is not what I'm arguing for.
I think popular voting, no matter the format, always comes down to the games that most people played. Divinity won GotY on RPGCodex? No shit. I bet everyone on that forum played it.
Look at IMDB top 250. It uses very sophisticated algorithm to take account for both popularity and score, yet the list is just 250 good and famous movies. I've heard about all the films on that list and have watched most of them. Compare it to a list form Sight & Sound and the difference is obvious.
My point is, what games people play the most is exactly what I want to know from our own GotY list. Apparently we don't have a lot of PC gamers here, and we prefer single player games, especially Japanese ones. And somehow XB2 is extremely big on our forum. Persona 5 lower than it should be shows the backlash is real. That's more valuable information to me than "Hollow Knight is great". I know Hollow Knight is great because we've got multiple threads on this forum about Hollow Knight. You know what is also great? A Mortician's Tale. None of us, or people at RPGCodex, voted for it. I heard it from Polygon.
I think popular voting, no matter the format, always comes down to the games that most people played. Divinity won GotY on RPGCodex? No shit. I bet everyone on that forum played it.
Look at IMDB top 250. It uses very sophisticated algorithm to take account for both popularity and score, yet the list is just 250 good and famous movies. I've heard about all the films on that list and have watched most of them. Compare it to a list form Sight & Sound and the difference is obvious.
My point is, what games people play the most is exactly what I want to know from our own GotY list. Apparently we don't have a lot of PC gamers here, and we prefer single player games, especially Japanese ones. And somehow XB2 is extremely big on our forum. Persona 5 lower than it should be shows the backlash is real. That's more valuable information to me than "Hollow Knight is great". I know Hollow Knight is great because we've got multiple threads on this forum about Hollow Knight. You know what is also great? A Mortician's Tale. None of us, or people at RPGCodex, voted for it. I heard it from Polygon.
That's not an argument, it is a mistake that I pointed in the graph.
66% of people over there played Divinity: Original Sin 2. It was the most played game. Second most played was Torment: Tides of Numenera which would have placed at third in a straight point tally despite most people over there feeling negative about it.
Also nobody voted for A Mortician's Tale over there because it wasn't an RPG.
Final thing I'll do then. Here's a visual representation of Era's vote:
Draw your own conclusions.
Yeah, I really don't like this scheme of collating votes as done by Era. It basically ignores how highly people rank a game and becomes more of a count of how many times a game was mentioned. The final list is more or less a vertical ordering of this graph, whereas IMO an ideal ordering would be more diagonal. It's a really simplistic voting scheme when there are more mathematically sound ways of doing it. Hats off to the team for all their hard work for collecting and tabulating the data, obviously. But the statistician in me is not very happy with the results, regardless of how I feel about the outcome.
Yes, and it's useful to see it that way, that's all I'm saying here.
I showed to you with data in this thread that more people has played NieR than Persona 5, the other 3 games are a given too. You keep pushing this narrative despite the fact that there's really nothing that supports this claim, other than a few comments of some users very vocal about their opinion of this game.
I mean, if you are very disappointed by a game, you just don't vote it right? What data reflects that? The discrepancy between the most anticipated game vote and the GotY vote is pretty obvious.
How does this work?
Is it a detriment in this counting to put a game low on a list? If it is, it is unfaithful to the data. I wouldn't put those games up on my ballot if it were a detriment to their tally; that is not at all why they are there: they are there because I think they are among the best games in the year, not because I think people are wrong to rate them higher still (i.e. is the point total calculated some weighted average?). I'd just suggest shorter ballots to achieve a similar effect.
Does it instead just weigh the upper positions at much much greater than their current point total? If it is that, it comes across as asking that our ballots just be made shorter. That would be detrimental especially to games that were on a lot of ballots but often not very high, which seems like the category trying to be excised here. Where was the "only first place votes" count? I think that was somewhat different. NieR was above Mario, for example.
(Notably shorter ballots probably mean less games mentioned overall--but not sure that matters as many just get "buried" anyway).
Is it a detriment in this counting to put a game low on a list? If it is, it is unfaithful to the data. I wouldn't put those games up on my ballot if it were a detriment to their tally; that is not at all why they are there: they are there because I think they are among the best games in the year, not because I think people are wrong to rate them higher still (i.e. is the point total calculated some weighted average?). I'd just suggest shorter ballots to achieve a similar effect.
Does it instead just weigh the upper positions at much much greater than their current point total? If it is that, it comes across as asking that our ballots just be made shorter. That would be detrimental especially to games that were on a lot of ballots but often not very high, which seems like the category trying to be excised here. Where was the "only first place votes" count? I think that was somewhat different. NieR was above Mario, for example.
(Notably shorter ballots probably mean less games mentioned overall--but not sure that matters as many just get "buried" anyway).
Last edited:
Final thing I'll do then. Here's a visual representation of Era's vote:
Draw your own conclusions.
Just a question. Do the calculations means that if I put a game on the number 10 of my list, it will skew the percentage of the game down? I'm asking this because if this is the case, I will vote for the game I don't like that might win GotY.
For example, 30 BotW fans put Divinity on number 10. How will it reflect on this graph?
It is not.
You realize that the same thing can be said about the other games right? There's no data that shows whether people are disappointed by one of the games in the top 5 or not. Is it really so difficult to accept that the placement of these games in Era's GOTY is due to popularity and sales? You can say all you want that one game was punished by Resetera's community, but all games received criticism and backlash in the old site and in here: there are threads asking when Horizon, BotW, NieR, etc "get good", there's a huge amount of backlash because of the outfit of 2B, threads dismissing Horizon as a GOTY contender, etc. and yet all those games ranked very high here.
Xenoblade 2 had tons of threads about its characters designs, yet the game made it into the list. If I applied the logic you followed to dismiss Persona 5, I could easily say that XB2 was punished by the community, but the truth is that the game is pretty good but it was released in December, and its audience is way smaller than the one of the other games in the list. It's the same story that happened with Uncharted 4: There were so many threads and comments dismissing the game as a GOTY contender, you could read how many people were disappointed by it and so on, and yet it was the GOTY 2017 and many acted surprised by the results. What's the difference between Uncharted and Persona 5? One sells like pancakes the other not as much.
Your argument is not solid at all.
As far as I can tell, is the list too bloated with mega-releases that aren't at the top of many lists but are on many lists or not? How do we deal with this if it is the case?
With a subcurrent of "is it okay that our method only allows for positive votes"?
Personally, as I think it works, the method being discussed creates negative votes in order to prioritizes votes at the top of ballots. It doesn't fit the numbers we have, because we do not have negative votes in our system. It is meant to work for RPG Codex where you can give something a low score.
I don't think negative votes are a productive thing personally, and I think a better answer to the same problem (accepting it as such for conversation) is just to limit the ballot lengths further, i.e. get rid of low point counts that can really build up for this category of "shows up a lot as it was played a lot, but not many people's favorite," which seems to be the category people currently have a problem with.
The thing is, a lot of people play a lot of games and want to recognize a lot of games, and so people generally want the option for longer ballots, I'd think. But it really seems to me that the length-10 ballot is what is causing the fuss atm.
With a subcurrent of "is it okay that our method only allows for positive votes"?
Personally, as I think it works, the method being discussed creates negative votes in order to prioritizes votes at the top of ballots. It doesn't fit the numbers we have, because we do not have negative votes in our system. It is meant to work for RPG Codex where you can give something a low score.
I don't think negative votes are a productive thing personally, and I think a better answer to the same problem (accepting it as such for conversation) is just to limit the ballot lengths further, i.e. get rid of low point counts that can really build up for this category of "shows up a lot as it was played a lot, but not many people's favorite," which seems to be the category people currently have a problem with.
The thing is, a lot of people play a lot of games and want to recognize a lot of games, and so people generally want the option for longer ballots, I'd think. But it really seems to me that the length-10 ballot is what is causing the fuss atm.
Why the heck do we need to dig for hidden gems in the first place? Since when has GotY been about being a commercial about what games you feel need to be advertised?
Seriously does it bother you that much that you game isn't higher on the list? And I'm not being biased as my votes are either in the same place or higher using this method.
Last edited:
Yeah actually both systems seem to be using ordinal data as interval data? I think the weighted mean system is a bit better than the Bayesian system, but the latter does give that nice image.
Cool graph.Final thing I'll do then. Here's a visual representation of Era's vote:
Draw your own conclusions.
That big cluster of PS4 games makes a lot of sense logically. For those who only had a PS4 so were voting only within that platform, those three big PS4 games essentially competed with each other for #1 of that demographic, and also those three could be split as second choices by Zelda/Mario voters, which would drag their percentages down.
What may be interesting is seeing sort of the inverse: what the #2 and #3 games of each of the top ranked games voters were. Eg, what were Zelda fans #2/3 games? I'm guessing based on pure platform grounds people who voted the PS4 games #1 wouldn't have Zelda or Mario as #2 very often, for example, and what percentage of Zelda #1 voters would have one of those three as their #2/3 - given they would have to own both platforms for that.
Very late response, but I just wanna emphasize how right this is. I don't think any actual publications awarded this game the title it deserved.
Best Sports Game: Pyre = 204 points, 3 honorable mentions
You can't use methods designed for scoring systems (i.e., a ratio scale, more or less) on a ranking system (i.e., an ordinal scale). If you do, I honestly have to question how much the person doing this understands about statistics.
The first problem is that it's not even a full list, just the best ten games. Just a small example: If half of the people who played a game really adore a game and half of it hate it, said game fares better in the graphs shown above than a game that is liked by most people, because those who hate a game simply won't mention it in their GotY lists. For this kind of method to work, you would at least need another category telling you how many people actually played a game, but didn't like it enough to place it on their lists.
Even then it would still be wrong to do it, because you simply cannot use Bayesian averages on an ordinal scale. That's statistics 101. The reason why has already been mentioned: You can't say anything about the difference between two ranks, which is quite important for a Bayesian average to work. Another example: If BotW and SMO are really close for me, I can both score them 4/4 on RPG Codex' system, but I have to make a choice here, even if it's just a small difference. If somebody else only really liked Horizon this year and would score it 4/4, but everything else 2/4 at best, he still gets another game that is "as close" in a ranking system as BotW and SMO are for me.
So while these graphs might be nice to look at, they really don't tell you anything.
The first problem is that it's not even a full list, just the best ten games. Just a small example: If half of the people who played a game really adore a game and half of it hate it, said game fares better in the graphs shown above than a game that is liked by most people, because those who hate a game simply won't mention it in their GotY lists. For this kind of method to work, you would at least need another category telling you how many people actually played a game, but didn't like it enough to place it on their lists.
Even then it would still be wrong to do it, because you simply cannot use Bayesian averages on an ordinal scale. That's statistics 101. The reason why has already been mentioned: You can't say anything about the difference between two ranks, which is quite important for a Bayesian average to work. Another example: If BotW and SMO are really close for me, I can both score them 4/4 on RPG Codex' system, but I have to make a choice here, even if it's just a small difference. If somebody else only really liked Horizon this year and would score it 4/4, but everything else 2/4 at best, he still gets another game that is "as close" in a ranking system as BotW and SMO are for me.
So while these graphs might be nice to look at, they really don't tell you anything.
An amazing community effort - well done everyone!
Splatoon 2 as best shooter rather than PUBG though...hmmm. Not a lot of PUBG love on these forums :-)
Splatoon 2 as best shooter rather than PUBG though...hmmm. Not a lot of PUBG love on these forums :-)
the individual genre stuff isn't really indicative of anything because they were all taken from the main goty rankings. so it's not that Splatoon 2 is the best shooter necessarily, it's that Splatoon 2 is the best game that just so happens to be a shooter. no one actually voted on a shooter category.
Splatoon 2 is the best shooter ever made tho ;p
Same.
People still upset their niche games didn't rank higher in a popularity contest I guess
They had an embarrassing 2017 while everyone else had a great year. They only published like 4 games all year. Halo Wars 2, Forza, Super Lucky's Tale and Cuphead. The latter was on steam as a self published game and it sold the most on there so most don't even consider it a xbox title.
- Status