GOTY 2018 Wrap-Up & Future Planning

SweetNicole

The Old Guard
Member
Oct 24, 2017
5,541
Hello community,

Thank you to everyone for another thrilling Games of the Year event here at ResetEra! Now that the votes have been counted and the results published, we on the GOTY team would like to take the opportunity to talk about what went well and what could go better next time. With this now being the second year that the team has handled this event, we've identified several recurring issues that I am going to highlight and address in this thread.

The Issues

Vote Formatting & Spelling
Although we've added tools to alleviate this issue, such as VoteBot and volunteers who alert voters to incorrect ballots, many posters are still formatting their votes incorrectly. Format mistakes make things harder for everyone involved! On a similar note, we've seen a lot of typos and spelling issues such as forgetting a colon in a game's name, leaving a letter off the end, forgetting a "the" at the start of a game name, and so on. This can result in votes not counting, and it creates additional work for vote tabulation to ensure that we've consolidated the votes for every misspelling and permutation of a game's name. This is something we want to try and reduce as much as possible going forward.

Remasters/Remakes
Remakes and remasters have been a common point of confusion. We received a lot of questions why a game such as Hollow Knight and Shadow of the Colossus was not eligible for Game of the Year voting while Let's Go, for example, was. While we've tried to be clear about what criteria is used to define the remake/remaster category, the results are still generating a lot of confusion.

Categories
There have been some issues with how categories are handled for GOTY voting. Because GOTY voting is a single ballot for Game of the Year, there is no individual voting for specific categories such as best RPG and such. Instead, we use meta tags to handle category placement and results. For example, God of War had the following meta tags: Playstation 4 and Action-Adventure. This means that the game was only eligible for those categories, and the votes for the game in those categories are the same as the votes for the overall GOTY list.

This can create strange scenarios where a game, for example, has a bad port on one system but still receives a high result for that platform's results because the game was fine on another system. Another strange scenario: In 2018 voting, Assassin's Creed Odyssey placed on the Best Switch Game list because of the release of the game in Japan which resulted in the meta tag for Switch being added to the game. It is clear team that meta tags for category placement are not producing satisfactory results.

Proposed Changes

With the issues outlined above in mind, we're looking at making the following changes to the GOTY voting process for 2019:

Category Nomination Round
The first major change would be changing the way the voting process works by adding an additional round of voting. Currently, voting is done in a single round with GOTY + categories done at the same time using the same list. For 2019, we'd like to add a round of voting before the GOTY voting which would likely be called "Category Nominations."

Category Nominations would be a round of voting where voters nominate games for consideration for certain categories (excluding GOTY). Ideally, voters would be able to nominate 3 games for each category. Once Category Nomination is over, the top 5 results from each category would be the nominees for that category and eligible for voting in the final round.

The final round of voting would look much like the current system with a general Top Ten GOTY list for the GOTY category. In addition, voters would be able to vote for one game for each category. The end result would be the standard top twenty list and category results that are not based on meta tags.

Here's a simplified breakdown of it:

Category Nomination round
  • Nominate up to three games for each category
  • Nominations are un-ordered (order does not matter, each nomination is one point)
Normal round
  • Top 5 from each category are eligible for voting for that category in the final round (choose one)
  • Standard GOTY vote like previous years
We believe this will allow for more accurate categories as category winners will not simply be determined off of the GOTY vote list. It will hopefully give opportunity for other games to shine and help drive discussion about aspects of games that haven't been covered in previous years. As part of this round voting, we would look at expanding the categories currently offered. Because meta tags wouldn't be used for determining category eligibility, there could be categories introduced that we weren't previously able to offer such as Best Story or Best Music.

Below are some of the categories that the team would like to see added. Do note that this list is no where near finalized. This is simply a rough draft of categories that we could include if we changed to a two round voting system.
  • All Existing Platform Categories
  • All Existing Genre Categories
  • Best Narrative/Story
  • Best Voice Acting
  • Best Graphics
  • Best Ongoing Game
  • Best Artistic Direction
  • Best Music
  • Best Indie Game
  • Best Sound Design
  • Best Level Design
  • Game We Can't Stop Playing
  • Most Fun with Friends
  • Most "Badass" Game
  • Cutest Game
  • Craziest Game
  • Best Competitive Multiplayer
  • Best Guilty Pleasure Game
  • Best Character Designs/Best Characters
  • Best Niche Game/Best Underrated
  • Best Experimental Game
Vote Formatting
In order to address formatting issues, we're looking at tech solutions to streamline the voting process. Rest assured, we know that people like to the format and create their own lists to look back on later, so even if there is a tech solution found, we would not remove the GOTY voting thread. Instead, the GOTY voting thread might simply become a thread for you to show others how you voted, formatting your list however you want.

Remake/Remasters Eligibility
Going forward, we'd like to either clarify the rules regarding what is considered a remake/remaster and what isn't. Resident Evil 2, for instance, is a substantial overhaul from the original and thus should be eligible for GOTY. This was the case with Pokemon Let’s Go. Shadow of the Colossus, however, was simply a graphical upgrade, and in theory should not be eligible. That said, we could always have a “Best Remake” category. Simply put, if a game remake/remaster is MORE than just a graphics overhaul, then it would be eligible for GOTY.

The alternate approach could be that, if voters feel that the category system ensures remakes/remasters will not overwhelm the system, that such games are eligible for GOTY and categories. Another alternative could be that the games are eligible for GOTY and not categories or even categories and not GOTY. There's a variety of different options and approaches we could take, and it largely depends on what you all want.

Conclusion

We are seeking your feedback regarding the proposed changes to the 2019 GOTY process. While we cannot respond to every post in this thread, we'll try to respond where and when we can. Our intention is to foster this discussion in the community and collect feedback. Either late this month or early next month, we will provide a breakdown of the received feedback and finalize what changes, if any, we'll be making to 2019 GOTY voting. Thanks for your continued support, and we look forward to your thoughts!
 

FarisR

Member
Oct 25, 2017
1,308
How about not saying a game is eligible for GOTY in the list and then changing your mind about it after the votes come in. Thanks.
 

Lump

Member
Oct 25, 2017
4,290
I’m not sure if this is the right place for this, but I think Era should have Resettis as awards for Era members the way Michael Scott had the Dundies for his employees.
 

Hecht

Ain't nothing but marking time
Administrator
Oct 24, 2017
4,529
How about not saying a game is eligible for GOTY in the list and then changing your mind about it after the votes come in. Thanks.
Hey, thank you for your feedback! As you can see in the OP, we're planning on making some changes that should alleviate any problems. Thanks again!
 

FarisR

Member
Oct 25, 2017
1,308
Yakuza Kiwami 2 was a substantial overhaul from the original and should have been eligible for GOTY, tbh
It actually was according to their official eligible voting list. They then changed their mind about it without any notice after the votes had completed.

I had big issues with that (and I didn't even vote for it)
 

Arttemis

The Fallen
Oct 28, 2017
2,527
Help with formatting or better tech to discern the differences is definitely my biggest point of contention. I googled half a dozen game titles to correctly list them and was still notified that my submissions had to be edited.
 

Bigkrev

Member
Oct 25, 2017
4,173
I think you guys REALLY fucked up with the "What is a remake/remaster" stuff.
The only stuff that should be excluded are clear "HD Remasters" the likes of which we saw a ton of early in the generation- Last of Us PS4, Dishonored, God of War 3, etc. Switch ports, like Skyrim, Doom, Saints Row 3 etc should also be excluded. Otherwise, it's all eligible. It's fucking absurd that new games like Shadow of the Colossus, Yakuza Kiwami 2, etc were excluded from last years awards.
 

Aeana

Member
Oct 25, 2017
3,577
Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
 

lt519

Member
Oct 25, 2017
3,019
In concept I agree with the categories but I think adding another round of voting will drive participation down. Getting participation up should be the #1 priority by streamlining the voting process and allowing the thread to be a showcase and place of discussion (as you've outlined). As it is now I think a lot of users enter the thread, see a wall of text and rules about formatting, and just say F it.

I didn't look this year, did we get the metrics back that showed which other members voted like you? That was always the best part.
 

Hecht

Ain't nothing but marking time
Administrator
Oct 24, 2017
4,529
Yakuza Kiwami 2 was a substantial overhaul from the original and should have been eligible for GOTY, tbh
It actually was according to their official eligible voting list. They then changed their mind about it without any notice after the votes had completed.

I had big issues with that (and I didn't even vote for it)
Let's just nip this in the bud. We're aware that there were issues with how some games were categorized in the 2018 GOTY thread. That's literally the point of this thread - we're trying to fix it going forward. Let's not derail this thread with how "X was robbed" in 2018. The goal is to make sure that doesn't happen in the future.
 

Mezoly

Member
Oct 25, 2017
2,724
Seconding the remakes point!

I voted for Hollow Knight and Shadow of the Colossus after being confirmed multiple times in the voting thread including the op that they are eligible. Only to find out they were not eligible and my vote was wasted.
 

Hecht

Ain't nothing but marking time
Administrator
Oct 24, 2017
4,529
In concept I agree with the categories but I think adding another round of voting will drive participation down. Getting participation up should be the #1 priority by streamlining the voting process and allowing the thread to be a showcase and place of discussion (as you've outlined). As it is now I think a lot of users enter the thread, see a wall of text and rules about formatting, and just say F it.
We have a few ideas on how to streamline the voting process, so hopefully that part will not be an issue. We're just trying to determine if it's feasible.
 

Necromanti

Member
Oct 25, 2017
2,638
Is there some sort of giant game database out there that might have something like an ID attached to each game that could be used by people instead?

There's probably no easy solution to some of the issues without being able to have something like a form that could standardize inputs/check for errors.

Anyway, it's a lot of work.
 
Nov 3, 2017
3,327
I feel like badass/cute/guilty pleasure etc. shouldn't really be things, they're all quite subjective (especially that last one) but it's not really important

Other than that I think it'd be cool to have categories along with the general vote

I’m not sure if this is the right place for this, but I think Era should have Resettis as awards for Era members the way Michael Scott had the Dundies for his employees.
Only if it comes with the death inducing cringe
 

FiveSide

Member
Oct 25, 2017
2,667
Regarding remasters/remakes. The truth of the matter is that, like the RPG Essentials thread, voters self-select their own standards for fulfilling some arbitrary requirement (in that case, "is the game an RPG?"; in this case, "is the game a remaster/remake that should be excluded?"). I think it would be best to stop the Sisyphean task of determining what remakes/remasters are or aren't eligible and just leave it up to the voters' crowdsourced wisdom to decide. A lot of people, including me, wouldn't vote for a remaster for GOTY, but in a scenario like RE2 or Let's Go where I feel there was a meaningful and distinctive remake, I would definitely include them. If more people feel the same then we'll end up with results that accurately reflect the community's thoughts and standards rather than an arbitrary blanket rule that can struggle when it comes to certain games.

In short: I think all remasters/remakes should be eligible and the community will just decide organically based on trends in the voting.
 

Arcade55

Member
Oct 27, 2017
1,444
I've always wondered why we don't have a setup where you can just click to vote. I understand people like the ballot. But the amount of screwed up posts is astonishing. It seems like 4 out of 5 posts have people not formatting correctly and the amount of time people spend fixing all of it is probably staggering.

Then again, maybe having a separate page for voting is just too much work or to costly to setup. I honestly know nothing about web design stuff.
 

Yossarian

The Fallen
Oct 25, 2017
5,540
Wouldn’t the simplest solution be just to say any game released that year is eligible, regardless of format, remake status and such?
 

Bigkrev

Member
Oct 25, 2017
4,173
Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
I would much rather have a "if it came out somewhere this year it's eligible" system where if someone wants to vote for Famicom Fire Emblem 1 because it came to Switch online, let them, then to have some weird judgement system where an unknown group arbitrates over if something is "new" enough
 

Zen Hero

Member
Oct 25, 2017
2,173
I think we can simplify the rules around ports, remakes, remasters, and the like.

Please just let them all count. Let the actual votes decide whether it's significant enough to win GOTY.

This is the approach that the RPG voting thread took on the difficult question of defining "RPG", and it worked well there. Let the users define it through their votes, no need for specific rules.

Anything released this year in any platform should be eligible.
 

kenta

Member
Oct 25, 2017
576
Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
This has been my understanding of how eligibility works, and I'm not sure why it ever changed. "Was this playable thing released in [calendar year]? If yes, it's eligible."

I don't know why genre categories need to be a thing either -- lines between genres are getting blurrier with each passing year so I'm not sure there's a need to swim upstream and try to delineate one genre from another. My proposal would be to nix that entirely

Also with regard to "cute" categories, those are extremely subjective and you're opening up a whole can of worms with it (see Giant Bomb's 2017 "best world" award)
 

rpm

The Fallen
Oct 25, 2017
2,294
the 24th century
Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
This. I absolutely disagree that SotC 2018 is a "remaster" while RE2 and Let's Go are "remakes". SotC is a remake, just like RE2 and Let's Go.
If I think Blaster Master via NSO is the best game of 2019, let me be alone in voting for it

Also, yeah, formatting. Implementing a system on Era itself would probably be cumbersome, but I'd like to see a ballot generator website that will auto suggest the correct names/not let you write incorrectly formatted names and can spit out a properly formatted ballot to paste in your post.
 
Last edited:

McNum

Member
Oct 26, 2017
1,163
Denmark
I find that simplicity is key when you need a lot of people to input something manually and it needs to be formatted right and all. I wonder if it's possible to have the forum software put a premade draft in the reply space for that thread with all the correct formatting and all the user has to do is replace the game title and add a description. Anything that can be done to make it easier to type the list the better.

As for reamkes/remasters, I'd say keep it very simple for when a game counts: If a game releases for the first time on an eligible system this year, it's eligible. Every new system release is someone's first time playing a game, after all. Yes that would mean Startropics would be facing off against Devil May Cry 5, but, to be frank, so what? It's not like an old NES game released on Switch is going to fight off DMC5, but they might be good for list fillers if you're short a few.

I'm not sure if I like the two round system, seems cumbersome, but there does need to be something done with multiplatform games that may or may not have a much better/worse version on some system. But I am hesistant to add an input for system as that just increases the chance of people getting it wrong.
 
OP
OP
SweetNicole

SweetNicole

The Old Guard
Member
Oct 24, 2017
5,541
I didn't look this year, did we get the metrics back that showed which other members voted like you? That was always the best part.
We have some metrics in the GOTY thread here, although I don't think any of them are what you are looking for.

Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
If that's what people want, we're all for it.
 

Weebos

Member
Oct 25, 2017
3,125
I like the proposed changes for the Category Nominations. I think that will lead to better Category lists.

On the Remake/Remaster discussion, I think we could just drop the distinction and leave all of them in the open voting pool. I think trying to distinguish between remakes and remasters is a lot of unnecessary work that ultimately doesn't impact too much.
 

Unknownlight

Member
Nov 2, 2017
4,040
Regarding remasters/remakes. The truth of the matter is that, like the RPG Essentials thread, voters self-select their own standards for fulfilling some arbitrary requirement (in that case, "is the game an RPG?"; in this case, "is the game a remaster/remake that should be excluded?"). I think it would be best to stop the Sisyphean task of determining what remakes/remasters are or aren't eligible and just leave it up to the voters' crowdsourced wisdom to decide. A lot of people, including me, wouldn't vote for a remaster for GOTY, but in a scenario like RE2 or Let's Go where I feel there was a meaningful and distinctive remake, I would definitely include them. If more people feel the same then we'll end up with results that accurately reflect the community's thoughts and standards rather than an arbitrary blanket rule that can struggle when it comes to certain games.

In short: I think all remasters/remakes should be eligible and the community will just decide organically based on trends in the voting.
I would much rather have a "if it came out somewhere this year it's eligible" system where if someone wants to vote for Famicom Fire Emblem 1 because it came to Switch online, let them, then to have some weird judgement system where an unknown group arbitrates over if something is "new" enough
Agreed with all of this. The number of people who would seriously vote for Famicom Fire Emblem 1 for GotY is so low that it's not worth making up a rule to exclude it. And if a whole bunch of people vote for something, then that probably means it should be included.
 

Vanillalite

The Fallen
Oct 25, 2017
3,403
Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
How would you handle Hollow Knight then?

Personally I think it was a 2017 title, but I can sorta see all sides on this.
 

Version 3.0

Member
Oct 27, 2017
2,089
On the topic of eligibility: I would think no rule is needed. If people are voting for a title, it's not for no reason. As long as it was released - on any platform - that year, count it. If you disqualify something popular enough to place, you're only making people angry.

No vote suppression! This is Era! :-p
 

Unknownlight

Member
Nov 2, 2017
4,040
How would you handle Hollow Knight then?

Personally I think it was a 2017 title, but I can sorta see all sides on this.
Hollow Knight should be eligible for 2018. It's when console-only players first got to play it.

If I were making my own list I wouldn't include it, but we need simple and consistent rules for voting threads.
 

More_Badass

Member
Oct 25, 2017
15,862
Personally, I feel like "if it released in X year, then it's eligible for that year" should be enough. Considering the notion of recency and how most people are probably going to put the newer and new games higher than a remaster or remake (outside of huge rarities like RE2 and SoTC), I just don't think the extra concern over remakes and remasters is warranted.

If a remake is so exceptional that it overshadows the new games of that year and captured the attention of the community, that should be reflected in GOTY instead of worrying about arbitrary case-by-case basis of what qualifies as a big overhaul or not, what is a remake or remaster, and so on

Basically this
I would much rather have a "if it came out somewhere this year it's eligible" system where if someone wants to vote for Famicom Fire Emblem 1 because it came to Switch online, let them, then to have some weird judgement system where an unknown group arbitrates over if something is "new" enough
Agreed with all of this. The number of people who would seriously vote for Famicom Fire Emblem 1 for GotY is so low that it's not worth making up a rule to exclude it. And if a whole bunch of people vote for something, then that probably means it should be included.
On the topic of eligibility: I would think no rule is needed. If people are voting for a title, it's not for no reason. As long as it was released - on any platform - that year, count it. If you disqualify something popular enough to place, you're only making people angry.
 

molnizzle

Avenger
Oct 25, 2017
8,025
My only thing is I think the "require everyone to write a throwaway sentence for a vote to count" is ridiculous.

The entire point here is to get the aggregate GOTY from ResetEra members. Not just ResetEra members who want to write a little paragraph. It's silly. The data is all that matters.
 
Oct 25, 2017
4,357
Canada
I guess my suggestion would be to allow folks to vote without making them write up their paragraphs for each game. I assume a sizeable chunk of folks are put-off by having to write a decent amount of text, but may end up coming back later to edit in their reasoning for each game after they've first engaged by making the initial post / vote.
That may help to drive up engagement.

Plus yeah, rules & formatting being simpler and all that would help too
 

spineduke

Member
Oct 25, 2017
3,715
Wouldn't this category nomination add a ton of work for the team? You're effectively running two vote cycles
 

Aeana

Member
Oct 25, 2017
3,577
How would you handle Hollow Knight then?

Personally I think it was a 2017 title, but I can sorta see all sides on this.
Hollow Knight would be eligible in any year it released in. There's no need to do anything more than that. The community self-selects by voting for it, and someone who played the game in 2017 is probably not going to re-vote for it again. And if they do, so what? This is how it worked at the old place, and it wasn't really an issue.
 

Kadin

Member
Oct 25, 2017
2,926
US, California
What is the downside to letting any game released in a year be eligible for a vote? Is the thinking that people who are simply fans of older, rereleased games will ruin the chance for an actual 'new' title to compete?
 

Nocturnowl

Member
Oct 25, 2017
10,635
I do like the idea of some of those nomination categories like best art direction, though I wouldn't go for too many of them as that would lead to more work on all sides.
And also leave soundtrack out of that equation since we give it its own thread (that people totally need to vote in more!)

Part of me also thinks that the kerfuffle surrounding genre and platform awards that happens each year isn't worth it, I'd consider dropping them.
 

Hecht

Ain't nothing but marking time
Administrator
Oct 24, 2017
4,529
Wouldn't this category nomination add a ton of work for the team? You're effectively running two vote cycles
The idea is that simplifying the voting format (e.g., with a form, ideally) wouldn't really add much work outside of the end results being posted. The tabulation should all be relatively simple to handle.
 

Version 3.0

Member
Oct 27, 2017
2,089
A tech solution to formatting issues is welcome. Too much of the voting thread was people helping others fix their posts.

I personally thought the categories were fine. I suspect participation in a pre-vote category vote would be quite low. And it also suspect that it wouldn't stop post-voting dissatisfaction with category placement, either. Leave it. People like to argue over the results, anyway. It's just part of the process, and all in good fun. If the results were complaint-proof (which is impossible, but still), you'd only rob everyone of the pleasure of pointing out how they think something is wrong.
 

Vanillalite

The Fallen
Oct 25, 2017
3,403
Hollow Knight should be eligible for 2018. It's when console-only players first got to play it.

If I were making my own list I wouldn't include it, but we need simple and consistent rules for voting threads.
This logic basically means if you're a PC only title you're fucked for ever winning the GOTY because what matters is what console players play as they outnumber PC folks.
 

Unknownlight

Member
Nov 2, 2017
4,040
This logic basically means if you're a PC only title you're fucked for ever winning the GOTY because what matters is what console players play as they outnumber PC folks.
I dont understand why making Hollow Knight eligible for two years makes PC-only titles fucked. Console players outnumber PC players, yeah, but that was already true. Nothing's changed.
 

Weebos

Member
Oct 25, 2017
3,125
This logic basically means if you're a PC only title you're fucked for ever winning the GOTY because what matters is what console players play as they outnumber PC folks.
That's just generally true of ERA though. If the majority of the user base doesn't play PC games, then they won't get voted for.
 

Cipher Peon

Member
Oct 25, 2017
1,983
My only thing is I think the "require everyone to write a throwaway sentence for a vote to count" is ridiculous.

The entire point here is to get the aggregate GOTY from ResetEra members. Not just ResetEra members who want to write a little paragraph. It's silly. The data is all that matters.
I agree with this.

Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
I disagree with this.

Not a fan of counting remasters or ports. Even remakes are pushing it. But if people disagree that's fine by me!
 

Vanillalite

The Fallen
Oct 25, 2017
3,403
That's just generally true of ERA though. If the majority of the user base doesn't play PC games, then they won't get voted for.
No disagreement, but we probably should say what the overall GOTY is. Just make it console GOTY or something. Then if we wanted we could have a PC GOTY and even a mobile GOTY (or handheld if that came back).
 

HanzSnubSnub

Member
Oct 27, 2017
168
Why does there need to be so much lawyering about whether something is enough of an upgrade to count or not? Why can't something that released in the given period be eligible, period?
Adding my vote for this. If a remaster earns enough votes to even enter the GOTY top 20, then it must be an incredible remaster.
 

Parsnip

Member
Oct 25, 2017
3,048
Finland
You should poll the platform the user is voting a particular game on. Seems like it would provide some interesting data points on multiplats and actually make platform specific lists useful and not total shit.
 

Unknownlight

Member
Nov 2, 2017
4,040
No disagreement, but we probably should say what the overall GOTY is. Just make it console GOTY or something. Then if we wanted we could have a PC GOTY and even a mobile GOTY (or handheld if that came back).
Isn't this what the categories are for? We already have a "Best PC game" category, and the proposal in the OP should help prevent bad PC ports from being counted.