Incline RPG Codex's Top 50 cRPGs - Results and Reviews

[CryptRat]

The thing is, a game rated as a 9 by two persons isn't the same as a game rated as a 9 by 50 persons.

That's why the bayesian average is useful here.

 

[DarkUnderlord]

Theoretically, that means that someone with less game played has more weight in his top games than someone who has a longer list of games. Of course, the latter could just as well distribute points to only a few of his favorites, but nonetheless, there's intrinsic tendency for the system to regard the preferences of those with fewer entries as more intense.

True. But then you also get spammers who vote for everything - even stuff they haven't played.

Mind you it'd mean re-coding the voting system I did last year and that's unlikely to happen soo...

All I need at this stage is a list of games.

 

[Kev Inkline]

Theoretically, that means that someone with less game played has more weight in his top games than someone who has a longer list of games. Of course, the latter could just as well distribute points to only a few of his favorites, but nonetheless, there's intrinsic tendency for the system to regard the preferences of those with fewer entries as more intense.

True. But then you also get spammers who vote for everything - even stuff they haven't played.

Mind you it'd mean re-coding the voting system I did last year and that's unlikely to happen soo...

All I need at this stage is a list of games.

Fair enough.

But do you mean that including an extra field for entering the rank of the respective game would mean a total recode effort as well?

 

[DarkUnderlord]

Kev Inkline: Adding another field wouldn't be a huge issue if required. Just that re-coding it so people have a limited number of votes (EG: spreading 25 points across all games) would be a slightly more involved effort (which is to say, do-able but I don't know my time availability over the next few weeks.).

 

[Kev Inkline]

Kev Inkline: Adding another field wouldn't be a huge issue if required. Just that re-coding it so people have a limited number of votes (EG: spreading 25 points across all games) would be a slightly more involved effort (which is to say, do-able but I don't know my time availability over the next few weeks.).

Of course, spreading a budget of 25 points over games already gives the ranking implicitly, so the extra field would in that case redundant. But if you'd have that likert scale instead, the extra field would yield additional information.

 

[Jason Liang]

Why not do both? Have each Codexer rate each game they've played both on a 1-5 scale, AND have them assign each game 0-5 points out of 25 total. I don't think that would be overwhelming information overload.

In fact, I would like a 3rd score as well, which is rating the new games alongside the old games (using the 25 points total method). I think the current top 50 list is a deserved list of the best computer RPGs ever made, but it clearly needs to be reranked. I've been playing BG: EE, and, while it should be on the list, it clearly shouldn't be ranked in the top 20 or anywhere near BG 2. The original Baldur's Gate is a good RPG, but it clearly is nowhere near the same level as a great RPG like Ultima IV or V, and it's ridiculous that we are still ranking it above those.

Similarly, I'm pretty sure the consensus now is that Ultima V should be ranked ahead of IV and VII.

And also, to be honest it's ridiculous to rate all-time RPGs on a scale of 1-5... I think the cap for any single game should be 4. So it should be rating the games 0 min, 4 max. What I mean is that when I try making a list of the best RPGs of all time and can only assign 25 points, I can't even assign a single 5. I have a 4, a 3 , three 2's and the rest are 1's- which means I am only rating 17 games total. In fact, a cap of 3 would probably make even more sense. In the 2014 poll, only 8 games in the top 50 had an average rating higher than 3.

25 points, cap of 3 max per game, minimum 15 games rated would be the most fair. Or 30 points, 20 games min.

Or how about scaling points by games rated?

20 points, 10 games
25 points, 15 games
30 points, 20 games
35 points, 25 games

etc... This way players who have played more games wont be weighed less than those who have played less games and you can collect information from players who have played more games.

Or actually give the players who have played more games some more weight, like 6 points/ 5 games feels right.

19 points, 10 games (roughly 2 ***, 5 **, 3 * or 3 ***, 3 **, 4 *)
25 points, 15 games
31 points, 20 games
37 points, 25 games
43 points, 30 games

etc...

... doing this poll feels just like min/maxing for Baldur's Gate or NWN...

 

[Kev Inkline]

Why not do both? Have each Codexer rate each game they've played both on a 1-5 scale, AND have them assign each game 0-5 points out of 25 total. I don't think that would be overwhelming information overload.

I think this would certainly be one fruitful way to carry this out. The assigned points implicitly yield the rankings, with which one can do all sort of funny things.

 

[felipepepe]

Ok, how about this: we do the voting with the 1-5 scale, one page for each year, then the last page is a text box where people sort out their 25 points.

That way we get both methods, DU doesn't have do perform great l33t hacking feats and we can play data analysis to our hearts content.

 

[Kev Inkline]

Ok, how about this: we do the voting with the 1-5 scale, one page for each year, then the last page is a text box where people sort out their 25 points.

That way we get both methods, DU doesn't have do perform great l33t hacking feats and we can play data analysis to our hearts content.

Sounds very good! However, I don't know if you would like to restrict to grading 1-5 to only those games you give at least one point out of 25? Well, if we don't restrict that, it's one way of testing voters' consistency .

 

[Crooked Bee]

EDIT 2: Here's the WIP list of games: https://docs.google.com/spreadsheets/d/1Onb7VZP-AfPZ1v4OfcptZgc3TSZqjf9G4xhjO51vANU/edit?usp=sharing

2013: Labyrinth of Touhou 2

 

[Hirato]

EDIT 2: Here's the WIP list of games: https://docs.google.com/spreadsheets/d/1Onb7VZP-AfPZ1v4OfcptZgc3TSZqjf9G4xhjO51vANU/edit?usp=sharing

2012:
Drox Operative

2014:
Abyss Odyssey

2016:
Final Fantasy X/X-2 HD
God Eater 2
Hyperdevotion Noire
Hyperdimension Neptunia U: Action Unleashed
Megadimension Neptunia VII
Princess maker 2 Refine

 

[Goral]

voting with the 1-5 scale

Can't it be with 1-10 scale? Otherwise There will be too many 3s. Or if 1-5 then allow +0.5 in-between.

 

[Haraldur]

voting with the 1-5 scale

Can't it be with 1-10 scale? Otherwise There will be too many 3s. Or if 1-5 then allow +0.5 in-between.

The trouble with a 1-10 scale is that most people underuse the bottom half, so that it is essentially a 6-10 (or 7-10) scale plus 1/10. A 1-5 scale accepts this reality in a more fitting way. Also, most games probably are 3s.

 

[Goral]

The trouble with a 1-10 scale is that most people underuse the bottom half

Maybe because unless you're a masochist you play a game of at least average quality (hence 3s are most common). In the case of 1s, these are usually games you couldn't finish or didn't even bother to play (like Fallout 3 or 4). 1-10 will at least let us better differentiate between an average game and above average game. When I see an average RPG which isn't that bad but isn't that good either I would rate it 3 instead of 2 but with 10 point scale it would be 5.

 

[Hirato]

may as well go full Ebert and make it at 1-4 scale if we're going to change it from 1-5.

 

[Hirato]

EDIT 2: Here's the WIP list of games: https://docs.google.com/spreadsheets/d/1Onb7VZP-AfPZ1v4OfcptZgc3TSZqjf9G4xhjO51vANU/edit?usp=sharing

2016:
Hyperdevotion Noire

You missed one.
Here's the steam page: http://store.steampowered.com/app/415480/

 

[Kev Inkline]

voting with the 1-5 scale

Can't it be with 1-10 scale? Otherwise There will be too many 3s. Or if 1-5 then allow +0.5 in-between.

The trouble with a 1-10 scale is that most people underuse the bottom half, so that it is essentially a 6-10 (or 7-10) scale plus 1/10. A 1-5 scale accepts this reality in a more fitting way. Also, most games probably are 3s.

It's an ordinal scale anyways, so it really doesn't matter if he bottom half is under represented.

 

[Sigourn]

Scales make sense only when one is expected to use all values.

For example, a scale of 1 to 10 makes sense when rating a game. I personally think it would be nice if everyone could rate the games they have played, as this is only a handful of years (as opposed to over 30 years of RPG history), using a scale from 1 to 5, 1 meaning "shit" and 5 meaning "brilliant".

 

[Sigourn]

I think we must not try to do too much with one system. Simply aggregating ordered preferences (w/missing data) is already a huge challenge. If you want to account for magnitude of preference ("I really, really, really loved my #1 much more than my #2") I think it's taking on too much at once.

What if we just went with "recommend" or "not recommend"?

I think, at the end of the day, all games can be classified like that.

 

[Jazz_]

Dark Souls is too high on that list.

 

[makiavelli747]

"recommend" or "not recommend"?

To whom?

 

[Sigourn]

"recommend" or "not recommend"?

To whom?

To the Codex.

I find it's just easier to say "yeah, I recommend this game" or "no, I don't recommend it". So people post the games they recommend, and the ones they don't.

 

[Turjan]

Meh, I think a 1 to 5 scale is completely sufficient. You can give DA:I a 5 ("riding the bull never looked that good") and NEO Scavenger a 1 ("that's not why I bought my new graphics card"), and everything else fits nicely in between (PoE 3 "gorgeous backgrounds, but why can't I move my camera?"). That's basically all you need.

 

[Sigourn]

Meh, I think a 1 to 5 scale is completely sufficient. You can give DA:I a 5 ("riding the bull never looked that good") and NEO Scavenger a 1 ("that's not why I bought my new graphics card"), and everything else fits nicely in between (PoE 3 "gorgeous backgrounds, but why can't I move my camera?"). That's basically all you need.

Of course, that's what I think is best.

I mean, the objectively accurate way to have rankings like these is to rate every single game one has played, but it's easy to say when we only give out votes. It becomes a nightmare for the people in charge of voting them.

 

[felipepepe]

And here... we... go: http://www.rpgcodex.net/article.php?id=10502