RPGDatabase.net

[Jahmis]

Some of you may have seen my other web project in the general RPG forum, CRPGDB.com. Well, that one met an untimely demise after being ambushed by a roving band of giant rats while rooting through the cellar for something to drink.

My new project is that one's younger, leaner brother out for revenge. Really, I was just being too ambitious before and felt like I was adding a bunch of stuff without a clear goal in mind. With RPGDatabase.net I'm focusing only on the database part and leaving behind any kind of user login stuff or personal game tracking. And I've moved to the workshop forum since this seems like a more appropriate place to post updates and whatnot.

So have a look. It's still early and development is going to be slow because you know, life. But I'll keep adding to it as I have time. If you have any suggestions or find any bugs or inaccuracies, let me know.

 

[Excidium II]

I like how there is an "explosion" tag.

Cool project, will be cooler when you allow users to contribute with entries/review and entries have their own page

 

[Stormcrowfleet]

I find it very amazing. Some of the choice are quite strange tho. You have futuristic, sci fi and what not, but no ''cyberpunk'' tag. But overall I find it very nice and useful ! Keep up the good work.

 

[Jahmis]

Thanks. I'll be working on individual pages for each game soon.

 

[WhiteGuts]

Why are there gay looking jRPGs

 

[CryptRat]

It will probably be awesome, but where do you take the tags from? It's sometimes illogical, and occasionally inaccurate.
For example :
- "Dungeon Crawler" includes the first Buck Rogers game and not the second one, while "Dungeon Crawling" includes none of the two.
- "Tactical Turn Based" combat doesn't include any of the two Buck Rogers games.
- "Female protagonists" includes a few RPGs were you can make your own character, all of them or none of them would be better.
- "Branching Story Line" includes Secret of mana.
- "Horror" doesn't include Koudelka nor the Shadow Hearts series, and the choice of included or not included Megami Tensei is weird (for example the second Raidou game is in the list but the first one is not in the list).

Random thoughts :
- Is there a way to look at the tags associated with a given game?
- Searching for game by name is probably useful too.

 

[Immortal]

What constitutes an RPG.

I did Fantasy / Steam Tags with sorting based on Rating. The first two entries were Binding of Isaac and Castle Crashers.

Are these really the two best RPG's offered on steam with a "Fantasy" Setting?

Other then that - Cool idea. Also, I couldn't find a filter for date of release ranges.

 

[Jahmis]

Thanks for all the feedback, except for the "gay jRPGs" comment. To answer some questions, the tags come from the various sites I get the data from. I added an "About" page that lists them all. I'd say most come from Steam because of all the user added tags. Some games have 50 or more. I'm relying on existing data and just aggregating it all in one place. I haven't done any hand editing at all, but will one day add the option for anyone to edit the database directly.

Same with the ratings. I use a normalized average of the ratings for each game on those other sites. If you're looking for the best games, I'd suggest also filtering by platform to get rid of mobile games or ones you don't really care about.

About what constitutes an RPG, I'm again relying on the categories used by the different sites. I'm actually trying to be conservative. I could easily double the number of games in the database because sites like Steam and Giant Bomb seem to be pretty liberal in what they consider an RPG. And I only add a game if it shows up in more than one site's data. I'm going to be working on that over time. I also need to deal with expansions, which I don't list yet.

I'll be adding individual game pages soon so you'll be able to see the tags for each. And I also think searching by name is needed. I'll add that right after probably.

So, overall this is still early days. I know the database needs work and cleaning up. But I'm going to be working on this for a long time, so keep letting me know your thoughts.

 

[Jahmis]

I have an update. Games have their own pages now. Just click on the box art image to bring it up.

 

[Immortal]

Thanks for all the feedback, except for the "gay jRPGs" comment. To answer some questions, the tags come from the various sites I get the data from. I added an "About" page that lists them all. I'd say most come from Steam because of all the user added tags. Some games have 50 or more. I'm relying on existing data and just aggregating it all in one place. I haven't done any hand editing at all, but will one day add the option for anyone to edit the database directly.

Same with the ratings. I use a normalized average of the ratings for each game on those other sites. If you're looking for the best games, I'd suggest also filtering by platform to get rid of mobile games or ones you don't really care about.

About what constitutes an RPG, I'm again relying on the categories used by the different sites. I'm actually trying to be conservative. I could easily double the number of games in the database because sites like Steam and Giant Bomb seem to be pretty liberal in what they consider an RPG. And I only add a game if it shows up in more than one site's data. I'm going to be working on that over time. I also need to deal with expansions, which I don't list yet.

I'll be adding individual game pages soon so you'll be able to see the tags for each. And I also think searching by name is needed. I'll add that right after probably.

So, overall this is still early days. I know the database needs work and cleaning up. But I'm going to be working on this for a long time, so keep letting me know your thoughts.

Yea I figure as much but again there has to be something wrong with your curl (or whatever your using). If I want to see Steam RPG's with a Fantasy Setting sorted by user Rating.. How the fuck is Castle Crashers my first index?

Like If I go to steam with that same criteria, pretty sure I see Witcher 3 or Skyrim. (Depending when you pulled this data)

EDIT:
It's a cool project and you could possibly open this up more with a REST API (or something) of some kinda to allow other web services to query your site for information that you are in theory aggregating from different online stores.. It just needs to work first..

 

[Jahmis]

If you're questioning the ratings, there's nothing wrong with the scraper (and yeah, I'm using curl). Crashers is just a really highly rated game. Check it out for yourself:
Castle Crashers: Overwhelmingly Positive
Witcher 3: Very Positive
Skyrim: Very Positive

Crashers wins. The system works.

If you don't agree that it's a better game you'll have to take it up with all the Steam users that rated it. I don't have any control over that. To me, the more useful question to ask is "Why is it even showing up in this database?". That's something I have control over. It's not really an RPG in the sense of all the "What is an RPG?" threads you can find scattered around. The reason it shows up here is because it's considered an RPG by two different source sites, which is my current criteria. Unfortunately, the two sites it shows up in are Steam and Giant Bomb which are the two with the most lax definition of RPG. As long as it has some vague RPG elements it gets put in that bucket.

So I have a couple options. One is to increase the minimum number of source sites a game is listed in, but that might remove some legit games, especially older ones that might only be in My Abandonware and another one. So I'd rather wait until I've added one or two more sources first. The other option is to remove games where the only two sources are Steam and Giant Bomb. That's easier for the short term so I'll do that first. It'll be interesting to see what games get kicked out. My main goal is to keep the games in the database focused on "real" RPGs.

About an API, maybe. I'm trying to see how far I can take this without actually spending money. I've had to buy the url, but that's about it so far. Opening up an API means bandwidth, means money. If the site gets super popular I'll have to deal with that issue anyway, but that's definitely not something I need to worry about just yet.

 

[Jahmis]

Oh, and I've added a way to search by name, for anyone that was wanting that. I'll going to improve the search logic in a bit, but it works for now.

 

[Jahmis]

I culled out about 50 non-rpgs, including Castle Crashers. Now I'm going to work on a semi-automated system to look for new games so that the database stays current.

 

[Immortal]

If you're questioning the ratings, there's nothing wrong with the scraper (and yeah, I'm using curl). Crashers is just a really highly rated game. Check it out for yourself:
Castle Crashers: Overwhelmingly Positive
Witcher 3: Very Positive
Skyrim: Very Positive

Crashers wins. The system works.

If you don't agree that it's a better game you'll have to take it up with all the Steam users that rated it. I don't have any control over that. To me, the more useful question to ask is "Why is it even showing up in this database?". That's something I have control over. It's not really an RPG in the sense of all the "What is an RPG?" threads you can find scattered around. The reason it shows up here is because it's considered an RPG by two different source sites, which is my current criteria. Unfortunately, the two sites it shows up in are Steam and Giant Bomb which are the two with the most lax definition of RPG. As long as it has some vague RPG elements it gets put in that bucket.

Just my 2 cents.

I want to preface that I agree with and understand your post - Your not wrong.. I understand the logistics of curling sites for meaningful data points and trying to represent that data in a meaningful way that makes sense. I just want to point out that with 0 curation or quality control on your part - you are gonna run into these weird scenarios.

Again despite what Steam says - If I search for a Fantasy RPG and get Castle Crashers, I'm gonna look elsewhere. Your saying the system works and maybe if your goal is simply to be a info dump of steam your right... However ~~~ Wouldn't it be maybe advantageous to add additional filters or data sources? (as you say later down below)..

Is there any other site you can poll from like maybe Amazon to see what games are actually listed as an RPG genre - Steam is pretty bad for this.

Examples Of Steam Failures:

• Although Castle Crashers rating is apparently higher then Skyrim, shouldn't one take into account the user pool voting?
  • 22 000 vs 140 000..
    
  • (I have a hunch it's mostly Newgrounds Fanboys Inflating the Rating)
  • Not to say Skyrim is an amazing game either but it's definately more universally loved especially in the realm of RPG''s.. The fact that Steam metrics are so prone to user bias kind of skews the reality of how good or bad these games are in relation to each other.
• Even though Castle Crashers has a "tag" of rpg - couldn't you maybe validate that in some other way.
  • There are games tagged with the genre "Womans Rights" "Buggy Game" etc
  • Again - When tagging systems are subject to a angry mob of opinionated people who feel slighted and can train wreck a game - it loses a lot of meaning. (Granted it's pretty fucking funny to watch)

I understand this is a pet project but Steam is basically a junk yard of games with occasional good sales.. it kind of falls to you on filtering and siffing through the data in a meaningful way or picking a better source of truth.

Granted - This is definitely harder then it sounds.

So I have a couple options.

• One is to increase the minimum number of source sites a game is listed in, but that might remove some legit games, especially older ones that might only be in My Abandonware and another one. So I'd rather wait until I've added one or two more sources first.
  
• The other option is to remove games where the only two sources are Steam and Giant Bomb. That's easier for the short term so I'll do that first. It'll be interesting to see what games get kicked out. My main goal is to keep the games in the database focused on "real" RPGs.

These are really great starts. I think it will really benefit your data pool.



EDIT:

I culled out about 50 non-rpgs, including Castle Crashers. Now I'm going to work on a semi-automated system to look for new games so that the database stays current.

Missed this post - Sounds cool.

 

[Jahmis]

Totally valid points. I do want to make clear though is that I'm not just relying on Steam. I have five sources I'm drawing on right now: Steam, Giant Bomb, Metacritic, Moby Games, and My Abandonware. And I plan on adding more over time, probably GOG and others. I would love to find one or two sites that I could rely on as a definitive source of "RPGness", but they all seem to have their share of junk entries in varying degrees. So my fallback plan is to try to find some king of consensus between them all and only include games in my database that show up in at two of the source's lists.

I admit that it's still no substitute for hand curation, but I'm only one dude and don't have time to go through the thousands of candidates one by one. So I'm trying to leverage the power of programming to find some set of smart filtering rules I can apply to all this data, which I find is a really interesting problem to try and solve. At some point I want to open it up for others to be able to add and remove games, but that adds a big level of complexity to the site that I'm ready to dive into yet.

This my long-winded way of saying it'll get better over time, I promise. But thanks for keeping me on my toes, and keep letting me know when you find problems or if you have any ideas on how to make it better.

 

[gaussgunner]

Nice job, bro. Just add a way to *exclude* tags (Adventure, lol) and it'll be a better discovery method than Steam. Could be a great way for niche developers to get noticed, maybe without even having to be on a distribution platform at all.

More ideas..
- sort A-Z, Rating, Release date (all ascending or descending)
- more results per page
- search by developer/publisher
- exclude words i.e. "-Spiderweb -Bioware"
- tastefulrape tag

 

[Severian Silk]

Judging by the tag names I'm guessing you're just copying info from Wikipedia?

 

[Taluntain]

Jahmis I think this would work nicely as part of the Codex if you want us to integrate it eventually (i.e. I like it and I expect DarkUnderlord would too). Until then, or if DU doesn't want it for some reason, I can give you our server resources at your disposal if you're not averse to sticking some ad code up.

 

[DarkUnderlord]

Yes, the Codex has longed for a game / company / people database.

I like the idea of an automated algorithm that just pulls data off of other sites. Maybe even matched with games we've covered. We can certainly link to it from the Codex.

 

[Jahmis]

Nice job, bro. Just add a way to *exclude* tags (Adventure, lol) and it'll be a better discovery method than Steam. Could be a great way for niche developers to get noticed, maybe without even having to be on a distribution platform at all.

More ideas..
- sort A-Z, Rating, Release date (all ascending or descending)
- more results per page
- search by developer/publisher
- exclude words i.e. "-Spiderweb -Bioware"

More ways to filter and sort the data that shows up is definitely on the To Do list. You can already sort A-Z and by rating though, they're the two mysterious icons on the right and they toggle between ascending and descending. I keep forgetting to add tooltips.

Judging by the tag names I'm guessing you're just copying info from Wikipedia?

Nope. See the About page, or various posts in this thread to see which sources I'm using for my data.

Jahmis I think this would work nicely as part of the Codex if you want us to integrate it eventually (i.e. I like it and I expect DarkUnderlord would too). Until then, or if DU doesn't want it for some reason, I can give you our server resources at your disposal if you're not averse to sticking some ad code up.

Thanks for the server offer. I have no problem with adding ads, but it complicates things because of the way I'm getting the data for the site. Most, if not all, of these other sites wouldn't like to know that mine is making money from their data. I knew going in to this that server space and how to pay for it was going to be an issue I'd have to solve at some point. Right now I'm coasting along with Heroku's free plan, but if this blows up I'm not sure what I'll do. My current plan of action is to contact the other sites and see what their stance is. Maybe since I'm providing links back to them it's OK.

Yes, the Codex has longed for a game / company / people database.
I like the idea of an automated algorithm that just pulls data off of other sites. Maybe even matched with games we've covered. We can certainly link to it from the Codex.

A link would be appreciated. I worry a bit about the server issue. I wouldn't want to have the Codex tidal wave come crashing down and smother it before I figure out what to do about that. But what the hell, go for it. Let's see what happens.

And you gave me an idea. I can add a Codex tag to the games you've covered. Where's the best place to get that list?

 

[Jahmis]

I'm adding this as an extra post so that it doesn't get lost in all the replies above. I've updated the site again with some new entries (not necessarily new games). And I've pruned out some more that shouldn't be in there, mostly DLC packs and a few that didn't have complete information. Now that I have a system in place to keep it up to date, I'll be running it every week.

 

[Taluntain]

I could (temporarily) host it for free if the ads would prove to be a problem for the reason you've stated. Though if you couldn't make any money whatsover off it that would significantly limit its usefulness as well, because providing links to buy the games listed from various gaming stores would be a logical and helpful feature to include. The data sources definitely are problematic if they mean that you'd essentially have to run it all at your expense while promoting other sites in the process.

 

[DarkUnderlord]

And you gave me an idea. I can add a Codex tag to the games you've covered. Where's the best place to get that list?

I can make something. Unless you're happy to pull them all out of http://rpgcodex.net/tags.php

I would love those tags to link to your database though, on the Codex. At the moment there's no info and we've struggled with how we'd get info.

Let me know what you need though and I can basically format it to what works best.

Re ads and shit, I'm not sure there's much illegal or wrong worth making an aggregating site like this. Their content is just dates and product names and wikipedia lets you use its content anyway.

 

[Jahmis]

Ok, it's been a while but I have an update. If you go to the site now you'll see a new tag available, "RPG Codex". Clicking on that will list all the games in the database that are also listed in the Codex tags pages that DarkUnderlord linked to above. And each of those games will have a link back to that games' tag page. It sounds more complicated than it is, just check it out and you'll see what I mean.

In other news, I've added a new section to the bottom of each game page that shows some recommendations for other similar games. It's just a first pass and could probably still use some tweaking. Let me know what you think.

I could (temporarily) host it for free if the ads would prove to be a problem for the reason you've stated. Though if you couldn't make any money whatsover off it that would significantly limit its usefulness as well, because providing links to buy the games listed from various gaming stores would be a logical and helpful feature to include. The data sources definitely are problematic if they mean that you'd essentially have to run it all at your expense while promoting other sites in the process.

I'm good on the server front for now. Monetizing the site will probably come at some point, but I'm taking it pretty slow because I'm still learning a lot of this stuff and I don't have much free time to put into it.

I can make something. Unless you're happy to pull them all out of http://rpgcodex.net/tags.php

I would love those tags to link to your database though, on the Codex. At the moment there's no info and we've struggled with how we'd get info.

Let me know what you need though and I can basically format it to what works best.

Re ads and shit, I'm not sure there's much illegal or wrong worth making an aggregating site like this. Their content is just dates and product names and wikipedia lets you use its content anyway.

See above.

And I agree that there's nothing illegal in what I'm doing, especially as long as I'm not making money off it. But the main issue I see with using other site's content isn't the dates and names, it's the descriptions. Each site puts work into making them unique (most of the time).

I'm not a legal expert, but I would at least be pissed if I found something I wrote on someone else's site and they were making money off of it through advertising. Maybe it's not as big of a deal if they cited where they got it and had a link back to me, but it would still be weird.

But maybe I'm being overly cautious. Anyway, about linking to the database from the Codex, what did you have in mind? I could give you a way of getting the game ids, which you would then use to generate the url for each game. Let me know.

 

[Immortal]

And I agree that there's nothing illegal in what I'm doing, especially as long as I'm not making money off it. But the main issue I see with using other site's content isn't the dates and names, it's the descriptions. Each site puts work into making them unique (most of the time).

I'm not a legal expert, but I would at least be pissed if I found something I wrote on someone else's site and they were making money off of it through advertising. Maybe it's not as big of a deal if they cited where they got it and had a link back to me, but it would still be weird.

But maybe I'm being overly cautious. Anyway, about linking to the database from the Codex, what did you have in mind? I could give you a way of getting the game ids, which you would then use to generate the url for each game. Let me know.

If you source the origin of that information with a link back to them, it just increases their page rank and potential clicks.
It's free advertising, it would be ridiculous for them to bitch about it.

Also not a legal expert but I would be thrilled to have every article of my site having a page rank boost from an external site. As long as they were curling my site responsibility and not DoSing me.