steam250

3 months, 3 targets

Published: October 3rd 2020, 3:07:53 pm

Last month may have looked quiet, but suffice it to say, I was working all month to accomplish what I set out to do: improve Club 250's back end to have consistent daily updates like Steam 250. You might recall from last month's post that updates were taking 4.5 hours manually, now they take about 2 hours running automatically. I also managed to squeeze in application monitoring at the very end, so if anything goes wrong I'll be alerted so it can be fixed.

Here's a breakdown of our new automatic update process:

1. Download Steam app list 2min
2. Download Steam app list again (secondary source) 1.5min
3.  Scrape app details for every app 75min
4. Compute first review date for new apps with reviews 1min
5. Compute review totals with interpolation and normalization passes 45min
6. Iteratively compute top 1000 ranking for all games 30sec
7. Iteratively compute hidden gems rankings for all games 30sec

You might notice we now have a new step for computing hidden gems, and that data is now available on the Club 250 app details pages! Check out the blue line in the screenshot. The page also had a minor facelift and tags are now clickable, too.

Looking forward

With three months remaining of 2020 and three clear goals in mind, here's how I see the rest of the year going for Club 250.

1. Track pricing data and publish a new chart on the app details page showing this. Often, discounts are linked to boosts in sales, owners and rankings, so displaying this data helps to understand our existing charts better. Moreover, project estimated sales using the review data and average selling price over the game's lifespan. For now, this will just be in USD since branching out to other currencies would greatly increase data acquisition time. If there's time, this might also include redoing the charts because I don't like how they currently look/behave.

2. Recommend games based on tag correlation. Using tag taxonomy to make better sense of the tags in the system and compute the correlation of tags between games, we can leverage this intelligence to recommend similar games to players that enjoyed a particular game. I believe this is a highly sought feature and will also improve discovery and navigation between games, without relying on rankings so much.

3. Power Search. We've mentioned it before, but a fully featured search is also highly sought after to customize finding games according to your criteria, such as tags (inclusive and exclusive), release dates, pricing ranges, review scores and features (such as single/multiplayer).