Steam game recommender with ALS

Problem

A games store lives on discovery. Most players own a handful of titles, the catalogue runs to thousands, and the store front has room for ten. The brief was to recommend the next games each player is likely to enjoy, using only what the platform already records, which is who bought what and how long they played it.

There are no genres, prices or reviews in the data, only behaviour. That settles the family of method before any modelling starts. With nothing to describe the games themselves, the signal has to come from patterns across players, which is collaborative filtering. The job ran on a Databricks cluster in Spark, with every training run tracked in MLflow so the tuning could be compared, reproduced and audited afterwards.

Data

The file holds 200,000 events from 12,393 users across 5,155 games. It has no header row, so the schema came from the dataset’s documentation and was applied by renaming Spark’s positional columns. Each row is a user, a game, a behaviour and a value. For a purchase the value is always 1. For play it is hours.

Two behaviours, one of them misleading

129,511 rows are purchases and 70,489 are play records. The obvious question is how many purchases were never played, and in SQL that is an anti-join.

SELECT p.user_id, p.game_name
FROM steam p
LEFT JOIN steam pl
  ON p.user_id = pl.user_id AND p.game_name = pl.game_name AND pl.behaviour = 'play'
WHERE p.behaviour = 'purchase' AND pl.user_id IS NULL

The play condition sits in the join’s ON clause, not the WHERE, so purchases with no matching play survive the join with nulls on the right, and the IS NULL test keeps exactly those. 58,750 purchases, 45.4% of everything bought, were never played. Store sales and bundles fill libraries with games people never open, so a purchase is evidence of awareness, not of liking.

A heavy tail and a thin middle

Play hours are extremely skewed. The median is 4.5 hours, the mean 48.88, the 95th percentile 189 and the maximum 11,754, which is more than a year of continuous play. I used PERCENTILE_APPROX for the median and 95th percentile because an exact percentile needs a full sort across the cluster, and an approximate one with a bounded error answers the same question in one pass.

That sparsity decides the method. Neighbourhood methods recommend what similar users liked, and similarity needs overlap. With most users holding one or two games, most pairs of users share nothing.

Choosing the method

Matrix factorisation over neighbourhoods. Factorisation learns a short vector for every user and every game, so that a user’s affinity for a game is the dot product of the two. Two users who never shared a game can still end up close, because each shares games with others in between. That is how it copes with an almost empty matrix.

ALS over gradient descent. The factors can be learned by stochastic gradient descent or by alternating least squares. ALS holds the game vectors fixed and solves for every user exactly, as an independent ridge regression, then holds the user vectors fixed and solves for every game. Each half step is thousands of small independent solves, which is why Spark implements ALS. It parallelises across the cluster, with users and games partitioned into blocks so each round moves as little data as possible. Spark also scales the regularisation for each user and game by the number of ratings it has, so heavy users don’t get a free pass on overfitting.

Implicit or explicit mode. This is the decision that matters most, so I ran both.

• Explicit mode treats each value as a rating and minimises squared error over the cells that were observed. The empty cells are simply unknown.
• Implicit mode, from Hu, Koren and Volinsky’s work on implicit feedback, treats every cell as a yes or no preference. A played game is a yes and everything else is a weak no. The hours become confidence, one plus alpha times the value, so a game played for 200 hours is a much surer yes than one played for an hour. It fits over all cells, including the empty ones, and predicts a preference score between about 0 and 1.

Play hours are textbook implicit feedback, so implicit mode is the principled default. Testing explicit mode as well checks whether the log-hours carry enough graded information to be treated as ratings.

Building the pipeline

play_df = (steam_df.filter(F.col("behaviour") == "play")
           .select("user_id", "game_name", F.log1p("hours").alias("rating")))

indexer = StringIndexer(inputCol="game_name", outputCol="game_id").fit(play_df)
play_df = indexer.transform(play_df).withColumn("game_id", F.col("game_id").cast("integer"))
game_names = indexer.labels   # index back to name, for reading recommendations

training, test = play_df.randomSplit([0.8, 0.2], seed=42)
training.cache()
test.cache()

Play hours, logged. With 45.4% of purchases unplayed, training on purchases would teach the model about sales events. Play hours carry intensity, but raw they let one user with 11,754 hours outweigh thousands of ordinary players. log1p keeps the order and compresses the range, so 0.1 to 11,754 hours becomes 0.1 to 9.4, and the plus one keeps a zero defined.

IDs built inside Spark. ALS needs integer IDs, and games arrive as names. StringIndexer builds the mapping as a fitted pipeline stage, ordered by frequency, so Dota 2 became 0 and the most common games get the smallest IDs. There is no lookup file to drift out of step with the data, and hashing names to integers would have risked collisions between games. Only 3,600 of the 5,155 games were ever played, so the model’s catalogue is the played ones.

A fixed split, cached. A seeded 80/20 split (56,518 training rows, 13,971 test rows) means every configuration is judged on the same rows. Spark is lazy, so without cache() each of the 36 fits would have re-read the CSV, refiltered and re-indexed before training. Caching computes that once and keeps it in cluster memory.

Cold users dropped from scoring. A random split by row leaves some users and games only in the test set. ALS has no vector for them and predicts nothing, and a single empty prediction turns the whole RMSE into NaN. coldStartStrategy="drop" removes those rows from scoring.

Tuning, tracked in MLflow

I fitted ALS 36 times across rank 10, 20 and 50, regularisation 0.01, 0.1 and 0.5, 10 or 15 iterations, and both modes. Every run logged its parameters, RMSE, MAE and the fitted model to one MLflow experiment.

for rank, reg, iters, implicit in itertools.product([10, 20, 50], [0.01, 0.1, 0.5], [10, 15], [True, False]):
    with mlflow.start_run(run_name=f"ALS_r{rank}_reg{reg}_iter{iters}_imp{implicit}"):
        mlflow.log_params({"rank": rank, "regParam": reg, "maxIter": iters, "implicitPrefs": implicit})
        model = ALS(rank=rank, regParam=reg, maxIter=iters, implicitPrefs=implicit,
                    userCol="user_id", itemCol="game_id", ratingCol="rating",
                    coldStartStrategy="drop", seed=42).fit(training)
        preds = model.transform(test)
        mlflow.log_metrics({"rmse": rmse.evaluate(preds), "mae": mae.evaluate(preds)})
        mlflow.spark.log_model(model, "als_model")

runs = mlflow.search_runs(order_by=["metrics.rmse ASC"])   # the comparison is a query, not a scroll

Tracking turns the comparison into a query against the experiment rather than a hunt through printed output, and keeps every model reloadable by its run ID. That mattered later, when I reloaded the best implicit model to inspect its recommendations without retraining it.

In implicit mode every run landed between 2.40 and 2.45, and rank 10 or 20 edged out rank 50. Iterations barely mattered in either mode, which says ALS had converged within 10.

The run logs also showed where the time went. The grid took about 30 minutes, and for every run MLflow spent around 27 seconds trying to work out the model’s package requirements before falling back to a default. That is roughly 16 of the 30 minutes on a step that passing pip_requirements to log_model skips entirely.

Evaluation

The first readout put explicit mode far ahead, RMSE 1.4568 against 2.3973. That gap is not evidence that explicit mode recommends better, and reading it as such would be the classic mistake with this algorithm. Implicit ALS never tries to predict hours. It predicts a preference score between about 0 and 1, so its error against log-hours mostly measures the difference in scale. The implicit baseline’s R² of -1.28, worse than predicting the average, is that mismatch showing. RMSE can compare explicit runs with each other, which is what the chart above does, but it can’t choose between the modes.

So I judged the models the way a player would, by reading what they recommend. That turned up two things RMSE could never show.

• Recommendations repeated what users already owned. recommendForAllUsers ranks every game, including ones the user has played. For user 76767, 4 of the explicit model’s top 10 and 9 of the implicit model’s top 10 were games they had already played. The implicit model looked fixated on shooters, but it was mostly handing the user their own library back, which is exactly what a model that predicts “has played this” should do.
• Cold users all got the same list. 3 of the first 10 users in the output received the same ten games in the same order. These are users with one or two games, whose vectors carry almost no information, so the model falls back on what is broadly popular. That is the sparsity from the Data section, now visible in the product.

The scoring itself leaves out the hardest cases. Because cold rows are dropped, the RMSE covers only users and games seen in training, so the near half of users with a single game are barely represented in the number.

What didn’t work

RMSE was the wrong yardstick for a recommender. A store front shows ten games, so the question is whether those ten include games the player goes on to play. That is a ranking question, answered by precision and recall at 10 with already-owned games removed, and compared against the simplest serious baseline, recommending the most popular games. Without that baseline, an RMSE of 1.4568 has nothing to be measured against.

Implicit mode was also under-tuned. Its most important setting, alpha, which controls how fast confidence grows with hours, stayed at Spark’s default of 1 throughout. Tuning rank and regularisation while leaving alpha fixed tunes the less important knobs.

Limitations

• The data has no timestamps, so the split is random by row rather than by time. A model judged on predicting the past from the future looks better than it will in production.
• One split was used, not cross-validation, so the small differences between the best configurations may not hold on another split.
• The source file sits in a workspace volume that isn’t public, so the notebook can’t be rerun from the repository alone. Every number here is read from the outputs and run logs Spark saved into it.
• Almost half the users have a single game. No collaborative model serves them well on its own.

Stack

• Databricks
• Spark MLlib
• ALS
• MLflow
• PySpark
• Spark SQL
• Unity Catalog

Links

• Code on GitHub ↗

← All work