Kwker

Tutorial: top-k recommendations

A recommender scores many candidate items for each user, then shows each user the few best. In this tutorial you take a table of scored candidates and pick the three best items for every user, in one call. Then you leave out the items a user already bought, and run it on a million candidates.

It takes about 10 minutes. You need Python with NumPy and Kwker installed (Installation).

Step 1: the candidates

The model's output is a long table with one row per (user, item) pair: who it is for, which item, and the score. Rows of one user are not next to each other, and the scores are in no order.

PythonRuns on your machine.
import numpy as np
import kwker

users = np.array([2, 0, 1, 0, 2, 1, 0, 2, 1, 0, 2, 1])
items = np.array([11, 14, 12, 10, 15, 13, 12, 10, 14, 15, 13, 11])
scores = np.array([0.31, 0.92, 0.55, 0.40, 0.88, 0.71, 0.66, 0.12, 0.59, 0.81, 0.47, 0.20])
print(len(scores), "candidates for", len(np.unique(users)), "users")
Output
12 candidates for 3 users

Step 2: the best items of one user

For a single user, take that user's rows and call top_k. It returns the k best scores and their positions in the array you passed. descending=True asks for the largest scores.

PythonRuns on your machine.
mine = users == 0
best, where = kwker.top_k(scores[mine], 3, descending=True)
print(best)
print(items[mine][where])
Output
[0.92 0.81 0.66]
[14 15 12]

Step 3: every user at once

Doing this user by user is slow with many users. top_k_by_group does all groups in one pass. It returns the distinct users, an offsets array, and the row numbers of each user's best rows: user i's rows are idx[offsets[i]:offsets[i + 1]], best first.

PythonRuns on your machine.
labels, offsets, idx = kwker.top_k_by_group(scores, users, 3, descending=True)
for i, user in enumerate(labels):
    rows = idx[offsets[i]:offsets[i + 1]]
    print(f"user {user}: items {items[rows].tolist()}, scores {scores[rows].tolist()}")
Output
user 0: items [14, 15, 12], scores [0.92, 0.81, 0.66]
user 1: items [13, 14, 12], scores [0.71, 0.59, 0.55]
user 2: items [15, 13, 11], scores [0.88, 0.47, 0.31]

Step 4: skip what they already bought

Users should not see items they already own. Mark those rows, keep the others, and run the same call on what is left. The row numbers it returns are positions in the filtered arrays.

PythonRuns on your machine.
bought = {(0, 14), (2, 15)}
keep = np.array([(int(u), int(i)) not in bought for u, i in zip(users, items)])
labels, offsets, idx = kwker.top_k_by_group(scores[keep], users[keep], 3, descending=True)
for i, user in enumerate(labels):
    rows = idx[offsets[i]:offsets[i + 1]]
    print(f"user {user}: items {items[keep][rows].tolist()}")
Output
user 0: items [15, 12, 10]
user 1: items [13, 14, 12]
user 2: items [13, 11, 10]

Item 14 is gone from user 0's list and item 15 from user 2's; each list moves up and takes the next best item, 10.

Step 5: a million candidates

Now score 100 candidates for each of 10,000 users, shuffle the rows, and pick five items per user. To check the answer, compute it a second way with NumPy: sort the rows by user and then by score (best first), and take each user's first five rows. Scores are unique here, so both ways must pick the same rows.

PythonRuns on your machine.
rng = np.random.default_rng(3)
n_users, per_user = 10_000, 100
users = np.repeat(np.arange(n_users), per_user)
items = rng.integers(0, 50_000, users.size)
scores = rng.permutation(users.size) / users.size
shuffle = rng.permutation(users.size)
users, items, scores = users[shuffle], items[shuffle], scores[shuffle]

labels, offsets, idx = kwker.top_k_by_group(scores, users, 5, descending=True)

order = np.lexsort((-scores, users))                 # NumPy: by user, then best score first
first = np.r_[0, np.flatnonzero(np.diff(users[order])) + 1]
expected = np.concatenate([order[s:s + 5] for s in first])
print(len(labels), "users")
print(np.array_equal(idx, expected))
Output
10000 users
True

What you built

Next steps