In today's digital world, one of the keys to personalizing the user experience is effective recommendation systems. From Netflix's movie suggestions to Amazon's product recommendations, many platforms increase engagement by offering engaging and relevant content to their users. In this article, we will teach software developers and AI enthusiasts how to set up and implement a simple collaborative filtering-based recommendation system step-by-step using Python's powerful Surprise library.

What is Collaborative Filtering?

Collaborative Filtering is a method of making new recommendations based on the past behaviors and interactions of users or items. It fundamentally has two main approaches:

• User-Based Collaborative Filtering: Finds users with similar tastes and makes recommendations by assuming that if one user liked something, similar users would also like it.
• Item-Based Collaborative Filtering: Finds items similar to those a user has liked and recommends these similar items to the user.

In this article, we will focus on a more modern approach that uses matrix factorization algorithms like SVD (Singular Value Decomposition), which internally handles both user and item similarities.

Introduction to the Surprise Library

Surprise is a Python library that simplifies the development of recommendation systems in the field of machine learning. It allows you to easily apply various algorithms (SVD, NMF, k-NN, etc.) using user rating data and evaluate models. Installation is quite simple:

pip install scikit-surprise

Data Preparation

For recommendation systems, we typically need a dataset consisting of user_id, item_id, and rating (or interaction) columns. In this example, we will simulate a small portion of the MovieLens 100k dataset.

import pandas as pdfrom surprise import Dataset, Readerfrom surprise.model_selection import train_test_split, cross_validate# Create sample data (in real applications, read from a database or file)data = {    'user_id': ['user1', 'user1', 'user2', 'user2', 'user3', 'user3', 'user1', 'user2', 'user3'],    'item_id': ['itemA', 'itemB', 'itemB', 'itemC', 'itemA', 'itemC', 'itemC', 'itemA', 'itemB'],    'rating': [4, 3, 5, 4, 3, 2, 5, 4, 3]}df = pd.DataFrame(data)# Create a Reader object that the Surprise library understands.The Reader object defines the rating scaler = Reader(rating_scale=(1, 5))# Load the DataFrame into Surprise library's 'Dataset' format (using load_from_df method)data_surprise = Dataset.load_from_df(df[['user_id', 'item_id', 'rating']], reader)print("Dataset successfully loaded.")

Model Selection and Training

The Surprise library offers many algorithms. We will train our model using one of the most popular and effective algorithms, SVD (Singular Value Decomposition). SVD learns latent factors by decomposing the rating matrix into two smaller matrices.

from surprise import SVD# Split the data into training and test sets (similar to traditional machine learning methods)trainset, testset = train_test_split(data_surprise, test_size=0.25, random_state=42)# Initialize and train the SVD algorithmalgo = SVD(random_state=42, n_epochs=20, lr_all=0.005, reg_all=0.02)algo.fit(trainset)print("Model successfully trained.")

Model Evaluation

To evaluate how well our trained model makes predictions, metrics such as RMSE (Root Mean Squared Error) and MAE (Mean Absolute Error) are commonly used. Lower values indicate better performance.

from surprise import accuracy# Make predictions on the test setpredictions = algo.test(testset)# Calculate RMSE and MAE valuesaccuracy.rmse(predictions, verbose=True)accuracy.mae(predictions, verbose=True)

Making Predictions and Generating Recommendations

Our model is now ready to make new predictions. We can predict what rating a specific user will give to a specific item.

# Predict what rating user1 will give to itemDprediction_user1_itemD = algo.predict('user1', 'itemD')print(f"Predicted rating for user1 for itemD: {prediction_user1_itemD.est:.2f}")# Predict what rating user3 will give to itemAprediction_user3_itemA = algo.predict('user3', 'itemA')print(f"Predicted rating for user3 for itemA: {prediction_user3_itemA.est:.2f}")

Now let's see how to make the top 3 recommendations for a user from items they haven't rated yet:

def get_top_n_recommendations(predictions, user_id, n=3):    # Filter all predictions for the user    user_predictions = [pred for pred in predictions if pred.uid == user_id]    # Sort by estimated rating    user_predictions.sort(key=lambda x: x.est, reverse=True)    # Get the top n recommendations    top_n = user_predictions[:n]    return [(item.iid, item.est) for item in top_n]# Let's create predictions for all users and items (including those not in the training set)all_predictions = []for uid in df['user_id'].unique():    for iid in df['item_id'].unique():        # Only predict for items the user has not yet rated        if not ((df['user_id'] == uid) & (df['item_id'] == iid)).any():            all_predictions.append(algo.predict(uid, iid))# Get the top 3 recommendations for 'user1'top_3_for_user1 = get_top_n_recommendations(all_predictions, 'user1', n=3)print(f"Top 3 recommendations for user1: {top_3_for_user1}")# Get the top 3 recommendations for 'user2'top_3_for_user2 = get_top_n_recommendations(all_predictions, 'user2', n=3)print(f"Top 3 recommendations for user2: {top_3_for_user2}")

Conclusion

In this article, you learned how to build a simple but effective recommendation system using the principles of collaborative filtering with Python's Surprise library. We covered all steps hands-on, from preparing your dataset and training your model with the SVD algorithm to evaluating performance and finally providing personalized recommendations to users. By building upon this foundation, you can further develop your system with hybrid recommendation systems, solutions for cold-start problems, or more advanced algorithms. Continue exploring this practical application of artificial intelligence!