Solving sparsity and scalability problems for book recommendations on e-commerce

Abstract

This study proposed a hierarchical density-based spatial clustering of applications with noise (HDBSCAN) and randomized singular value decomposition (RSVD) collaborative filtering (CF) method to overcome sparsity and scalability problems for book recommendations on e-commerce. CF is an information retrieval system that assumes a user has the same interest in an object as other users have in the past. When handling large volumes of data, sparsity problems can arise, where finding a similarity relation of user preferences results from a small assessment of an object by users. The scalability is the increased computation of an algorithm caused by increased users or objects, which makes recommendations take longer to form, therefore making them less accurate. HDBSCAN is a density-based clustering method that simplifies the hierarchical arrangement of the most significant clusters for extraction to group users in the same cluster. RSVD is a linear dimension reduction method that breaks a matrix into three sub matrices by reconstructing the size of that matrix without removing its dominant part, especially for cluster result matrices. The HDBSCAN RSVD-CF model reduced the root mean squared error (RMSE) by 21.83%, being 3793.73 seconds faster than the CF model. It also performed very well compared to both RSVD-CF and HDBSCAN-CF.