Data Analysis Machine Learning and Applications Episode 3 Part 2 pdf

Applying Small Sample Tests for Behavior-based Recommendations 549 GEYER-SCHULZ, A. and HAHSLER, M. and NEUMANN, A. and THEDE, A. (2003a): Behavior-Based Recommender Systems as Value-Added Services for ScientificLi- braries. In: H. Bozdogan: Statistical Data Mining & Knowledge Discovery. Chapman & Hall / CRC, Boca Raton, 433–454. GEYER-SCHULZ, A. and NEUMANN, A. and THEDE, A. (2003b): An Architecture for Behavior-Based Library Recommender Systems. Journal of Information Technology and Libraries, 22(4). KOTLER, P. (1980): Marketing management: analysis, planning, and control. Prentice-Hall, Englewood Cliffs. MADDALA, G.S. (2001): Introduction to Econometrics. John Wiley, Chichester. NARAYANA, C.L. and MARKIN, R.J. (1975): Consumer Behavior and Product Performance: An Alternative Conceptualization. Journal of Marketing, 39(4), 1–6. PRIGOGINE, I. (1962): Non-equilibrium statistical mechanics. John Wiley & Sons, New York, London. ROTHSCHILD, M. and STIGLITZ, J. (1976): Equilibrium in Competitive Insurance Markets: An Essay on the Economics of Imperfect Information. Quarterly Journal of Economics, 90(4), 629–649. SAMUELSON, P.A. (1938a): A Note on the Pure Theory of Consumer’s Behaviour. Econom- ica, 5(17), 61–71. SAMUELSON, P.A. (1938b): A Note on the Pure Theory of Consumer’s Behaviour: An Ad- dendum. Economica, 5(19), 353–354. SAMUELSON, P.A. (1948): Consumption Theory in Terms of Revealed Preference. Econom- ica, 15(60), 243–253. SPENCE, M.A. (1974): Market Signaling: Information Transfer in Hiring and Related Screen- ing Processes. Harvard University Press, Cambridge, Massachusetts. SPIGGLE, S. and SEWALL, M.A. (1987): A Choice Sets Model of Retail Selection. Journal of Marketing, 51(2), 97–111. Collaborative Tag Recommendations Leandro Balby Marinho and Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Samelsonplatz 1, University of Hildesheim, D-31141 Hildesheim, Germany {marinho,schmidt-thieme}@ismll.uni-hildesheim.de Abstract. With the increasing popularity of collaborative tagging systems, services that as- sist the user in the task of tagging, such as tag recommenders, are more and more required. Being the scenario similar to traditional recommender systems where nearest neighbor algo- rithms, better known as collaborative filtering, were extensively and successfully applied, the application of the same methods to the problem of tag recommendation seems to be a natural way to follow. However, it is necessary to take into consideration some particularities of these systems, such as the absence of ratings and the fact that two entity types in a rating scale corre- spond to three top level entity types, i.e., user, resources and tags. In this paper we cast the tag recommendation problem into a collaborative filtering perspective and starting from a view on the plain recommendation task without attributes, we make a ground evaluation comparing different tag recommender algorithms on real data. 1 Introduction The process of building the Semantic Web (Berners-Lee et al. 2001) is currently an area of high activity. Both the theory and technology to support it have been al- ready defined and now one must fill this structure with life. In spite of the sounding simplicity, this task actually represents the biggest challenge towards its realization, i.e., adding semantic annotation to Web documents and resources in order to pro- vide knowledge access instead of unstructured material. Annotation represents an extra effort which certainly will not be voluntarily done without good reasons. In this sense, it is necessary to incentive and educate the user into this practice, e.g., showing the benefits that can be achieved through it and alleviating the extra bur- den with the recommendation of relevant annotations. With the recent appearing and increasing popularity of the so called collaborative tagging systems this is finally possible (Golber et al. (2005)). Recommending tags can serve various purposes, such as: increasing the chances of getting a resource annotated (or tagged) and reminding a user what a resource is about. Furthermore, lazy annotating users would not need to come up with a tag themselves but just select the ones readily available in the recommendation list ac- cording to what they think is more suitable for the given resource. 534 Leandro Balby Marinho and Lars Schmidt-Thieme Tag recommender systems recommend relevant tags for an untagged user re- source. Relevant here can assume different perspectives, for example, a tag can be judged relevant to a given resource according to the society point of view, through the opinion of experts in the domain or even based on the personal profile of an indi- vidual user. The question would be, which concept of relevance would the user prefer the most when using tag recommender services. This paper attempts to address this question through the following contributions: (i) formulation of the tag recommenda- tion problem and the introduction of a collaborative filtering-based tag recommender algorithm, (ii) presentation of a simple protocol for tag recommender evaluation (iii) and (iv) a ground and quantitative evaluation on real-life data comparing different tag recommender algorithms. 2 Related work The literature regarding the specific problem of collaborative tag recommendation is still sparse. The majority of the recent research work about collaborative tagging systems and folksonomies is concerned in devising approaches to better structure the data for browsing and searching where the recommendation problem is sometimes only highlighted as a potential property to be further explored in future work (Mika (2005), Hotho et al. (2006), Brooks and Montanez (2006), Heymann and Garcia- Molinay (2006)). We briefly describe below the works specifically investigating the problem of collaborative tag recommendation. Autotag (Mishne (2006)) is a tool that suggests tags for weblog posts using col- laborative filtering methods. Given a new weblog post, posts which are similar to it are identified through traditional information retrieval similarity measures. Next, the tags assigned to these posts are aggregated creating a ranked list of likely tags. De- spite the collaborative filtering scenario, there is no real personalization because the user is not taken directly into account. Furthermore, the evaluation is done in a semi- automatically fashion where the assumption of tag relevance for a given resource is defined to some extent by human experts. Xu et al. (2006) introduce a collaborative tag suggestion algorithm based on a set of general criteria to identify high quality tags. Some of the considered criteria are: high coverage of multiple facets to ensure good recall, least effort to reduce the cost involved in browsing, and high popularity to ensure tag quality. A goodness measure for tags, derived from collective user authorities, is iteratively adjusted by a reward- penalty algorithm, which also incorporates other sources of tags, e.g., content-based auto-generated tags. There is no quantitative evaluation. Benz et al. (Benz et al. (2006)) introduce a collaborative approach for book- mark classification based on a combination of nearest-neighbor-classifiers. Two sep- arate kinds of recommendations are generated: Keyword recommendations on the one hand, i.e. which keywords to use for annotating a new bookmark, and a recom- mendation of a classification on the other hand. The keyword recommender can be regarded as a collaborative tag recommender but its just a component of the overall Collaborative Tag Recommendations 535 algorithm, and therefore there is no information about its effectiveness as a stand- alone tool. The state-of-the-art tag recommenders in practice are services that provide the most-popular tags used by the society for a particular resource (Fig. 2). This is usu- ally done by means of tag clouds where the most frequently used tags are depicted in a larger font or otherwise emphasized. The approaches described above address important aspects of the problem, but there is still a lack regarding quantitative evaluation on basic tag recommender al- gorithms. Furthermore, there is no common or agreed protocol where the different algorithms should be compared. 3 Recommender Systems Recommender systems (RS) recommend products to customers based on ratings or past customer behavior. In general, RS predict ratings of items or suggest a list of unknown items to the user. They usually take the users, items and the ratings of items into account. A recommender system can be briefly formulated as: • A set of users U • A set of items I •AsetS⊆ R of possible ratings where r : U ×I → S is a partial function that associates ratings to user/item pairs. In datasets r typically is represented as a list of tuples (u, i, r(u, i)) with u ∈U, i ∈I and r defined for the domain dom r ⊆U ×I • Task: In recommender systems the recommendations are for a given user u ∈U aset ˜ I(u) ⊆ I of items. Usually ˜ I(u) is computed by first generating a ranking on the set of items according to some quality or relevance criterion, from which then the top n elements are selected (see Eq. 2 below). In CF, for m users and n items, the user profiles are represented in a user-item matrix X ∈ R m×n . The matrix can be decomposed into row vectors: X :=[x 1 , ,x m ]  with x u :=[x u,1 , ,x u,n ]  ,foru := 1, ,m, where x u,i indicates that user u rated item i by x u,i ∈ R. Each row vector x u corre- sponds thus to a user profile representing the item ratings of a particular user. This decomposition leads to user-based CF. The matrix can alternatively be represented by its column vectors: X :=[x 1 , ,x m ] with x i :=[x i,1 , ,x i,m ]  ,fori := 1, ,n, where each column vector x i corresponds to a specific item’s ratings by all m users. This representation leads to item-based recommendation algorithms. The pairwise similarities between users is usually computed by means of vector similarity: sim(prof u ,prof v ) := prof u ,prof v   prof u  prof v  (1) where u, v ∈U are two users and prof u and prof v are their profile vectors. 536 Leandro Balby Marinho and Lars Schmidt-Thieme Let B ⊆ I be the basket of items of the active user u ⊆ U and N u his/her best- neighbors. The topN recommendations usually consists of a list of items ranked by decreasing frequency of occurrence in the ratings of the neighbors: ˜ I(u) := n argmax i∈ I |{v ∈ N u | i ∈ r v,i }| (2) where B ∩ ˜ I(u) :=   and n is the size of the recommendation list. The brief discussion above refers only to the user-based CF case, since it is the focus of our work. Moreover, we consider only the recommendation task since in collaborative tagging systems there are no ratings and therefore no prediction. For a detailed description about the item-based CF algorithm see Deshpande et al. (2004). 4 Tag Recommender Systems Tag recommender systems recommend relevant tags for a given resource. As already discussed in section 1, the notion of relevance here can assume different perspectives and is usually hard to judge what concept of relevance would be preferable to a particular user. Collaborative tagging systems usually allow the users to see the most popular tags used for a given resource. This can be thought of a social-based tag recommender service since it represents the society opinion as a whole. Through CF we can measure the extent to which personalized notions of tag relevance are preferable in comparison with the socialized ones. Collaborative tagging systems are usually composed of users, resources and tags and allow users to assign tags to resources. What is considered a resource depends on the type of the system, e.g. URLs (del.icio.us 1 ), pictures (Flickr 2 ), music(Last.fm 3 ), etc. A tag recommender system can be formulated as follows: • A set of users U • A set of resources R •AsetoftagsT • A function s : U ×R → ˜ T associating tags to user/resources pairs, where ˜ T ⊆ T and s is defined for the domain dom s ⊆U ×R • Task: In tag recommender systems the recommendations are for a given user u ∈ U and a resource r ∈ R aset ˜ T(u,r) ⊆ T of tags. As well as in the tradi- tional formulation (section 3), ˜ T(u,r) can also be computed by first generating a ranking on the set of tags according to some quality or relevance criterion, from which then the top n elements are selected (see Algo.1 below). When comparing the formulation above with the one in section 3, we observe that CF cannot be applied directly. This is due to the additional dimension represented by 1 http://del.icio.us 2 http://www.flickr.com 3 http://www.last.fm Collaborative Tag Recommendations 537 T. Either we use more complex methods do deal directly with it or reduce it to a lower dimensional space where we could apply CF. We follow the latter one. To this end we take all the two dimensional projections of the original matrix preserving the user information. Letting K := |U|, M := |I| and L := |T |, the pro- jections result in two user profile matrices: a user-resource K ×M matrix X and a user-tag K ×L matrix Y. In collaborative tagging systems there is usually no rating information. The only information available is whether or not a resource and/or a tag occurred with the user. This can be encoded in the binary matrices X ∈{0,1} k×m and Y ∈{0,1} k×l indicating occurrence, e.g. x k,m = 1 and y k,l = 1, or non-occurrence of resources and tags with the users. Now we have the required setup to apply collabo- rative filtering. The algorithm starts selecting the users who have tagged the resource in question. Next, the pairwise similarity computation is performed (Eq.1). Notice that now we have two possible setups in which the neighborhood can be formed, either based on the profile matrix X or Y. The neighborhood’s tags for the resource in question are aggregated and weighted based on the neighbors’ similarities with the active user. Next the weights of each particular tag are summed up and the recommendation list is ranked by decreasing value of the summed weights. Ties are broken by smaller index. The overall CF procedure for tag recommendations is summarized in Algo.1. Algorithm 1 CF for tag recommendations • Given a new and/or untagged resource r ∈ R for the active user u ∈U • Let A := {v ⊆U |s v,r ≡  } denote the set of users who have tagged r where s is a function associating tags to user/resources pairs –Findk best neighbors: N u := k argmax v∈A sim(prof u ,prof v ) – Output the top n tags: ˜ T(u,r) := n argmax t∈T  v∈N u sim(prof u ,prof v )G(v,r,t) where G(v,r,t) := 1if(v, r,t) ∈U ×R×T and 0 else. 5 Experimental setup and results For our experiments we used the data made available by the Audioscrobbler 4 sys- tem, a music engine based on a collection of music profiles. These profiles are built through the use of the company’s flagship product, Last.fm, a system that provides personalized radio stations for its users and updates their profiles using the music they listen to and also makes personalized artist recommendations. In addition, Au- dioscrobbler exposes large portions of data through their web services API. 4 http://www.audioscrobbler.net 538 Leandro Balby Marinho and Lars Schmidt-Thieme Fig. 1. Most popular tags for a given artist Here we considered only the resources with 10 or more tag assignments. This gave us 2.917 users, 1.853 artists (playing the role of resources), 2.045 tags and 219.702 instances ((user, resource, tag) triples). We evaluated four tag recommenders: (i) a most global frequent tags, which rec- ommend the most used tags in the sample dataset, (ii) a most popular tag by re- source, which recommends the most used tags for a particular resource (in our case an artist), (iii) a user-resource-based CF, which computes the neighborhood based on the user-resource matrix and (iv) a user-tag-based CF, which computes the neigh- borhood based on the user-tag matrix. Notice that (ii) represents the state-of-the-art recommender used in practice (Fig.1). To evaluate the recommenders we used a variant of the leave-one-out holdout estimation that we named leave-tags-out. The idea is to choose a resource at random for each user in the test set and hide the tags attached to it. The algorithm must try to predict the hidden tags. To count the hits made by the algorithms we used the usual recall measure, recall macro (D) := 1 | D | |D|  1=1 |Y i ∩Z i | |Y i | (3) where D is the test set, Y i the true tags and Z i the predicted ones. Since the precision is forced by taking into account only a restricted number n of recommendations there is no need to evaluate precision or F1 measures, i.e., for this kind of scenario precision is just the same as recall up to a multiplicative constant. Each algorithm was evaluated 10 times for n=10 (size of recommendation list) and the results averaged (Fig. 2). Looking at the Figure 2 we see that the most popular by resource recommender reached a surprisingly high recall and that the user-resource-based CF did not per- form significantly better than that. The good results of the most popular by resource algorithm can in part be explained by the fact that this service is already available by Collaborative Tag Recommendations 539 Fig. 2. Recall of tag recommenders for n=10 Fig. 3. Recall for n varying from 1 to 10 the system. Besides that, it shows the strong influence of the society’s vocabulary on the user’s personal opinion. In the other hand, the user-tag-based CF recommender performed at least 2% better 5 than both the most-popular tag by resource and user- resource-based CF. Also notice that the improvement is consistent for different val- ues of n (Fig. 3). The best k-neighbors values were estimated through successive runnings where k was incremented until a point where no more improvements in the results were observed. 6 Conclusions In this paper we applied CF to the tag recommendation problem and made a quan- titative evaluation of its performance in comparison with other simpler tag recom- menders. Furthermore, we used a simple and suitable protocol with which further approaches can be compared. Despite the already good results of the baseline algorithms, the straightforward CF based on the user-tag profile matrix showed a significant improvement. This shows that users with similar tag vocabulary tend to tag alike, which indicates a preference for personalized tag recommendation services. It is also notorious the reasonable good results achieved by the most global fre- quent tags recommender, which indicates its adequacy for cold-start related prob- lems, where just a few tags are available in the system. In future work we plan to reproduce the same experiments with different datasets from different domains to confirm the results here presented. We also want to refine the CF algorithms exploring different combinations between the user similarities obtained from the two profile matrices, i.e., user-resources and user-tags. Moreover, 5 T-test for a significance level of 0.05. 540 Leandro Balby Marinho and Lars Schmidt-Thieme we will compare the CF approach with more complex models such as multi-label and relational classifiers. 7 Acknowledgments This work is supported by CNPq, an institution of Brazilian Government for scien- tific and technologic development. References BENZ, D., TSO, K., SCHMIDT-THIEME, L. (2006): Automatic Bookmark Classification: A Collaborative Approach. In: Proceedings of the Second Workshop on Innovations in Web Infrastructure (IWI 2006), Edinburgh, Scotland. BERNERS-LEE, T., HENDLER, J. and LASSILA, O. (2001): "Semantic Web", Scientific American, May 2001. BROOKS, C. H., MONTANEZ, N. (2006): Improved annotation of the blogosphere via au- totagging and hierarchical clustering. New York, NY, USA : ACM Press, WWW ’06: Proceedings of the 15th international conference on World Wide Web : 625 ˚ U632. DESHPANDE, M. and KARYPIS, G. (2004): Item-based top-n recommendation algorithms. ACM Transactions on Information Systems, 22(1):1-34. GOLBER, S., HUBERMAN, B.A. (2005): "The Structure of Collaborative Tagging System", Information Dynamics Lab: HP Labs, Palo Alto, USA, available at: http://arxiv.org/abs/cs.DL/0508082 HEYMANN, P. and GARCIA-MOLINAY, H. (2006): Collaborative Creation of Communal Hierarchical Taxonomies in Social Tagging Systems. Technical Report InfoLab 2006-10, Department of Computer Science, Stanford University, Stanford, CA, USA, April 2006. HOTHO, A., JAESCHKE, R., SCHMITZ, C., STUMME, G. (2006): Information Retrieval in Folksonomies: Search and Ranking. Heidelberg : Springer , The Semantic Web: Research and Applications 4011 : 411-426. MIKA, P. (2005): Ontologies Are Us: A Unified Model of Social Networks and Semantics. In: Y. Gil, E. Motta, V. R. Benjamins and M. A. Musen (Eds.), ISWC 2005, vol. 3729 of LNCS, pp. 522 ˝ U536. Springer-Verlag, Berlin Heidelberg. MISHNE, G. (2006): AutoTag: a collaborative approach to automated tag assignment for we- blog posts. New York, NY, USA : ACM Press , WWW ’06: Proceedings of the 15th international conference on World Wide Web : 953 ˚ U954. SARWAR, B., KARYPIS, G., KONSTAN, J. and REIDL, J. (2001): Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. New York, NY, USA: ACM Press, pp. 285-295. XU, Z., FU, Y., MAO J., SU, D. (2006): Towards the Semantic Web: Collaborative Tag Sug- gestions. Edinburgh, Scotland: Proceedings of the Collaborative Web Tagging Workshop at the WWW 2006. Comparison of Recommender System Algorithms Focusing on the New-item and User-bias Problem Stefan Hauger 1 , Karen H. L. Tso 2 and Lars Schmidt-Thieme 2 1 Department of Computer Science, University of Freiburg Georges-Koehler-Allee 51, 79110 Freiburg, Germany hauger@informatik.uni-freiburg.de 2 Information Systems and Machine Learning Lab, University of Hildesheim Samelsonplatz 1, 31141 Hildesheim, Germany {tso,schmidt-thieme}@ismll.uni-hildesheim.de Abstract. Recommender systems are used by an increasing number of e-commerce websites to help the customers to find suitable products from a large database. One of the most popular techniques for recommender systems is collaborative filtering. Several collaborative filtering algorithms claim to be able to solve i) the new-item problem, when a new item is introduced to the system and only a few or no ratings have been provided; and ii) the user-bias problem, when it is not possible to distinguish two items, which possess the same historical ratings from users, but different contents. However, for most algorithms, evaluations are not satisfying due to the lack of suitable evaluation metrics and protocols, thus, a fair comparison of the algorithms is not possible. In this paper, we introduce new methods and metrics for evaluating the user-bias and new- item problem for collaborative filtering algorithms which consider attributes. In addition, we conduct empirical analysis and compare the results of existing collaborative filtering algo- rithms for these two problems by using several public movie datasets on a common setting. 1 Introduction A Recommender system is a type of customization tool in e-commerce that gener- ates personalized recommendations, which match with the taste of the users. Col- laborative filtering (CF) (Sarwar et al. (2000, 2001)) is a popular technique used in recommender systems. It is used to predict the user interest for a given item based on user profiles. The concept of this technique is that the user, who received a recom- mendation for some sorts of items, would prefer the same items as other individuals with a similar mind set. However, besides its simplicity, one of the shortcomings of CF are the new-item or cold-start problem. If no ratings are given for new items, it is difficult for standard CF algorithms to determine their own clusters by using rating similarity and thus they fail to give accurate predictions. Another problem is the user-bias from historical rat- ings (Kim and Li (2004)), which occurs when two items, based on historical ratings [...]... Artificial Intelligence, pp 43 52, July 1998 BURKE, R (20 02) : Hybrid Recommender Systems: Survey and Experiments User Modeling and User-Adapted Interaction vol 12( 4), pp 33 1 37 0 HERLOCKER, J.L., KONSTAN, J.A., TERVEEN, L.G and RIEDL, J.T (20 04): Evaluating collaborative filtering recommender systems ACM Transactions on Information Systems, vol 22 , no 1, pp 5– 53, 20 04 HOFMANN, T (20 04): Latent Semantic Models... these drawbacks (Kim and Li (20 04)) In fact, there exist many approaches for combining content Comparison of RS Algorithms on the New-Item and User-Bias Problem 527 information with CF (Burke (20 02) , Melville et al (20 02) , Kim and Li (20 04), Tso and Schmidt-Thieme (20 05)) However, there has been lack of suitable evaluations which compute comparative analysis of attribute-aware and non attribute-aware... A.I., POPESCUL, A., UNGAR, L.H and PENNOCK, D.M (20 02) :Methods and metrics for cold-start recommendations In Proceedings of the 25 th annual international ACM SIGIR conference on Research and development in information retrieval New York, NY, USA: ACM Press, 20 02, pp 2 53 26 0 TSO, K and SCHMIDT-THIEME L (20 05): Attribute-aware Collaborative Filtering In Proceedings of 29 th Annual Conference of the Gesellschaft... experiments - the EachMovie, containing 2, 558,871 votes from 61, 1 32 users on 1,6 23 movies, and the MovieLens100k dataset, containing 100,000 ratings from 9 43 users on 1,6 82 movies The datasets also contain genre 530 Stefan Hauger, Karen H L Tso and Lars Schmidt-Thieme information for every movie in binary presentation, which we used as attributes The EachMovie dataset contains 10 different genres, MovieLens... and RIEDL, J (20 00): Analysis of recommendation algorithms for e-commerce In Proceedings of the Second ACM Conference on Electronic Commerce (ECÕ00), 20 00, pp 28 5 29 5 SARWAR, B.M., KARYPIS, G., KONSTAN, J.A and RIEDL, J (20 01): Itembased collaborative filtering recommendation algorithms In Proceedings of the 10th international conference on World Wide Web New York, NY, USA: ACM Press, 20 01, pp 28 5 29 5... Information Systems, 20 04, Vol 22 (1), pp 89–115 KIM, B.M and LI, Q (20 04): Probabilistic Model Estimation for Collaborative Filtering Based on Item Attributes IEEE International Conference on Web Intelligence MELVILLE, P., MOONEY, R and NAGARAJAN, R (20 02) : Contentboosted collaborative filtering In Proceedings of Eighteenth National Conference on Artificial Intelligence (AAAI -20 02) , pp 187–1 92 SARWAR, B.M.,... for both datasets 10 samples, in which 10 trials were run For each sample 1500 movies are selected, whereas a 1000 users in EachMovie and 600 users in MovieLens are selected and 20 neighbours for MovieLens and EachMovie for both userand the item-based CF No normalization is used in the aspect model and z is set to 40 for both datasets In the Kim & Li approach, we used 20 attribute-groups and 40 item... distance b and w1 , w2 , w3 the weights, which are used as parameters in the evaluation: w1 · h + w2 · f + w3 · b (4) k= 3 According to the entries of the consolidated NETV the most probable hypernym candidate can be chosen 4 Evaluation For the evaluation setup we extracted websites from Google for 90 named entities, which resulted in 90 corpora with each including 10 to 20 documents For a Goldstandard... coefficient (as shown in Formula 1) and signifies the co-occurrence of a hypernym candidate and the named entity based on term frequency 3. 2 Term distance The term distance approach takes the notion into account that smaller distances between hypernym candidate and named entity signify a more probable hypernym relation Hence, smaller distances are considered to be more valuable and are, therefore, preferred... et al (20 00)), item-based CF (Sarwar et al (20 01)) and Gaussian aspect model by Hofmann (20 04) as well as an approach, which takes attributes into account, by Kim & Li (20 04) In the next section, we present the related work In section 3, a brief description of the aspect model by Hofmann and the approach by Kim & Li will be presented An introduction of the evaluation techniques for the new-item and the . containing 2, 558,871 votes from 61, 1 32 users on 1,6 23 movies, and the MovieLens100k dataset, contain- ing 100,000 ratings from 9 43 users on 1,6 82 movies. The datasets also contain genre 530 Stefan. Wide Web : 625 ˚ U 6 32 . DESHPANDE, M. and KARYPIS, G. (20 04): Item-based top-n recommendation algorithms. ACM Transactions on Information Systems, 22 (1):1 -34 . GOLBER, S., HUBERMAN, B.A. (20 05): "The. Artificial Intelligence, pp. 43 52, July 1998. BURKE, R. (20 02) : Hybrid Recommender Systems: Survey and Experiments. User Modeling and User-Adapted Interaction. vol. 12( 4), pp. 33 1 37 0. HERLOCKER, J.L.,