How to handle 0 entries in book crossing dataset
<p>I am working with book <a href="http://www2.informatik.uni-freiburg.de/~cziegler/BX/" rel="nofollow">crossing Data-set</a> , It have a file which given user X's rating for book Y, but a lot of entries contains value 0 that means user X liked the book Y but did not give rating to it. I am using collaborative filtering hence these 0 entries are creating problems for me as if taken an 0 they decrease overall rating of the book.</p>

<p>I am new to Data science field can some one help how to handle this ?</p>

<p>What I can think of is replace 0 rating by user's average book rating but than again I don't any argument to support my Idea.</p>

<p>ISBN codes are very messy, contain a lot of incorrect ISBNs, and are not unified.</p>

<p>Here are just a few examples:</p>

<pre><code>"User-ID";"ISBN";"Book-Rating"
"11676";" 9022906116";"7"
"11676";"\"0432534220\"";"6"
"11676";"\"2842053052\"";"7"
"11676";"0 7336 1053 6";"0"
"11676";"0=965044153";"7"
"11676";"0000000000";"9"
"11676";"00000000000";"8"
"146859";"01402.9182(PB";"7"
"158509";"0672=630155(P";"0"
"194500";"(THEWINDMILLP";"0"
</code></pre>

<p>So i would suggest to clean it up a little bit first:</p>

<pre><code>df.ISBN = df.ISBN.str.replace(r'[^\w\d]+', '')
</code></pre>

<p>then calculate average ratings:</p>

<pre><code>avg_ratings = df.groupby('ISBN')['Book-Rating'].mean().round().astype(np.int8)
</code></pre>

<p>and finally set average ratings to those books, having zero rating:</p>

<pre><code>df.ix[df['Book-Rating'] == 0, 'Book-Rating'] = df.ix[df['Book-Rating'] == 0, 'ISBN'].map(avg_ratings)
</code></pre>

python pandas machine-learning data-science