Modify tf-idf vectorizer for some keywords
<p>I am creating a tf-idf matrix for finding cosine similarity. But I want some frequent words from a set to have more weightage(i.e, tf-idf value).     </p>

<pre><code>tfidf_vectorizer = TfidfVectorizer()
tfidf_matrix = tfidf_vectorizer.fit_transform(documents)
</code></pre>

<p>How can I modify the above tfidf_matrix for words in a particular set.</p>

<p>I converted tfidf-matrix of csr-type to 2-D array using,</p>

<pre><code>my_matrix = tfidf_matrix.toarray()
</code></pre>

<p>Then, found out the index of keyword using,</p>

<pre><code>tfidf_vectorizer.vocabulary_.get(keyword)
</code></pre>

<p>After that, Iterated over 2-D matrix and changed the tf-idf value according to requirements. Here, keyword_list contains the index of keywords for which we want to modify the tf-idf value.</p>

<pre><code>    for i in range(0, len(my_matrix)):
    for key in keyword_list:
        if key != None:
            key = (int)(key)
        if my_matrix[i][key] &gt; 0.0:
            my_matrix[i][key] = new_value
</code></pre>

<p>Again, changed my_matrix to csr_type using,</p>

<pre><code>tfidf_matrix = sparse.csr_matrix(my_matrix)
</code></pre>

<p>Hence, tfidf_matrix was modified for the list of keywords.</p>

python machine-learning scipy nlp nltk