Updating transformer parameters after grid search on Pipeline
<p>I have a simple Pipeline for text analysis and classification consisting of a CountVectorizer, a TfidfTransformer, and finally a Multinomial Naive Bayes classifier. </p>

<pre><code>from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline

text_clf = Pipeline([('vect', CountVectorizer()),
                 ('tfidf', TfidfTransformer()),
                 ('clf', MultinomialNB())])
</code></pre>

<p>I now determine the best parameters using GridSearchCV (stop_words contains a previously loaded list of stop words):</p>

<pre><code>from sklearn.model_selection import GridSearchCV

parameters = {'vect__ngram_range': [(1,1), (1,2), (1,3)],
          'vect__stop_words': [None, stop_words],
          'tfidf__use_idf': [True, False],
          'clf__alpha': np.arange(0.0, 1.05, 0.05)
}

grid_clf = GridSearchCV(text_clf, parameters, n_jobs = 1)
_ = grid_clf.fit(X_train, y_train)
</code></pre>

<p>I can now see the best parameters of the model using <code>grid_clf.best_params_</code>:</p>

<pre><code>{'clf__alpha': 0.050000000000000003,
'tfidf__use_idf': True,
'vect__ngram_range': (1, 3),
'vect__stop_words': None}
</code></pre>

<p>My question is: how can I get back an updated pipeline with the best parameters returned by grid search? I would like to be able to call the first two steps of the pipeline (CountVectorizer and TfidfTransformer) with the appropriate parameters.</p>

<p>One workaround I have found is explicitly creating a new pipeline with the best parameters returned by grid search:</p>

<pre><code>multinomial_clf = Pipeline([('vect', CountVectorizer(stop_words=None, ngram_range=(1,3))),
                 ('tfidf', TfidfTransformer(use_idf = True)),
                 ('clf', MultinomialNB(alpha=0.05))])
_ = multinomial_clf.fit(X_train, y_train)
</code></pre>

<p>I can now access the CountVectorizer and TfidfTransformer using <code>multinomial_clf.steps</code> but I am sure there must be an easier way.</p>

<p>Thanks a lot for your help!</p>

<p>The Pipeline with the best parameters can be found with <code>grid_clf.best_estimator_</code></p>

<pre><code>grid_clf.best_estimator_

Pipeline(steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',
    dtype=&lt;class 'numpy.int64'&gt;, encoding='utf-8', input='content',
    lowercase=True, max_df=1.0, max_features=None, min_df=1,
    ngram_range=(1, 3), preprocessor=None, stop_words=None,
    strip...near_tf=False, use_idf=True)), ('clf', MultinomialNB(alpha=0.02, class_prior=None, fit_prior=True))])
</code></pre>

<p>However I am still confused as to how to transform using the transformers of the pipeline. The first two steps implement transform methods, whereas the last step does not. But if I try to call:</p>

<pre><code>grid_clf.best_estimator_.transform(['ok computer'])
</code></pre>

<p>I get the following error:</p>

<pre><code>AttributeError: 'MultinomialNB' object has no attribute 'transform'
</code></pre>

<p>Thanks for your help</p>

python scikit-learn pipeline grid-search