How to write a fit_transformer with two inputs and include it in a pipeline in python sklearn?
<p>Given some fake data:</p>

<pre><code>X = pd.DataFrame( np.random.randint(1,10,28).reshape(14,2) )
y = pd.Series( np.repeat([0,1], [10,4]) ) # imbalanced with more 0s than 1s
</code></pre>

<p>I write a sklearn fit-transformer that under-samples the majority of y to match the length of the minority label. I want to use it in a pipeline. </p>

<pre><code>from sklearn.base import BaseEstimator, TransformerMixin

class UnderSampling(BaseEstimator, TransformerMixin):
    def fit(self, X, y): # I don't need fit to do anything
        return self

    def transform(self, X, y):
        is_pos = y == 1
        idx_pos = y[is_pos].index
        random.seed(random_state)
        idx_neg = random.sample(y[~is_pos].index, is_pos.sum())
        idx = sorted(list(idx_pos) + list(idx_neg))
        X_resampled = X.loc[idx]
        y_resampled = y.loc[idx]
        return X_resampled, y_resampled

    def fit_transform(self, X, y):
        return self.transform(X,y)
</code></pre>

<p>Most unfortunately, I cannot use it in a pipeline. </p>

<pre><code>from sklearn.pipeline import make_pipeline
us = UnderSampling()
rfc = RandomForestClassifier()
model = make_pipeline(us, rfc)
model.fit(X,y)
</code></pre>

<p>How can I make this pipeline work?</p>

<p>You're not meant to call the estimator methods on the class directly, you're meant to call it on a class instance; this is because estimators often to have some type of stored state (the model coefficients for example):</p>

<pre><code>u = UnderSampling()
a,b = u.fit(X, y)
a,b = u.fit_transform(X, y)
</code></pre>

python scikit-learn pipeline transformer