sklearn function transformer in pipeline
<p>Writing my first pipeline for sk-learn I stumbled upon some issues when only a subset of columns is put into a pipeline:</p>

<pre><code>mydf = pd.DataFrame({'classLabel':[0,0,0,1,1,0,0,0],
                   'categorical':[7,8,9,5,7,5,6,4],
                   'numeric1':[7,8,9,5,7,5,6,4],
                   'numeric2':[7,8,9,5,7,5,6,"N.A"]})
columnsNumber = ['numeric1']
XoneColumn = X[columnsNumber]
</code></pre>

<p>I use the <code>functionTransformer</code> like:</p>

<pre><code>def extractSpecificColumn(X, columns):
    return X[columns]

pipeline = Pipeline([
    ('features', FeatureUnion([
        ('continuous', Pipeline([
            ('numeric', FunctionTransformer(columnsNumber)),
            ('scale', StandardScaler())
        ]))
    ], n_jobs=1)),
    ('estimator', RandomForestClassifier(n_estimators=50, criterion='entropy', n_jobs=-1))
])

cv.cross_val_score(pipeline, XoneColumn, y, cv=folds, scoring=kappaScore)
</code></pre>

<p>This results in: <code>TypeError: 'list' object is not callable</code> when the function transformer is enabled.</p>

<h1>edit:</h1>

<p>If I instantiate a <code>ColumnExtractor</code> like below no error is returned. But isn't the <code>functionTransformer</code> meant just for simple cases like this one and should just work?</p>

<pre><code>class ColumnExtractor(TransformerMixin):
    def __init__(self, columns):
        self.columns = columns

    def transform(self, X, *_):
        return X[self.columns]

    def fit(self, *_):
        return self
</code></pre>

<p><code>FunctionTransformer</code> is used to "lift" a function to a transformation which I think can help with some data cleaning steps. Imagine you have a mostly numeric array and you want to transform it with a Transformer that that will error out if it gets a <code>nan</code> (like <code>Normalize</code>). You might end up with something like</p>

<pre><code>df.fillna(0, inplace=True)
...
cross_val_score(pipeline, ...)
</code></pre>

<p>but maybe you that <code>fillna</code> is only required in one transformation so instead of having the <code>fillna</code> like above, you have</p>

<pre><code>normalize = make_pipeline(
    FunctionTransformer(np.nan_to_num, validate=False),
    Normalize()
)
</code></pre>

<p>which ends up normalizing it as you want. Then you can use that snippet in  more places without littering your code with <code>.fillna(0)</code></p>

<p>In your example, you're passing in <code>['numeric1']</code> which is a <code>list</code> and not an extractor like the similarly typed <code>df[['numeric1']]</code>. What you may want instead is more like </p>

<pre><code>FunctionTransformer(operator.itemgetter(columns))
</code></pre>

<p>but that still wont work because the object that is ultimately passed into the FunctionTransformer will be an <code>np.array</code> and not a <code>DataFrame</code>.</p>

<p>In order to do operations on particular columns of a <code>DataFrame</code>, you may want to use a library like <a href="https://github.com/paulgb/sklearn-pandas" rel="nofollow">sklearn-pandas</a> which allows you to define particular transformers by column.</p>

python scikit-learn pipeline transformer