split-apply-combine to sklearn pipeline
<p>I am trying to generate a pipeline using sklearn, and am not really sure how to go about it. Here is a minimal example:</p>

<pre><code>def numFeat(data):
    return data[['AGE', 'WASTGIRF']]

def catFeat(data):
    return pd.get_dummies(data[['PAI', 'smokenow1']])

features = FeatureUnion([('f1',FunctionTransformer(numFeat)),
                         ('f2',FunctionTransformer(catFeat))  ] )

pipeline = Pipeline( [('f', features), ('lm',LinearRegression())] )

data = pd.DataFrame({'AGE':[1,2,3,4], 
                     'WASTGIRF': [23,5,43,1], 
                     'PAI':['a','b','a','d'], 
                     'smokenow1': ["lots", "some", "none", "some"]})

pipeline.fit(data, y)
print pipeline.transform(data)
</code></pre>

<p>In the above example, <code>data</code> is a Pandas DataFrame that contains the columns <code>['AGE', 'WASTGIRF', 'PAI', 'smokenow1']</code> among others. </p>

<p>Of course, in the <code>FeatureUnion</code> example, I want to supply many more transformation operations, but, all of them take a Pandas DataFrame and return another Pandas DataFrame. So in effect, I want to do something like this ...</p>

<pre><code>data --+--&gt;num features--&gt;num transforms--+--&gt;FeatureUnion--&gt;model
       |                                  |
       +--&gt;cat features--&gt;cat transforms--+
</code></pre>

<p>How do I go about doing this?</p>

<p>For the example above, the error i get is ...</p>

<pre><code>TypeError: float() argument must be a string or a number
</code></pre>

<p>You need to initialise <code>FunctionTransformer</code> with <code>validate=False</code> (IMO this is a bad default that should be changed):</p>

<pre><code>features = FeatureUnion([('f1',FunctionTransformer(numFeat, validate=False)),
                         ('f2',FunctionTransformer(catFeat, validate=False))] )
</code></pre>

<p>See also <a href="http://stackoverflow.com/questions/39001956/sklearn-pipeline-how-to-apply-different-transformations-on-different-coluns/39009125">sklearn pipeline - how to apply different transformations on different columns</a></p>

python scikit-learn