Binding outputs of transformers in FeatureUnion
<p>New to python and sklearn so apologies in advance. I have two transformers and I would like to gather the results in a `FeatureUnion (for a final modelling step at the end). This should be quite simple but FeatureUnion is stacking the outputs rather than providing an nx2 array or DataFrame. In the example below I will generate some data that is 10 rows by 2 collumns. This will then generate two features that are 10 rows by 1 collumn. I would like the final feature union to have 10 rows and 1 collumn but what I get are 20 rows by 1 collumn.</p>

<p>I will try to demonstrate with my example below:</p>

<p>some imports</p>

<pre><code>import numpy as np
import pandas as pd
from sklearn import pipeline
from sklearn.base import TransformerMixin
</code></pre>

<p>some random data</p>

<pre><code>df = pd.DataFrame(np.random.rand(10, 2), columns=['a', 'b'])
</code></pre>

<p>a custom transformer that selects a collumn</p>

<pre><code>class Trans(TransformerMixin):
    def __init__(self, col_name):
        self.col_name = col_name
    def fit(self, X):
        return self                                                                    
    def transform(self, X):                                                           
        return X[self.col_name]
</code></pre>

<p>a pipeline that uses the transformer twice (in my real case I have two different transformers but this reproduces the problem)</p>

<pre><code>pipe = pipeline.FeatureUnion([
    ('select_a', Trans('a')),
    ('select_b', Trans('b'))
    ])
</code></pre>

<p>now i use the pipeline but it returns an array of twice the length</p>

<pre><code>pipe.fit_transform(df).shape

(20,)
</code></pre>

<p>however I would like an array with dimensions (10, 2).</p>

<p>Quick fix?</p>

<p>The transformers in the <code>FeatureUnion</code> need to return 2-dimensional matrices, however in your code by selecting a column, you are returning a 1-dimensional vector. You could fix this by selecting the column with <code>X[[self.col_name]]</code>.</p>

python scikit-learn pipeline