Convert Pandas Dataframe from Row based to Columnar
<p>My Dataframe (df) looks like this:</p>

<pre><code>Date       FieldA  ValueA ValueB
09-02-2016 TypeA   3       5
09-02-2016 TypeB   6       7
</code></pre>

<p>I want the dataframe to look like below:</p>

<pre><code>Date       TypeA_ValueA   TypeA_ValueB  TypeB_ValueA TypeB_ValueB
09-02-2016  3             5             6            7
</code></pre>

<p>I tired the df.pivot in pandas where I can provide single Value column. It doesnt take more than one. When I provide more than one i get below exception. <a href="http://pandas.pydata.org/pandas-docs/stable/reshaping.html" rel="nofollow">pandas_pivot</a></p>

<pre><code>Exception: Data must be 1-dimensional
</code></pre>

<pre><code>In [36]: df
Out[36]: 
        Date FieldA  ValueA  ValueB
0 2016-09-02  TypeA       3       5
1 2016-09-02  TypeB       6       7
2 2016-09-03  TypeA       4       8
3 2016-09-03  TypeB       3       9

In [37]: v_cols = df.columns.difference(['FieldA', 'Date'])

In [38]: def func(x):
     ...:     d = {'_'.join([t, c]): x[x['FieldA'] == t][c].iloc[0] for t in x.FieldA for c in v_cols}
     ...:     for k, v in d.iteritems():
     ...:         x[k] = v
     ...:     return x
     ...: 

In [39]: newdf = df.groupby('Date').apply(func)

In [40]: newdf.drop(v_cols.tolist() + ['FieldA'], axis=1).drop_duplicates()
Out[340]: 
        Date  TypeA_ValueA  TypeA_ValueB  TypeB_ValueA  TypeB_ValueB
0 2016-09-02             3             5             6             7
2 2016-09-03             4             8             3             9
</code></pre>

python sql pandas pyspark spark-dataframe