Have Pandas column containing lists, how to pivot unique list elements to columns?
<p>I wrote a web scraper to pull information from a table of products and build a dataframe. The data table has a Description column which contains a comma separated string of attributes describing the product. I want to create a column in the dataframe for every unique attribute and populate the row in that column with the attribute's substring. Example df below.</p>

<pre><code>PRODUCTS     DATE        DESCRIPTION
Product A    2016-9-12   Steel, Red, High Hardness
Product B    2016-9-11   Blue, Lightweight, Steel
Product C    2016-9-12   Red
</code></pre>

<p>I figure the first step is to split the description into a list.</p>

<pre><code>In: df2 = df['DESCRIPTION'].str.split(',')

Out:
DESCRIPTION
['Steel', 'Red', 'High Hardness']
['Blue', 'Lightweight', 'Steel']
['Red']
</code></pre>

<p>My desired output looks like the table below. The column names are not particularly important.</p>

<pre><code>PRODUCTS     DATE        STEEL_COL  RED_COL    HIGH HARDNESS_COL  BLUE COL   LIGHTWEIGHT_COL
Product A    2016-9-12   Steel      Red        High Hardness
Product B    2016-9-11   Steel                                    Blue       Lightweight
Product C    2016-9-12              Red
</code></pre>

<p>I believe the columns can be set up using a Pivot but I'm not sure the most Pythonic way to populate the columns after establishing them. Any help is appreciated.</p>

<h2>UPDATE</h2>

<p>Thank you very much for the answers. I selected @MaxU's response as correct since it seems slightly more flexible, but @piRSquared's gets a very similar result and may even be considered the more Pythonic approach. I tested both version and both do what I needed. Thanks!</p>

<p>How about something that places an 'X' in the feature column if the product has that feature.</p>

<p>The below creates a list of unique features ('Steel', 'Red', etc.), then creates a column for each feature in the original df.  Then we iterate through each row and for each product feature, we place an 'X' in the cell.</p>

<pre><code>ml = []  

a = [ml.append(item) for l in df.DESCRIPTION for item in l]

unique_list_of_attributes = list(set(ml)) # unique features list

# place empty columns in original df for each feature
df = pd.concat([df,pd.DataFrame(columns=unique_list_of_attributes)]).fillna(value='')

# add 'X' in column if product has feature
for row in df.iterrows():
    for attribute in row[1]['DESCRIPTION']:
        df.loc[row[0],attribute] = 'X'
</code></pre>

<p>updated with example output:</p>

<pre><code>    PRODUCTS       DATE                 DESCRIPTION Blue HighHardness  \
0  Product A  2016-9-12  [Steel, Red, HighHardness]                 X   
1  Product B  2016-9-11  [Blue, Lightweight, Steel]    X                
2  Product C  2016-9-12                       [Red]                     

  Lightweight Red Steel  
0               X     X  
1           X         X  
2               X       
</code></pre>

python pandas numpy dataframe pivot