Include two different sized data frames together in a pandas panel
<p>I'm crafting a panel of multiple data frames. Each is rather long. </p>

<p>I create the dfs, combine in a dictionary, then combine into a panel;</p>

<pre><code>for name in names: # large list of paths
    # Do some code to get data info (dI), dataframe (df) and nameID
    # Create a dictionary out of dfs by nameID
    dictDFs[nameID] = df 

# Collect all dataframes into one from dictionary dictDFs
pn = pd.Panel(dictDFs)
</code></pre>

<p>I then create a pickle file, <code>pn.to_pickle(path)</code></p>

<p>I would like to attach other info to the data frame not in the array. I don't want to change the size or shape of the data, keeping the array uniquely integers. </p>

<p>I cannot pack them as a tuple; disliked by Panel. However, this is what I thought it should look like:</p>

<pre><code># Create a dictionary out of df and dI by nameID
dictDFs[nameID] = (df,dI)
</code></pre>

<p>Thanks</p>

<p>I was able to solve this. The key to it was converting the dataframe to a tuple and using the tuple as the dictionary key such that the panel key was immutable:</p>

<pre><code>for name in names: # List of names
    nm = base(name)[:-4]

    # Uses each name to extract, trim, cure, and make meaningful
    dfInfo,df = some_function(name)
    dfInfo = dfInfo.rename(index=str, columns={0: nm})
</code></pre>

<p>Transform the tuples into a tuple of <code>pandas.core.frame.Pandas</code>:</p>

<pre><code>    tups = tuple(dfInfo.itertuples(index=False)) 
</code></pre>

<p>The <code>_fields</code> is the same for each tuple item for the one column dataframe:</p>

<pre><code>    nmT = tups[0]._fields[0]
</code></pre>

<p>Create a tuple from the tupled dataframe info:</p>

<pre><code>    dfInfo = (nmT, tuple(pd.Series(tup).loc[0] for tup in tups))
</code></pre>

<p>Now we can use the immutable tuple to create a dictionary with the key from the data info:</p>

<pre><code>    dictDFs[dfInfo] = df

# Collect all dataframes into one from dictionary dictDFs
pn = pd.Panel(dictDFs)
pn.to_pickle(path)
</code></pre>

python pandas panel pickle