pandas.concat of multiple data frames using only common columns
<p>I have multiple pandas data frame objects cost1, cost2 , cost3 ....</p>

<ol>
<li>They have different column names (and number of columns) but have some in common.</li>
<li>Number of columns are fairly large in each dataframe, hence handpicking the common columns manually will be painful.</li>
</ol>

<p>How can I append rows from all of these data frames into one single data frame 
while retaining elements from only the common column names ?</p>

<p>As of now I have</p>

<p>frames=[cost1,cost2,cost3...]</p>

<p>new_combined = pd.concat( frames,ignore_index=True)</p>

<p>This obviously contains columns which are not common across all data frames.</p>

<p>You can find the common columns with Python's <a href="https://docs.python.org/2/library/sets.html" rel="nofollow"><code>set.intersection</code></a>:</p>

<pre><code>common_cols = list(set.intersection(*(set(df.columns) for df in frames)))
</code></pre>

<p>To concatenate using only the common columns, you can use</p>

<pre><code>pd.concat([df[common_cols] for df in frames], ignore_index=True)
</code></pre>

python pandas dataframe