Perform operation on each cell of one column with each cell of other columns in pandas
<p>I have a data frame(df) consisting of more than 1000 columns. Each cell consists of a list.
e.g.</p>

<pre><code>           0            1             2  .... n
0    [1,2,3]      [3,7,9]       [1,2,1]  ....[x,y,z]
1    [2,5,6]      [2,3,1]       [3,3,3]  ....[x1,y1,z1]
2       None      [2,0,1]       [2,2,2]  ....[x2,y2,z2]
3       None      [9,5,9]          None  ....None
</code></pre>

<p>This list is actually the dimensions. I need to find the euclidean distance of each cell in column 0 with every other cell in column 1 and store the minimum.
Similarly from column 0 to column 2 and then to column 3 so on..</p>

<h2>Example</h2>

<pre><code>distance of df[0][0] from df[1][0], df[1][1], df[1][2]
then of df[0][1] from df[1][0], df[1][1], df[1][2] and so on...
</code></pre>

<p>Currently i am doing it with help of for loops but it is taking a lot of time for large data. 
Following is the implementation::</p>

<pre><code>for n in range(len(df.columns)):
    for m in range(n+1,len(df.columns)):
        for q in range(df.shape[0]):
            min1=9999
            r=0
            while(r&lt;df.shape[0]):
                if(df[n][q] is not None or df[m][r] is not None):
                    dist=distance.euclidean(df[n][q],df[m][r])
                    if(d&lt;min1):
                            min1=d
                    if(min1==0):  *#because distance can never be less than zero*
                        break
                    r=r+1
</code></pre>

<p>Is there any other way to do this?</p>

<p>since you don't tell me how to handle Nond in the distance calculation, I just give a example. You need to handle the exception of None type yourself.</p>

<pre><code>import pandas as pd
import numpy as np
from scipy.spatial.distance import pdist

df = pd.DataFrame([{'a':[1,2,3], 'b':[3,7,9], 'c':[1,2,1]},
                   {'a':[2,5,6], 'b':[2,3,1], 'c':[3,3,3]}])
df.apply(distance, axis=1).apply(min, axis=1)    
</code></pre>

python pandas dataframe