Traversing multiple dataframes simultaneously
<p>I have three dataframes of three users with same column names like time, compass data,accelerometer data, gyroscope data and camera panning information. I want to traverse all the dataframes simultaneously to check for a particular time which user has performed camera panning and return the user(like in which data frame panning has been detected for a particular time). I have tried using dash for achieving parallelism but in vain. below is my code</p>

<pre><code>import pandas as pd
import glob
import numpy as np
import math
from scipy.signal import butter, lfilter
order=3
fs=30
cutoff=4.0

data=[]
gx=[]
gy=[]
g_x2=[]
g_y2=[]


dataList = glob.glob(r'C:\Users\chaitanya\Desktop\Thesis\*.csv')
for csv in dataList:
       data.append(pd.read_csv(csv))
for i in range(0, len(data)):
    data[i] = data[i].groupby("Time").agg(lambda x: x.value_counts().index[0])
    data[i].reset_index(level=0, inplace=True)

def butter_lowpass(cutoff,fs,order=5):
    nyq=0.5 * fs
    nor=cutoff / nyq
    b,a=butter(order,nor,btype='low', analog=False)
return b,a
def lowpass_filter(data,cutoff,fs,order=5):
    b,a=butter_lowpass(cutoff,fs,order=order)
    y=lfilter(b,a,data)
return y

for i in range(0,len(data)):
    gx.append(lowpass_filter(data[i]["Gyro_X"],cutoff,fs,order))
    gy.append(lowpass_filter(data[i]["Gyro_Y"],cutoff,fs,order))

    g_x2.append(gx[i]*gx[i])
    g_y2.append(gy[i]*gy[i])


g_rad=[[] for _ in range(len(data))]
g_ang=[[] for _ in range(len(data))]

for i in range(0,len(data)):
    for j in range(0,len(data[i])):
           g_ang[i].append(math.degrees(math.atan(gy[i][j]/gx[i][j])))


    data[i]["Ang"]=g_ang[i]


panning=[[] for _ in range(len(data))]
for i in range(0,len(data)):
    for j in data[i]["Ang"]:
        if 0-30&lt;=j&lt;=0+30:
            panning[i].append("Panning")
        elif 180-30&lt;=j&lt;=180+30:
            panning[i].append("left")
        else:
           panning[i].append("None")
    data[i]["Panning"]=panning[i]
result=[[] for _ in range(len(data))]
for i in range (0,len(data)):
    result[i].append(data[i].loc[data[i]['Panning']=='Panning','Ang'])
</code></pre>

<p>I'm going to make the assumption that you want to traverse simultaneously in time.  In any case, you want your three dataframes to have an index in the dimension you want to traverse.  </p>

<p>I'll generate 3 dataframes with rows representing random seconds in a 9 second period.  </p>

<p>Then, I'll align these with a <code>pd.concat</code> and <code>ffill</code> to be able to reference the last known data for any gaps.</p>

<pre><code>seconds = pd.date_range('2016-08-31', periods=10, freq='S')

n = 6
ssec = seconds.to_series()
sidx = ssec.sample(n).index

df1 = pd.DataFrame(np.random.randint(1, 10, (n, 3)),
                   ssec.sample(n).index.sort_values(),
                   ['compass', 'accel', 'gyro'])

df2 = pd.DataFrame(np.random.randint(1, 10, (n, 3)),
                   ssec.sample(n).index.sort_values(),
                   ['compass', 'accel', 'gyro'])

df3 = pd.DataFrame(np.random.randint(1, 10, (n, 3)),
                   ssec.sample(n).index.sort_values(),
                   ['compass', 'accel', 'gyro'])

df4 = pd.concat([df1, df2, df3], axis=1, keys=['df1', 'df2', 'df3']).ffill()
df4
</code></pre>

<p><a href="http://i.stack.imgur.com/H6Sh0.png" rel="nofollow"><img src="http://i.stack.imgur.com/H6Sh0.png" alt="enter image description here"></a></p>

<p>you can then proceed to walk through via <code>iterrows()</code></p>

<pre><code>for tstamp, row in df4.iterrows():
    print tstamp
</code></pre>

python pandas dataframe