Iterating Over CSV Deleting Analyzed Data
<p>Hello I am trying to take a CSV file and iterate over each customers data. To explain, each customer has data for 12 months. I want to analyze their yearly data, save the correlations of this data to a new list and loop this until all customers have been analyzed.</p>

<p>For instance here is what a customers data might look like (simplified case):
<a href="http://i.stack.imgur.com/gGeR5.png" rel="nofollow"><img src="http://i.stack.imgur.com/gGeR5.png" alt="enter image description here"></a></p>

<p>I have been able to get this to work to generate correlations in a CSV of one customers data. However, there are thousands of customers in my datasheet. I want to use a nested for loop to get all of the correlation values for each customer into a list/array. The list would have a row of a specific customer's correlations then the next row would be the next customer.</p>

<p>Here is my current code:</p>

<pre><code>import numpy
from numpy import genfromtxt
overalldata = genfromtxt('C:\Users\User V\Desktop\CUSTDATA.csv', delimiter=',')
emptylist = []
overalldatasubtract = overalldata[13::]
#This is where I try to use the four loop to go through all the customers. I     don't know if len will give me all the rows or the number of columns.
for x in range(0,len(overalldata),11):
    for x in range(0,13,1):
            cust_months = overalldata[0:x,1]
            cust_balancenormal = overalldata[0:x,16]
            cust_demo_one = overalldata[0:x,2]
            cust_demo_two = overalldata[0:x,3]
            num_acct_A = overalldata[0:x,4]
            num_acct_B = overalldata[0:x,5]
    #Correlation Calculations
            demo_one_corr_balance = numpy.corrcoef(cust_balancenormal, cust_demo_one)[1,0]
            demo_two_corr_balance = numpy.corrcoef(cust_balancenormal, cust_demo_two)[1,0]
            demo_one_corr_acct_a = numpy.corrcoef(num_acct_A, cust_demo_one)[1,0]
            demo_one_corr_acct_b = numpy.corrcoef(num_acct_B, cust_demo_one)[1,0]
            demo_two_corr_acct_a = numpy.corrcoef(num_acct_A, cust_demo_two)[1,0]
            demo_two_corr_acct_b = numpy.corrcoef(num_acct_B, cust_demo_two)[1,0]

            result_correlation = [demo_one_corr_balance, demo_two_corr_balance, demo_one_corr_acct_a, demo_one_corr_acct_b, demo_two_corr_acct_a, demo_two_corr_acct_b]

result_correlation_combined = emptylist.append(result_correlation)
#This is where I try to delete the rows I have already analyzed.
overalldata = overalldata[11**x::]

print result_correlation_combined
print overalldatasubtract
</code></pre>

<p>It seemed that my subtraction method was working, but when I tried it with my larger data set, I realized my method is totally wrong.</p>

<p>Would you do this a different way? I think that it can work, but I cannot find my mistake.</p>

<p>You use the same variable <code>x</code> for both loops. In the second loop <code>x</code> goes from 0 to 12 whatever the customer, and since you set the line number only with <code>x</code> you're stuck on the first customer.</p>

<p>Your double loop should rather look like this :</p>

<pre><code># loop over the customers
for x_customer in range(0,len(overalldata),12):
    # loop over the months
    for x_month in range(0,12,1):
        # line number: x
        x = x_customer*12 + x_month
        ...
</code></pre>

<p>I changed the bounds and steps of the loops because :</p>

<ul>
<li><strong>loop 1:</strong> there are 12 months so 12 lines per customer -> step = 12</li>
<li><strong>loop 2:</strong> there are 12 months, so month number ranges from 0 to 11 -> <code>range(0,12,1)</code></li>
</ul>

python list csv append