Creating .csv based on values in dictionary
<p>I'm trying to parse an XML file, return the values and put it into a .csv file. I have the following code so far:</p>

<pre><code>for shift_i in shift_list :

    # Iterates through all values in 'shift_list' for later comparison to ensure all tags are only counted once
    for node in tree.xpath("//Data/Status[@Name and @Reason]"):
    # Iterates through all nodes containing a 'Name' and 'Reason' attribute
        state = node.attrib["Name"]
        reason = node.attrib["Reason"]
        end = node.attrib["End"]
        start = node.attrib[u'Start']
        # Sets each of the attribute values to the name of the attribute all lowercase
        try:
        shift = node.attrib[u'Shift']
        except:
            continue
        # Tries to set shift attribute value to 'shift' variable, sometimes fails if no Shift attribute is present
        if shift == shift_i :
        # If the Shift attribute is equal to the current iteration from the 'shift_list', takes the difference of start and end and appends that value to the list with the given Name, Reason, and Shift
            tdelta = datetime.strptime(end, FMT) - datetime.strptime(start, FMT)
            d[state, reason, shift].append((tdelta.total_seconds()) / 60)

    for node in tree.xpath("//Data/Status[not(@Reason)]"):
    # Iterates through Status nodes with no Reason attribute
        state = node.attrib["Name"]
        end = node.attrib["End"]
        start = node.attrib[u'Start']
        # Sets each of the attribute values to the name of the attribute all lowercase
        try:
            shift = node.attrib[u'Shift']
        except:
            continue
        # Tries to set shift attribute value to 'shift' variable, sometimes fails if no Shift
        #      attribute is present
        if shift == shift_i:
            # If the Shift attribute is equal to the current iteration from the 'shift_list',
            #   takes the difference of start and end and appends that value to the list with
            #   the given Name, "No Reason" string, and Shift
            tdelta = datetime.strptime(end, FMT) - datetime.strptime(start, FMT)
            d[state, 'No Reason', shift].append((tdelta.total_seconds()) / 60)

    for item in d :
     # Iterates through all items of d
        d[item] = sum(d[item])
        # Sums all values related to 'item' and replaces value in dictionary
    a.update(d)
    # Current keys and values in temporary dictionary 'd' to permanent
    #   dictionary 'a' for further analysis
    d.clear()
    # Clears dictionary d of current iterations keys and values to start fresh for next
    #   iteration. If this is not done, d[item] = sum(d[item]) returns
    #   "TypeError: 'float' object is not iterable"
</code></pre>

<p>This creates a dictionary with values that look like this:</p>

<pre><code>{('Name1','Reason','Shift'):Value,('Name2','Reason','Shift'):Value....}
</code></pre>

<p>print(a) returns this</p>

<pre><code>defaultdict(&lt;class 'list'&gt;, {('Test Run', 'No Reason', 'Night'): 5.03825, ('Slow Running', 'No Reason', 'Day'): 10.72996666666667, ('Idle', 'Shift Start Up', 'Day'): 5.425433333333333, ('Idle', 'Unscheduled', 'Afternoon'): 470.0, ('Idle', 'Early Departure', 'Day'): 0.32965, ('Idle', 'Break Creep', 'Day'): 24.754250000000003, ('Idle', 'Break', 'Day'): 40.0, ('Micro Stoppage', 'No Reason', 'Day'): 39.71673333333333, ('Idle', 'Unscheduled', 'Night'): 474.96175, ('Running', 'No Reason', 'Day'): 329.4991500000004, ('Idle', 'No Reason', 'Day'): 19.544816666666666})
</code></pre>

<p>I want to create a .csv that has columns of 'Names'+'Reasons' with the totals, and the rows are described by the 'Shift'. Like this:</p>

<pre><code>         Name1-Reason    Name2-Reason    Name3-Reason    Name4-Reason
Shift1      value          value            value           value
Shift2      value          value            value           value
Shift3      value          value            value           value
</code></pre>

<p>I'm not sure how to go about doing this. I tried using nested Dicts to better describe my data but I got a TypeError when using </p>

<pre><code>d[state][reason][shift].append((tdelta.total_seconds()) / 60)
</code></pre>

<p>If there is a better way to do this please let me know, I'm a very new to this and would love to hear all advice.</p>

<p>I think the following may do what you want or at least be close. One important consideration that was ignored by the way you say the CSV file should be formatted, is the fact that each row must have a <code>Name-Reason</code> column for <em>every possible</em> combination of the two, even if weren't any of that particular mixture in one or more of the shift rows â because, well, that just that's how the CSV file format works.</p>

<pre><code>from collections import defaultdict
import csv

# Dictionary keys are (Name, Reason, Shift)
d = {('Test Run', 'No Reason', 'Night'): 5.03825,
     ('Slow Running', 'No Reason', 'Day'): 10.72996666666667,
     ('Idle', 'Shift Start Up', 'Day'): 5.425433333333333,
     ('Idle', 'Unscheduled', 'Afternoon'): 470.0,
     ('Idle', 'Early Departure', 'Day'): 0.32965,
     ('Idle', 'Break Creep', 'Day'): 24.754250000000003,
     ('Idle', 'Break', 'Day'): 40.0,
     ('Micro Stoppage', 'No Reason', 'Day'): 39.71673333333333,
     ('Idle', 'Unscheduled', 'Night'): 474.96175,
     ('Running', 'No Reason', 'Day'): 329.4991500000004,
     ('Idle', 'No Reason', 'Day'): 19.544816666666666}

# Transfer data to a defaultdict of dicts.
dd = defaultdict(dict)
for (name, reason, shift), value in d.items():
    name_reason = name + '-' + reason  # Merge together to form lower level keys
    dd[shift][name_reason] = value

# Create a csv file from the data in the defaultdict.
ABSENT = '---'  # Placeholder for empty fields
name_reasons = sorted(name_reason for shift in dd.keys()
                                    for name_reason in dd[shift])
with open('dict.csv', 'wb') as csv_file:
    writer = csv.writer(csv_file, delimiter=',')
    writer.writerow(['Shift'] + name_reasons)  # column headers row
    for shift in sorted(dd):
        row = [shift] + [dd[shift].get(name_reason, ABSENT)
                                        for name_reason in name_reasons]
        writer.writerow(row)
</code></pre>

<p>Here's the contents of the <code>dict.csv</code> file the code above creates:</p>

<pre class="lang-none prettyprint-override"><code>Shift,Idle-Break,Idle-Break Creep,Idle-Early Departure,Idle-No Reason,Idle-Shift Start Up,Idle-Unscheduled,Idle-Unscheduled,Micro Stoppage-No Reason,Running-No Reason,Slow Running-No Reason,Test Run-No Reason
Afternoon,---,---,---,---,---,470.0,470.0,---,---,---,---
Day,40.0,24.754250000000003,0.32965,19.544816666666666,5.425433333333333,---,---,39.71673333333333,329.4991500000004,10.72996666666667,---
Night,---,---,---,---,---,474.96175,474.96175,---,---,---,5.03825
</code></pre>

python csv dictionary