Compiling a dictionary by pulling data from other dictionaries
<p>I am doing a project in which I extract data from three different data sets and combine it to look at campaign contributions. To do this I turned the relevant data from two of the sets into dictionaries (canDict and otherDict) with ID numbers as keys and the information I need (party affiliation) as values. Then I wrote a program to pull party information based on the key (my third set included these ID numbers as well) and match them with the employer of the donating party, and the amount donated. That was a long winded explanation, but I thought it would help with understanding this chunk of code.</p>

<p>My problem is that, for some reason, my third dictionary (employerDict) won't compile. By the end of this step I should have a dictionary containing employers as keys, and a list of tuples as values, but after running it, the dictionary remains blank. I've been over this line by line a dozen times and I'm pulling my hair out - I can't for the life of me think why it won't work, which is making it hard to search for answers. I've commented almost every line to try to make it easier to understand out of context. <strong><em>Can anyone spot my mistake?</em></strong></p>

<p><em>Update:</em> I added a counter, n, to the outermost for loop to see if the program was iterating at all.</p>

<p><em>Update 2:</em> I added another if statement in the creation of the variable <code>party</code>, in case the ID at data[0] did not exist in canDict or in otherDict. I also added some already suggested fixes from the comments.</p>

<pre><code>n=0
with open(path3) as f:                         # path3 is a txt file
    for line in f:
        n+=1
        if n % 10000 == 0:
            print(n)
        data = line.split("|")                 # Splitting each line into its entries (delimited by the symbol |)
        party = canDict.get(data[0])           # data[0] is an ID number. canDict and otherDict contain these IDs as keys with party affiliations as values
        if party is None:
            party = otherDict[data[0]]         # If there is no matching ID number in canDict, search otherDict
            if party is None:
                party = 'Other'
        else:
            print('ERROR: party is None')
        x = (party, int(data[14]))             # Creating a tuple of the the party (found through the loop) and an integer amount from the file path3
        employer = data[11]                    # Index 11 in path3 is the employer of the person
        if employer != '':
            value = employerDict.get(employer) # If the employer field is not blank, see if this employer is already a key in employerDict
            if value is None:
                employerDict[employer] = [x]   # If the key does not exist, create it and add a list including the tuple x as its value
            else:
                employerDict[employer].append(x) # If it does exist, add the tuple x to the existing value
        else:
            print('ERROR: employer == ''')
</code></pre>

<p>Suppose we simplify and rearrange things a bit:</p>

<pre><code>import sys
from collections import defaultdict

employerDict = defaultdict(list)

ID, EMPLOYER, AMOUNT = 0, 11, 14

with open(path3) as f:  # path3 is a *.txt file

    for n, line in enumerate(f):
        if n % 10000 == 0:
            print(n)

        data = line.rstrip().split('|')  # Splitting each line into its entries

        employer = data[EMPLOYER]  # the employer of the person

        if employer == '':  # verify that you get 'employer' occasionally!
            print("ERROR: employer == ''", file=sys.stderr)
            continue

        id_string = data[ID]  # is ID always a string or is it an int (e.g. in other dicts)?

        # If there is no matching ID number in canDict, search otherDict
        party = canDict.get(id_string, otherDict.get(id_string, 'Other'))

        # Create a tuple of the the party (found through the loop) and an integer amount from the file path3
        x = (party, int(data[AMOUNT]))

        employerDict[employer].append(x)  # Add the tuple x to the (automatically) existing list
</code></pre>

<p>Does this help?</p>

python dictionary