Merging two dictionaries together and adding None to the desired location
<p>I've scraped some <a href="http://www.richmond.com/data-center/salaries-local-government-employees-2014/?Agency=&amp;Last_Name=&amp;First_Name=&amp;Job_Title=PRINCIPAL&amp;appSession=82100926574322760625479516407609655632811717398403303845564132296054295099051816928601026788234447191565075848437884339109744304&amp;RecordID=29541&amp;PageID=8&amp;PrevPageID=2&amp;cpipage=1&amp;CPIsortType=&amp;CPIorderBy=&amp;cbCurrentRecordPosition=1&amp;Mod0LinkToDetails=True" rel="nofollow">data</a> and due to the way the website was structured I put the data into two dictionaries. </p>

<pre><code>&gt;&gt;&gt;pprint(dict(data))
{u'Additional compensation': [u'$32,241'],
 u'Agency': [u'Chesterfield County Schools', u'City of Richmond Schools'],
 u'Bonuses or other allowances': [u'$12,500'],
 u'COMMENTS': [u'$28,088 - Board Paid Annuity; $4,153 - Excess Health Benefit Contribution;',
               u''],
 u'Full Name': [u'Marcus J. Newsome', u'Dana T. Bedden'],
 u'Total Compensation': [u'$282,258', u'']}

&gt;&gt;&gt;pprint(dict(data2))
{u'Base Salary': [u'$229,758', u'$234,068'],
 u'COMMENTS': [u'12,500 CAR ALLOWANCE, 40,000 DEFFERRED COMPENSATION'],
 u'Deferred compensation': [u'$40,000'],
 u'Job Title': [u'SUPERINTENDENT', u'SUPERINTENDENT'],
 u'Total Compensation': [u'$266,309'],
 u'Work location': [u'Office Of Superintendent']}
</code></pre>

<p>I've combined the data into one master dictionary and I've attempted to put it into a csv file. </p>

<pre><code>for d in data2, data:
    for k, v in d.iteritems():
        master_data[k].append(v)

with open('test2.csv', 'wb') as f:
    writer = csv.writer(f)
    writer.writerows(zip(*([k] + master_data[k] for k in sorted(master_data))))
</code></pre>

<p>The problem is that only the first person's (<code>Marcus J. Newsome</code>) information is exported to the csv. I think this is due to fact that there are keys/values (for example <code>Additional compensation</code>) that belong to <code>Dana T. Bedden</code> that aren't present in <code>Marcus J Newsome</code>'s data. </p>

<p>To address this I've tried adding <code>None</code> to the positions to fix this problem. </p>

<pre><code>for d in data2, data:
    master_data.update((k, [None, master_data[k]]) for k in master_data if k not in d)

&gt;&gt;&gt;pprint(dict(master_data))
{u'Additional compensation': [None, [[u'$32,241']]],
 u'Agency': [None,
             [[u'Chesterfield County Schools', u'City of Richmond Schools']]],
 u'Base Salary': [None, [[u'$229,758', u'$234,068']]],
 u'Bonuses or other allowances': [None, [[u'$12,500']]],
 u'COMMENTS': [[u'12,500 CAR ALLOWANCE, 40,000 DEFFERRED COMPENSATION'],
               [u'$28,088 - Board Paid Annuity; $4,153 - Excess Health Benefit Contribution;',
                u'']],
 u'Deferred compensation': [None, [[u'$40,000']]],
 u'Full Name': [None, [[u'Marcus J. Newsome', u'Dana T. Bedden']]],
 u'Job Title': [None, [[u'SUPERINTENDENT', u'SUPERINTENDENT']]],
 u'Total Compensation': [[u'$266,309'], [u'$282,258', u'']],
 u'Work location': [None, [[u'Office Of Superintendent']]]}
</code></pre>

<p>Unfortunately, this doesn't seem to work the way I want it to. Ultimately I would like to have my output look like this:</p>

<p><strong>Desired Output</strong></p>

<pre><code>{u'Additional compensation': [[None, [u'$32,241']]],
 u'Agency': [[u'Chesterfield County Schools'], [u'City of Richmond Schools']]],
 u'Base Salary': [[u'$229,758'], [u'$234,068']]],
 u'Bonuses or other allowances': [[u'$12,500'], None]],
 u'COMMENTS': [[u'12,500 CAR ALLOWANCE, 40,000 DEFFERRED COMPENSATION'],
               [u'$28,088 - Board Paid Annuity; $4,153 - Excess Health Benefit Contribution;',
                u'']],
 u'Deferred compensation': [[u'$40,000'], None]],
 u'Full Name': [[u'Marcus J. Newsome'], [u'Dana T. Bedden']]],
 u'Job Title': [[u'SUPERINTENDENT'], [u'SUPERINTENDENT']]],
 u'Total Compensation': [[u'$266,309'], [u'$282,258', u'']],
 u'Work location': [None, [u'Office Of Superintendent']]]}
</code></pre>

<p>Anyone have any ideas?</p>

<p>I would restructure the data such that you have one dictionary for all relevant fields for each person. You can than easily use the dictwriter class of csv to export that data. </p>

python python-2.7 csv dictionary