Recursive dictionary walking in python
<p>I am new to python and trying to create a recursive dictionary walker that outputs a path to each item in the dictionary.</p>

<p>Below is my code including some sample data.  The goal is to import weather data from weather underground and publish into mqtt topics on a raspberry pi.  (The mqtt code all runs fine and is independent of this, for this I just need to generate the paths)</p>

<p>The issue that I am having is that when I reach the 'forecastday' element of the dict, my function is returning what looks like json data instead of recursing further into the path.</p>

<p>I have tried a few methods of walking the dict, but when the examples started getting into 'generators' etc, things were starting to get above my head a little.</p>

<p>(The test data does not contain any private information, it is the general weather report for my nearest town)</p>

<p>Thanks</p>

<pre><code>import json

def print_path(root, data, path):

    for element in data.keys():
        if not isinstance(data[element], dict):
            print root + path + element, data[element]
        else:
            print_path(root, data[element], element+"/")

json_string = u'{"response":{"version":"0.1","termsofService":"http://www.wunderground.com/weather/api/d/terms.html","features":{"almanac":1,"astronomy":1,"conditions":1,"forecast":1}},"current_observation":{"image":{"url":"http://icons.wxug.com/graphics/wu2/logo_130x80.png","title":"Weather Underground","link":"http://www.wunderground.com"},"display_location":{"full":"Llanelli, United Kingdom","city":"Llanelli","state":"","state_name":"United Kingdom","country":"UK","country_iso3166":"GB","zip":"00000","magic":"11","wmo":"03605","latitude":"51.67610931","longitude":"-4.15666723","elevation":"17.00000000"},"observation_location":{"full":"Llanelli, CARMARTHENSHIRE","city":"Llanelli","state":"CARMARTHENSHIRE","country":"GB","country_iso3166":"GB","latitude":"51.679951","longitude":"-4.140789","elevation":"40 ft"},"estimated":{},"station_id":"ICARMART4","observation_time":"Last Updated on October 18, 8:07 AM BST","observation_time_rfc822":"Tue, 18 Oct 2016 08:07:38 +0100","observation_epoch":"1476774458","local_time_rfc822":"Tue, 18 Oct 2016 08:07:44 +0100","local_epoch":"1476774464","local_tz_short":"BST","local_tz_long":"Europe/London","local_tz_offset":"+0100","weather":"Mostly Cloudy","temperature_string":"49.0 F (9.4 C)","temp_f":49.0,"temp_c":9.4,"relative_humidity":"90%","wind_string":"From the West at 4.5 MPH Gusting to 6.9 MPH","wind_dir":"West","wind_degrees":272,"wind_mph":4.5,"wind_gust_mph":"6.9","wind_kph":7.2,"wind_gust_kph":"11.1","pressure_mb":"1019","pressure_in":"30.09","pressure_trend":"0","dewpoint_string":"46 F (8 C)","dewpoint_f":46,"dewpoint_c":8,"heat_index_string":"NA","heat_index_f":"NA","heat_index_c":"NA","windchill_string":"47 F (9 C)","windchill_f":"47","windchill_c":"9","feelslike_string":"47 F (9 C)","feelslike_f":"47","feelslike_c":"9","visibility_mi":"6.2","visibility_km":"10.0","solarradiation":"--","UV":"0","precip_1hr_string":"0.00 in ( 0 mm)","precip_1hr_in":"0.00","precip_1hr_metric":" 0","precip_today_string":"0.07 in (2 mm)","precip_today_in":"0.07","precip_today_metric":"2","soil_moisture":"255.0","icon":"mostlycloudy","icon_url":"http://icons.wxug.com/i/c/k/mostlycloudy.gif","forecast_url":"http://www.wunderground.com/global/stations/03605.html","history_url":"http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=ICARMART4","ob_url":"http://www.wunderground.com/cgi-bin/findweather/getForecast?query=51.679951,-4.140789","nowcast":""},"forecast":{"txt_forecast":{"date":"7:03 AM BST","forecastday":[{"period":0,"icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","title":"Tuesday","fcttext":"Sun and clouds mixed. High around 55F. Winds WNW at 10 to 20 mph.","fcttext_metric":"Sun and clouds mixed. High 12C. Winds WNW at 15 to 30 km/h.","pop":"0"},{"period":1,"icon":"nt_partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/nt_partlycloudy.gif","title":"Tuesday Night","fcttext":"Partly cloudy skies. Low 43F. Winds WNW at 5 to 10 mph.","fcttext_metric":"Partly cloudy. Low 6C. Winds WNW at 10 to 15 km/h.","pop":"10"},{"period":2,"icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","title":"Wednesday","fcttext":"Partly cloudy skies in the morning will give way to cloudy skies during the afternoon. High 56F. Winds NW at 10 to 15 mph.","fcttext_metric":"Partly to mostly cloudy. High 13C. Winds NW at 15 to 25 km/h.","pop":"10"},{"period":3,"icon":"nt_partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/nt_partlycloudy.gif","title":"Wednesday Night","fcttext":"A few clouds. Low 41F. Winds N at 5 to 10 mph.","fcttext_metric":"Partly cloudy. Low around 5C. Winds N at 10 to 15 km/h.","pop":"10"},{"period":4,"icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","title":"Thursday","fcttext":"Intervals of clouds and sunshine. High 59F. Winds N at 5 to 10 mph.","fcttext_metric":"Partly cloudy skies. High around 15C. Winds N at 10 to 15 km/h.","pop":"10"},{"period":5,"icon":"nt_partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/nt_partlycloudy.gif","title":"Thursday Night","fcttext":"Partly cloudy. Low 41F. Winds NE at 5 to 10 mph.","fcttext_metric":"Partly cloudy. Low near 5C. Winds NE at 10 to 15 km/h.","pop":"10"},{"period":6,"icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","title":"Friday","fcttext":"Intervals of clouds and sunshine. High 58F. Winds E at 5 to 10 mph.","fcttext_metric":"Sunshine and clouds mixed. High 14C. Winds E at 10 to 15 km/h.","pop":"10"},{"period":7,"icon":"nt_partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/nt_partlycloudy.gif","title":"Friday Night","fcttext":"Partly cloudy. Low 44F. Winds E at 5 to 10 mph.","fcttext_metric":"Partly cloudy. Low 6C. Winds E at 10 to 15 km/h.","pop":"10"}]},"simpleforecast":{"forecastday":[{"date":{"epoch":"1476813600","pretty":"7:00 PM BST on October 18, 2016","day":18,"month":10,"year":2016,"yday":291,"hour":19,"min":"00","sec":0,"isdst":"1","monthname":"October","monthname_short":"Oct","weekday_short":"Tue","weekday":"Tuesday","ampm":"PM","tz_short":"BST","tz_long":"Europe/London"},"period":1,"high":{"fahrenheit":"55","celsius":"13"},"low":{"fahrenheit":"43","celsius":"6"},"conditions":"Partly Cloudy","icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","skyicon":"","pop":0,"qpf_allday":{"in":0.00,"mm":0},"qpf_day":{"in":0.00,"mm":0},"qpf_night":{"in":0.00,"mm":0},"snow_allday":{"in":0.0,"cm":0.0},"snow_day":{"in":0.0,"cm":0.0},"snow_night":{"in":0.0,"cm":0.0},"maxwind":{"mph":20,"kph":32,"dir":"WNW","degrees":293},"avewind":{"mph":16,"kph":26,"dir":"WNW","degrees":293},"avehumidity":71,"maxhumidity":0,"minhumidity":0},{"date":{"epoch":"1476900000","pretty":"7:00 PM BST on October 19, 2016","day":19,"month":10,"year":2016,"yday":292,"hour":19,"min":"00","sec":0,"isdst":"1","monthname":"October","monthname_short":"Oct","weekday_short":"Wed","weekday":"Wednesday","ampm":"PM","tz_short":"BST","tz_long":"Europe/London"},"period":2,"high":{"fahrenheit":"56","celsius":"13"},"low":{"fahrenheit":"41","celsius":"5"},"conditions":"Partly Cloudy","icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","skyicon":"","pop":10,"qpf_allday":{"in":0.00,"mm":0},"qpf_day":{"in":0.00,"mm":0},"qpf_night":{"in":0.00,"mm":0},"snow_allday":{"in":0.0,"cm":0.0},"snow_day":{"in":0.0,"cm":0.0},"snow_night":{"in":0.0,"cm":0.0},"maxwind":{"mph":15,"kph":24,"dir":"NW","degrees":318},"avewind":{"mph":12,"kph":19,"dir":"NW","degrees":318},"avehumidity":79,"maxhumidity":0,"minhumidity":0},{"date":{"epoch":"1476986400","pretty":"7:00 PM BST on October 20, 2016","day":20,"month":10,"year":2016,"yday":293,"hour":19,"min":"00","sec":0,"isdst":"1","monthname":"October","monthname_short":"Oct","weekday_short":"Thu","weekday":"Thursday","ampm":"PM","tz_short":"BST","tz_long":"Europe/London"},"period":3,"high":{"fahrenheit":"59","celsius":"15"},"low":{"fahrenheit":"41","celsius":"5"},"conditions":"Partly Cloudy","icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","skyicon":"","pop":10,"qpf_allday":{"in":0.00,"mm":0},"qpf_day":{"in":0.00,"mm":0},"qpf_night":{"in":0.00,"mm":0},"snow_allday":{"in":0.0,"cm":0.0},"snow_day":{"in":0.0,"cm":0.0},"snow_night":{"in":0.0,"cm":0.0},"maxwind":{"mph":10,"kph":16,"dir":"N","degrees":2},"avewind":{"mph":7,"kph":11,"dir":"N","degrees":2},"avehumidity":78,"maxhumidity":0,"minhumidity":0},{"date":{"epoch":"1477072800","pretty":"7:00 PM BST on October 21, 2016","day":21,"month":10,"year":2016,"yday":294,"hour":19,"min":"00","sec":0,"isdst":"1","monthname":"October","monthname_short":"Oct","weekday_short":"Fri","weekday":"Friday","ampm":"PM","tz_short":"BST","tz_long":"Europe/London"},"period":4,"high":{"fahrenheit":"58","celsius":"14"},"low":{"fahrenheit":"44","celsius":"7"},"conditions":"Partly Cloudy","icon":"partlycloudy","icon_url":"http://icons.wxug.com/i/c/k/partlycloudy.gif","skyicon":"","pop":10,"qpf_allday":{"in":0.00,"mm":0},"qpf_day":{"in":0.00,"mm":0},"qpf_night":{"in":0.00,"mm":0},"snow_allday":{"in":0.0,"cm":0.0},"snow_day":{"in":0.0,"cm":0.0},"snow_night":{"in":0.0,"cm":0.0},"maxwind":{"mph":10,"kph":16,"dir":"E","degrees":91},"avewind":{"mph":7,"kph":11,"dir":"E","degrees":91},"avehumidity":79,"maxhumidity":0,"minhumidity":0}]}},"moon_phase":{"percentIlluminated":"93","ageOfMoon":"17","phaseofMoon":"Waning Gibbous","hemisphere":"North","current_time":{"hour":"8","minute":"07"},"sunrise":{"hour":"7","minute":"46"},"sunset":{"hour":"18","minute":"15"},"moonrise":{"hour":"20","minute":"10"},"moonset":{"hour":"10","minute":"24"}},"sun_phase":{"sunrise":{"hour":"7","minute":"46"},"sunset":{"hour":"18","minute":"15"}},"almanac":{"airport_code":"EGFF","temp_high":{"normal":{"F":"55","C":"12"},"record":{"F":"66","C":"18"},"recordyear":"1997"},"temp_low":{"normal":{"F":"46","C":"7"},"record":{"F":"33","C":"0"},"recordyear":"1998"}}}'

weather_report = json.loads(json_string)

print_path("dev/blah/weather/", weather_report, "")
</code></pre>

<p>EDIT>></p>

<p>This is some of the output that I get from the script</p>

<pre><code>...
dev/blah/weather/sunrise/minute 46
dev/blah/weather/sunrise/hour 7
dev/blah/weather/sunset/minute 15
dev/blah/weather/sunset/hour 18
dev/blah/weather/sunrise/minute 46
dev/blah/weather/sunrise/hour 7
dev/blah/weather/txt_forecast/date 7:03 AM BST
dev/blah/weather/txt_forecast/forecastday [{u'title': u'Tuesday', u'icon_url': u'http://icons.wxug.com/i/c/k/partlycloudy.gif', u'fcttext_metric': u'Sun and clouds mixed. High 12C. Winds WNW at 15 to 30 km/h.', u'period': 0, u'pop': u'0', u'fcttext': u'Sun and clouds mixed. High around 55F. Winds WNW at 10 to 20 mph.', u'icon': u'partlycloudy'}, {u'title': u'Tuesday Night', u'icon_url': u'http://icons.wxug.com/i/c/k/nt_partlycloudy.gif', u'fcttext_metric': u'Partly cloudy. Low 6C. Winds WNW at 10 to 15 km/h.', u'period': 1, u'pop': u'10', u'fcttext': u'Partly cloudy skies. Low 43F. Winds WNW at 5 to 10 mph.', u'icon': u'nt_partlycloudy'}, {u'title': u'Wednesday', u'icon_url': u'http://icons.wxug.com/i/c/k/partlycloudy.gif', u'fcttext_metric': u'Partly to mostly cloudy. High 13C. Winds NW at 15 to 25 km/h.', u'period': 2, u'pop': u'10', u'fcttext': u'Partly cloudy skies in the morning will give way to cloudy skies during the afternoon.
</code></pre>

<p>For the last entry here it should say</p>

<pre><code>dev/blah/weather/txt_forecast/forecastday/0/title Tuesday
dev/blah/weather/txt_forecast/forecastday/0/icon partlycloudy
</code></pre>

<p>but instead it shows</p>

<pre><code>[{u'title': u'Tuesday', u'icon_url': u'http://icons.wxug.com/i/c/k/partlycloudy.gif', u'fcttext_metric': u'Sun and clouds mixed. High 12C. Winds WNW at 15 to 30 km/h.', u'perio...
</code></pre>

<p>Just mix-in a regular for-loop into that recursion to iterate over the lists. </p>

<pre><code>def print_path(root, data, path):

    for element, val in data.items():
        if isinstance(val, dict):
            print_path(root, val, element+"/")
        elif isinstance(val, list):
            list_path = path+element+"/"
            for i, item in enumerate(val):
                print_path(root, item, list_path+str(i)+"/")
        else:
            print root + path + element, val
</code></pre>

<p>Should see <code>dev/blah/weather/txt_forecast/forecastday/0/title Tuesday</code> in the output</p>

python json dictionary recursion python-2.x