How to fetch JSON data from API, format / encode / write to a file?
<p>I need to fetch some data from a weather API, extract certain info and send it to std. output (in my case this is the console/terminal; I am playing around with python API scripting and do not yet have a web site/app do show fetched data).</p>

<p><strong>Example Python code from the API provider (simple to understand and works):</strong></p>

<pre><code>import urllib2
import json

API_KEY='mysuperawesomekey'


f = urllib2.urlopen('http://api.wunderground.com/api/' + API_KEY + '/geolookup/conditions/q/IA/Cedar_Rapids.json')

json_string = f.read()
parsed_json = json.loads(json_string)
location = parsed_json['location']['city']
temp_f = parsed_json['current_observation']['temp_f']

print "Current temperature in %s is: %s" % (location, temp_f)

f.close()
</code></pre>

<p>Since I am on a free plan, I don't want to use up my API requests but rather fetch the JSON data, save it to a file and use in another script to practice. I found a couple of solutions here on <a href="http://stackoverflow.com/questions/12309269/how-do-i-write-json-data-to-a-file-in-python">StackOverflow</a> but neither seem to work completely (nice formatting of the file):</p>

<p><strong>Attempt 1. to save the fetched data, add. to the orig. code above:</strong></p>

<pre><code>import io
import codecs

with open('data.json', 'w') as outfile:
json.dump(json_string, outfile, indent=4, sort_keys=True, separators=(",", ':'))
</code></pre>

<p><strong>Attempt 2:</strong></p>

<pre><code>with codecs.open('data.json', 'w', 'utf8') as jasonfile:
  jasonfile.write(json.dumps(parsed_json, sort_keys = True, ensure_ascii=False))
</code></pre>

<p>Both of my attempts work ("kind of") as I do get a <strong>.json</strong> file. But, upon inspecting it in my editor (Atom) I am seeing this (first few soft-break lines):</p>

<p><strong>Output:</strong></p>

<pre><code>"\n{\n  \"response\": {\n  \"version\":\"0.1\",\n\"termsofService\":\"http://www.wunderground.com/weather/api/d/terms.html\",\n  \"features\": {\n  \"geolookup\": 1\n  ,\n  \"conditions\": 1\n  }\n\t}\n\t\t,\t\"location\": {\n\t\t\"type\":\"CITY\",\n\t\t\"country\":\"US\",\n\t\t\"country_iso3166\":\"US\",\n\t\t\"country_name\":\"USA\",\n\t\t\"state\":\"IA\",\n\t\t\"city\":\"Cedar Rapids\",\n\t\t\"tz_short\":\"CDT\",\n\t\t\"tz_long\":\"America/Chicago\",\n\t\t\"lat\":\"41.97171021\",\n\t\t\"lon\":\"-91.65871429\",\n\t\t\"zip\":\...
</code></pre>

<p>It is all on one line, with newlines and tabs showing. I have a couple of questions:</p>

<ol>
<li>How can I check what format / kind of data-type API is returning? (FOr example is it just a raw string which needs to be JSON-iffied?)</li>
<li>Is my JSON not written in a human readable format because of encoding, delimiters fault?</li>
<li>Line <code>json_string = f.read()</code> from example code seems to "mimic" working with a "real" internal file object, is this correct?</li>
</ol>

<p>Many thanks!</p>

<p>Seems that it was quite a simple solution. In my original code, I was saving a "non-parsed" variable to a file:</p>

<pre><code>import urllib2
import json

API_KEY='key'

f = urllib2.urlopen('http://api.wunderground.com/api/' 
+ API_KEY + '/geolookup/conditions/q/IA/Cedar_Rapids.json')

# Saving the below variable into a .json file gives all the JSON data
# on a single line
json_string = f.read()

with open('data.json', 'w') as outfile:
    json.dump(json_string, outfile, indent=4, sort_keys=True, separators=(",", ':'))
</code></pre>

<p>which, of course, resulted in a file with JSON data on a single line. What I should have been doing is saving the <strong>parsed</strong> variable:</p>

<pre><code>import urllib2
import json

API_KEY='key'

f = urllib2.urlopen('http://api.wunderground.com/api/' + API_KEY + '/geolookup/conditions/q/IA/Cedar_Rapids.json')

json_string = f.read()

# THIS one is correct!
parsed_json = json.loads(json_string)

with open('data.json', 'w') as outfile:
  json.dump(parsed_json, outfile, indent=4, sort_keys=True, separators=(",", ':'))
</code></pre>

python json