InvalidStateError with asyncio futures and RuntimeError with aiohttp when using Futures with callback
<p>I'm new to <code>asyncio</code> and <code>aiohttp</code>. I am currently getting this error and not sure why I am getting <code>InvalidStateError</code> for my <code>asyncio</code> future and <code>RuntimeError</code> for my session:</p>

<pre><code>Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/bli1/Development/QE/idea/trinity-tracer/tracer/tests/tracer.py", line 100, in &lt;module&gt;
    sys.exit(main(sys.argv))
  File "/Users/bli1/Development/QE/idea/trinity-tracer/tracer/tests/tracer.py", line 92, in main
    poster.post()
  File "/Users/bli1/Development/QE/idea/trinity-tracer/tracer/utils/poster.py", line 87, in post
    results = event_loop.run_until_complete(self.async_post_events(events))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py", line 387, in run_until_complete
    return future.result()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/Users/bli1/Development/QE/idea/trinity-tracer/tracer/utils/poster.py", line 79, in async_post_events
    task.add_done_callback(self.send_oracle, task.result(), session)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/futures.py", line 268, in result
    raise InvalidStateError('Result is not ready.')
asyncio.futures.InvalidStateError: Result is not ready.
Task exception was never retrieved
future: &lt;Task finished coro=&lt;Poster.async_post_event() done, defined at /Users/bli1/Development/QE/idea/trinity-tracer/tracer/utils/poster.py:62&gt; exception=RuntimeError('Session is closed',)&gt;
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/Users/bli1/Development/QE/idea/trinity-tracer/tracer/utils/poster.py", line 64, in async_post_event
    async with session.post(self.endpoint, data=event) as resp:
  File "/Users/bli1/Development/QE/idea/trinity-tracer/lib/python3.5/site-packages/aiohttp/client.py", line 565, in __aenter__
    self._resp = yield from self._coro
  File "/Users/bli1/Development/QE/idea/trinity-tracer/lib/python3.5/site-packages/aiohttp/client.py", line 161, in _request
    raise RuntimeError('Session is closed')
RuntimeError: Session is closed
</code></pre>

<p>What I am trying to do is <code>POST</code> to an endpoint, and then use the same event <code>POST</code>ed to post to another endpoint. This will be ran as another <code>async</code> method as a <code>callback</code></p>

<p>Here is my code:</p>

<pre><code>    async def async_post_event(self, event, session):
        async with session.post(self.endpoint, data=event) as resp:
            event["tracer"]["post"]["timestamp"] = time.time() * 1000.0
            event["tracer"]["post"]["statusCode"] = await resp.status
            return event

    async def send_oracle(self, event, session):
        async with session.post(self.oracle, data=event) as resp:
            return event["event"]["event_header"]["event_id"], await resp.status

    async def async_post_events(self, events):
        tasks = []
        conn = aiohttp.TCPConnector(verify_ssl=self.secure)
        async with aiohttp.ClientSession(connector=conn) as session:
            for event in events:
                task = asyncio.ensure_future(self.async_post_event(event, session))
                task.add_done_callback(self.send_oracle, task.result(), session)
                tasks.append(task)
        return await asyncio.gather(*tasks)

    def post(self):
        event_loop = asyncio.get_event_loop()
        try:
            events = [self.gen_random_event() for i in range(self.num_post)]
            results = event_loop.run_until_complete(self.async_post_events(events))
            print(results)
        finally:
            event_loop.close()
</code></pre>

<p><code>add_done_callback</code> accepts a <em>callback</em>, not a <em>coroutine</em>.
Moreover it's a part of very low level API which should be avoided by a casual developer.</p>

<p>But your main mistake is calling <code>session.post()</code> outside of <code>ClientSession</code> async context manager, the stacktrace explicitly points on it.</p>

<p>I've modified your snippet for getting something which looks like a working code:</p>

<pre><code>async def async_post_event(self, event, session):
    async with session.post(self.endpoint, data=event) as resp:
        event["tracer"]["post"]["timestamp"] = time.time() * 1000.0
        event["tracer"]["post"]["statusCode"] = await resp.status
    async with session.post(self.oracle, data=event) as resp:
        return event["event"]["event_header"]["event_id"], await resp.status

async def async_post_events(self, events):
    coros = []
    conn = aiohttp.TCPConnector(verify_ssl=self.secure)
    async with aiohttp.ClientSession(connector=conn) as session:
        for event in events:
            coros.append(self.async_post_event(event, session))
        return await asyncio.gather(*coros)

def post(self):
    event_loop = asyncio.get_event_loop()
    try:
        events = [self.gen_random_event() for i in range(self.num_post)]
        results = event_loop.run_until_complete(self.async_post_events(events))
        print(results)
    finally:
        event_loop.close()
</code></pre>

<p>You may extract two <em>posts</em> from <code>async_post_event</code> into separate coroutines but the main idea remains the same.</p>

python python-3.5 python-asyncio aiohttp