Python 3.5 async/await with real code example
<p>I've read tons of articles and tutorial about Python's 3.5 async/await thing. I have to say I'm pretty confused because some use get_event_loop() and run_until_complete() some use ensure_future() some use asyncio.wait() and some use call_soon().</p>

<p>It seems like I have a lot choices but I have no idea if they are completely identical or there are cases where you use loops and there are cases where you use wait().</p>

<p>But the thing is all examples work with <code>asyncio.sleep()</code> as simulation of real slow operation which returns awaitable object. Once I try to swap this line for some real code whole thing fails. So my question is what the heck are differences between approaches written above and how should I run 3rd party library which is not ready for async/await. I do use Quandl service to fetch some stock data.</p>

<pre><code> import asyncio
 import quandl

 async def slow_operation(n):
     # await asyncio.sleep(1) # Works because it's await ready.
     await quandl.Dataset(n) # Doesn't work because it's not await ready.


 async def main():
     await asyncio.wait([
         slow_operation("SIX/US9884981013EUR4"),
         slow_operation("SIX/US88160R1014EUR4"),
     ])

 # You don't have to use any code for 50 requests/day.
 quandl.ApiConfig.api_key = "MY_SECRET_CODE"

 loop = asyncio.get_event_loop()
 loop.run_until_complete(main())
</code></pre>

<p>Hope you get the point how lost I feel and how simple thing I would like to have running in paralel.</p>

<p>Thank you.</p>

<p>If 3rd party library is not compatible with <code>async/await</code> then obviously you can't use it easily. There are 2 cases:</p>

<p>1) Let's say that the function in the library is asynchronous and it gives you a callback, e.g.</p>

<pre><code>def fn(..., clb):
    ...
</code></pre>

<p>So you can do:</p>

<pre><code>def on_result(...):
    ...

fn(..., on_result)
</code></pre>

<p>In that case you can wrap such function into asyncio protocol like this:</p>

<pre><code>from asyncio import Future

def wrapper(...):
    future = Future()
    def my_clb(...):
        future.set_result(xyz)
    fn(..., my_clb)
    return future
</code></pre>

<p>(use <code>future.set_exception(exc)</code> on exception)</p>

<p>Then you can simply call that wrapper in some <code>async</code> function with <code>await</code>:</p>

<pre><code>value = await wrapper(...)
</code></pre>

<p>Note that <code>await</code> works with any <code>Future</code> object, you don't have to declare <code>wrapper</code> as <code>async</code>.</p>

<p>2) If the function in the library is synchronous then you can run it in a separate thread (probably you would use some thread pool for that). The whole code may look like this:</p>

<pre><code>import asyncio
import time
from concurrent.futures import ThreadPoolExecutor

# Initialize 10 threads
THREAD_POOL = ThreadPoolExecutor(10)

def synchronous_handler(param1, ...):
    # do something synchronous
    time.sleep(2)
    return "foo"

# somewhere else
async def main():
    loop = asyncio.get_event_loop()
    futures = [
        loop.run_in_executor(THREAD_POOL, synchronous_handler, param1, ...),
        loop.run_in_executor(THREAD_POOL, synchronous_handler, param1, ...),
        loop.run_in_executor(THREAD_POOL, synchronous_handler, param1, ...),
    ]
    await asyncio.wait(futures)
    for future in futures:
        print(future.result())

with THREAD_POOL:
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
</code></pre>

<p>If you can't use threads for whatever reason then using such library simply makes entire asynchronous code pointless.</p>

<p>Note however that using synchronous library with async is probably a bad idea. You won't get much and yet you complicate the code alot.</p>

python asynchronous async-await python-3.5