python - When asyncio task gets stored after creation, exceptions from task get muted

Question

I was using asyncio for a project, and encountered this strange behavior.

import asyncio

def schedule_something():
    global f
    tsk = asyncio.async(do_something())
    f = tsk #If this line is commented out, exceptions can be heard.

@asyncio.coroutine
def do_something():
    raise Exception()

loop = asyncio.get_event_loop()
loop.call_soon(schedule_something)
loop.run_forever()
loop.close()

For some reason, storing the resulting task when you call asyncio.async() stops exceptions from doing anything.

Could someone shed some light on this situation? I need to have a way to catch exceptions in my current project.

Answer 1

This is because the exception only gets raised if the Task is destroyed without ever having its result retrieved. When you assigned the Task to a global variable, it will always have an active reference, and therefore never be destroyed. There's a docstring in asyncio/futures.py that goes into detail on this:

class _TracebackLogger:
    """Helper to log a traceback upon destruction if not cleared.

    This solves a nasty problem with Futures and Tasks that have an
    exception set: if nobody asks for the exception, the exception is
    never logged.  This violates the Zen of Python: 'Errors should
    never pass silently.  Unless explicitly silenced.'

    However, we don't want to log the exception as soon as
    set_exception() is called: if the calling code is written
    properly, it will get the exception and handle it properly.  But
    we *do* want to log it if result() or exception() was never called
    -- otherwise developers waste a lot of time wondering why their
    buggy code fails silently.

    An earlier attempt added a __del__() method to the Future class
    itself, but this backfired because the presence of __del__()
    prevents garbage collection from breaking cycles.  A way out of
    this catch-22 is to avoid having a __del__() method on the Future
    class itself, but instead to have a reference to a helper object
    with a __del__() method that logs the traceback, where we ensure
    that the helper object doesn't participate in cycles, and only the
    Future has a reference to it.

    The helper object is added when set_exception() is called.  When
    the Future is collected, and the helper is present, the helper
    object is also collected, and its __del__() method will log the
    traceback.  When the Future's result() or exception() method is
    called (and a helper object is present), it removes the the helper
    object, after calling its clear() method to prevent it from
    logging.

If you want to see/handle the exception, just use add_done_callback to handle the result of the task, and do whatever is necessary when you get an exception:

import asyncio

def handle_result(fut):
    if fut.exception():
        fut.result()  # This will raise the exception.

def schedule_something():
    global f
    tsk = asyncio.async(do_something())
    tsk.add_done_callback(handle_result)
    f = tsk

@asyncio.coroutine
def do_something():
    raise Exception()

loop = asyncio.get_event_loop()
loop.call_soon(schedule_something)
loop.run_forever()
loop.close()