multithreading - Understanding Python contextvars

Question

Regarding the following SO answer . I've made some changes in order to understand the difference between do use Contextvars and don't.

I expect at some point the variable myid gets corrupted but changing the range to a higher number seems doesn't affect at all.

import asyncio
import contextvars

# declare context var
request_id = contextvars.ContextVar('Id of request.')


async def some_inner_coroutine(myid):
    # get value
    print('Processed inner coroutine of myid   : {}'.format(myid))
    print('Processed inner coroutine of request: {}'.format(request_id.get()))
    if myid != request_id.get():
        print("ERROR")


async def some_outer_coroutine(req_id):
    # set value
    request_id.set(req_id)

    await some_inner_coroutine(req_id)

    # get value
    print('Processed outer coroutine of request: {}'.format(request_id.get()))


async def main():
    tasks = []
    for req_id in range(1, 1250):
        tasks.append(asyncio.create_task(some_outer_coroutine(req_id)))

    await asyncio.gather(*tasks)


if __name__ == '__main__':
    asyncio.run(main())

Results

Processed inner coroutine of myid   : 1
Processed inner coroutine of request: 1
Processed outer coroutine of request: 1
Processed inner coroutine of myid   : 2
Processed inner coroutine of request: 2
Processed outer coroutine of request: 2
Processed inner coroutine of myid   : 3
Processed inner coroutine of request: 3
Processed outer coroutine of request: 3
Processed inner coroutine of myid   : 4
Processed inner coroutine of request: 4
Processed outer coroutine of request: 4
...
...
Processed inner coroutine of myid   : 1244
Processed inner coroutine of request: 1244
Processed outer coroutine of request: 1244
Processed inner coroutine of myid   : 1245
Processed inner coroutine of request: 1245
Processed outer coroutine of request: 1245
Processed inner coroutine of myid   : 1246
Processed inner coroutine of request: 1246
Processed outer coroutine of request: 1246
Processed inner coroutine of myid   : 1247
Processed inner coroutine of request: 1247
Processed outer coroutine of request: 1247
Processed inner coroutine of myid   : 1248
Processed inner coroutine of request: 1248
Processed outer coroutine of request: 1248
Processed inner coroutine of myid   : 1249
Processed inner coroutine of request: 1249
Processed outer coroutine of request: 1249

What should I change to see an unexpected behaviour of the variable myid?

Answer 1

Context variables are convenient when you need to pass a variable along the chain of calls so that they share the same context, in the case when this cannot be done through a global variable in case of concurrency. Context variables can be used as an alternative to global variables both in multi-threaded code and in asynchronous (with coroutines).

Context variables are natively supported in asyncio and are ready to be used without any extra configuration. Because when a Task is created it copies the current context and later runs its coroutine in the copied context:

# asyncio/task.py
class Task:
    def __init__(self, coro):
        ...
        # Get the current context snapshot.
        self._context = contextvars.copy_context()
        self._loop.call_soon(self._step, context=self._context)

    def _step(self, exc=None):
        ...
        # Every advance of the wrapped coroutine is done in
        # the task's context.
        self._loop.call_soon(self._step, context=self._context)
        ...

Below is your example showing the corruption of a global variable compared to context vars:

import asyncio
import contextvars

# declare context var
current_request_id_ctx = contextvars.ContextVar('')
current_request_id_global = ''


async def some_inner_coroutine():
    global current_request_id_global

    # simulate some async work
    await asyncio.sleep(0.1)

    # get value
    print('Processed inner coroutine of request: {}'.format(current_request_id_ctx.get()))
    if current_request_id_global != current_request_id_ctx.get():
        print(f"ERROR! global var={current_request_id_global}")


async def some_outer_coroutine(req_id):
    global current_request_id_global

    # set value
    current_request_id_ctx.set(req_id)
    current_request_id_global = req_id

    await some_inner_coroutine()

    # get value
    print('Processed outer coroutine of request: {}
'.format(current_request_id_ctx.get()))


async def main():
    tasks = []
    for req_id in range(1, 10000):
        tasks.append(asyncio.create_task(some_outer_coroutine(req_id)))

    await asyncio.gather(*tasks)


if __name__ == '__main__':
    asyncio.run(main())

Output:

...
Processed inner coroutine of request: 458
ERROR! global var=9999
Processed outer coroutine of request: 458

Processed inner coroutine of request: 459
ERROR! global var=9999
Processed outer coroutine of request: 459
...

---

An example of converting code that uses threading.local() can be found in PЕP 567