what's Python asyncio.Lock() for?

Question

Is it because coroutines may be preempted in the future? Or it allows people to use yield from in critical section (which IMO shouldn't be encouraged)?

question from:https://stackoverflow.com/questions/25799576/whats-python-asyncio-lock-for

Answer 1

You use it for the same reason you'd use a lock in threaded code: to protect a critical section. asyncio is primarily meant for use in single-threaded code, but there is still concurrent execution happening (any time you hit a yield from or await), which means sometimes you need synchronization.

For example, consider a function that fetches some data from a web server, and then caches the results:

async def get_stuff(url):
    if url in cache:
        return cache[url]
    stuff = await aiohttp.request('GET', url)
    cache[url] = stuff
    return stuff

Now assume that you've got multiple co-routines running concurrently that might potentially need to use the return value of get_stuff:

async def parse_stuff():
    stuff = await get_stuff()
    # do some parsing

async def use_stuff():
    stuff = await get_stuff()
    # use stuff to do something interesting

async def do_work():
     out = await aiohttp.request("www.awebsite.com")
     # do some work with out


loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(
    parse_stuff(),
    use_stuff(),
    do_work(),
))

Now, pretend that fetching data from url is slow. If both parse_stuff and use_stuff run concurrently, each will be hit with the full cost of going over the network to fetch stuff. If you protect the method with a lock, you avoid this:

stuff_lock = asyncio.Lock()

async def get_stuff(url):
    async with stuff_lock:
        if url in cache:
            return cache[url]
        stuff = await aiohttp.request('GET', url)
        cache[url] = stuff
        return stuff

One other thing to note is that while one coroutine is inside get_stuff, making the aiohttp call, and another waits on stuff_lock, a third coroutine that doesn't need to call get_stuff at all can also be running, without being affected by the coroutine blocking on the Lock.

Obviously this example is a little bit contrived, but hopefully it gives you an idea of why asyncio.Lock can useful; it allows you to protect a critical section, without blocking other coroutines from running which don't need access to that critical section.