python - How does dask.delayed handle mutable inputs?

Question

If I have an mutable object, let's say for example a dict, how does dask handle passing that as an input to delayed functions? Specifically if I make updates to the dict between delayed calls?

I tried the following example which seems to suggest that some copying is going on but can you elaborate what exactly dask is doing?

In [3]: from dask import delayed

In [4]: x = {}

In [5]: foo = delayed(print)

In [6]: foo(x)
Out[6]: Delayed('print-73930550-94a6-43f9-80ab-072bc88c2b88')

In [7]: foo(x).compute()
{}

In [8]: p1 = foo(x)

In [9]: x['a'] = 1

In [10]: p2 = foo(x)

In [11]: p1.compute()
{}

In [12]: p2.compute()
{'a': 1}

Answer 1

Dask does not support mutable inputs. Dask expects inputs to not change. Dask also expects functions to not mutate inputs inplace.

It turns out to be hard to support both mutation and resilience at the same time.

In this case it looks like dask has deconstructed your dictionary into another object. Dictionaries are special in this case. I would not expect this behavior for most mutable objects

In [1]: from dask import delayed

In [2]: x = {}

In [3]: foo = delayed(print)

In [4]: p1 = foo(x)

In [5]: dict(p1.__dask_graph__())
Out[5]: {'print-26d52543-57fc-4873-9722-1a8fd2f1641c': (<function print>, (dict, []))}