io - Permission denied doing os.mkdir(d) after running shutil.rmtree(d) in Python

Question

Very often in the Windows 7 console if I run a python program twice very quickly that does

if os.path.isdir(d):
    shutil.rmtree(d)
if not os.path.exists(d):
    os.mkdir(d)

where d is the name of a directory with many files, I get a "Permission denied" for the mkdir command. But if I run once, then wait some seconds, then run again I do not get such error. What is the problem here?

Answer 1

There are three things that come to mind:

1. Windows itself delays some file operations in order to preserve metadata. If you for example rename a file and create another one in its location, Windows has a time-window where things like ACLs are transferred to the new file. This is a "feature" to preserve this metadata even for programs that write a new file before deleting the old one, in order to not loose data when something fails in the middle.

2. Malware scanners sometimes hook into filesystem operations and perform a scan on files, searching for malware (or government-critic texts, if you're paranoid, and maybe even if you're not paranoid). During that scan, some other accesses to the file are blocked.

3. Lastly, I'm not sure how shutil.rmtree() is implemented, but under Windows, some tree operations are actually implemented not by the OS core but by the shell (i.e. Explorer) and they could be executed asynchronously, which would explain a short time window in which the path is still blocked even though the call already returned.

I believe that e.g. Subversion or rather the Apache Portable Runtime stumbled across the same problem and worked around it by simply retrying with a delay. This solution doesn't win a beauty contest, but it seems to Do The Job(tm).