python - Open file for reading and writing with truncate

Question

How do I open a file for reading and writing that is also truncated to 0?

I've tried:

f = open(PATH, 'w+')

I'm unable to read from the file (f.read() returns an empty string), however.

Also:

f = os.fdopen(os.open(PATH, os.O_RDWR | os.O_TRUNC), 'r+')

Same behavious as the previous code.

Answer 1

Here's a table of modes:

mode-string  | truncate? | create? | what's allowed
-------------+-----------+---------+----------------
     r       |    no     |  no     |  reading (only)
     w       |    yes    |  yes    |  writing (only)
     a       |    no     |  yes    |  writing (only), auto-appends
     r+      |    no     |  no     |  read and write
     w+      |    yes    |  yes    |  read and write
     a+      |    no     |  yes    |  read and (auto-appending) write

Note that there's a missing mode ("read and non-auto-append write, that will not truncate but will create"). If you want that one you must use the os functions.

(With all of these, add b to the end of the mode sequence to operate on bytes. The behavior without b depends on Python2 vs Python3 and also the universal_newlines option.)

In addition to all of this, switching from "read mode" to "write mode", or vice versa, on any file opened with the + update-mode, requires a seek operation in most cases. See this answer for more.

---

Edit: here are the various os operations for opening files as defined for Unix-like systems:

os.open(path, flags, mode)

The path argument is pretty obvious, and the mode argument is used only if the file is going to be created (hence may be omitted if you leave out the os.O_CREAT flag). If you do supply mode, the most common value is 0666 which corresponds to rw-rw-rw-. The user's "umask" (see os.umask) will take away the unwanted permissions, e.g., a umask of 002 takes away the final write bit, resuling in rw-rw-r--, while 077 takes away all but the initial rw-. The two other common modes are 0777 (rwxrwxrwx for executable files) and 0600 (rw------- for user-private files, such as temporary files when working with the user's email).

The flags value should include exactly one of os.O_RDONLY, os.O_WRONLY, or os.O_RDWR: open for reading (only), writing (only), or both.

To these, you can add¹ os.O_CREAT, meaning "create file if it does not exist"; os.O_TRUNC, meaning "truncate file to zero bytes in length immediately"; and/or os.O_APPEND, meaning "all os-level write operations implicitly seek to the current end-of-file just before writing". There may (depending on OS flavor) be more flags such as O_EXCL, O_NDELAY, O_NOFOLLOW and more. (The one that is perhaps most commonly useful and dependable is O_EXCL, which means "fail if this would open an existing file" and hence is only really useful when combined with O_CREAT. Using the two together, you can make a new file that you can guarantee no other cooperating process in the system is also using. The O_NOFOLLOW flag, if it exists, is also useful in some security contexts, to avoid symlink traps.)

In all cases, given whatever read/write settings you supply at the os level—i.e., O_RDONLY, O_WRONLY, or O_RDWR—if you then wrap the file descriptor into a Python stream with os.fdopen, you cannot gain additional powers that you did not grant yourself at open time. You can only subtract some away, i.e., you can open with O_RDWR but then fdopen the stream as either read- or write-only. In addition, when using fdopen with update mode, the annoying restriction of requiring seek or flush operations between different I/O directions remains in place.

Note that the a (append) mode at the fdopen level, if it's obeyed at all, is "less powerful" than the O_APPEND mode (or using a mode with open or io.open, all of which result in setting the underlying O_APPEND mode). This mostly has to do with cooperating processes that share a file: if two or more such processes have files opened with O_APPEND, as long as their writes are "small enough" (the details vary again with OSes), their writes will not intermingle. If the files are merely opened as O_WRONLY or O_RDWR, two cooperating processes can call the underlying seek function and then the underlying write function, but if the two are racing each other, one may overwrite the other's data.

(Some flags, including O_APPEND, can be switched off and on using fcntl (from import fcntl) with the F_SETFL argument. This was a relatively late addition to Python2.)

---

¹"Add" here can mean literal addition, but using bitwise or | operations is more conventional. That is, os.open(os.O_RDWR | os.O_CREAT | os.O_TRUNC, 0666), not os.open(os.O_RDWR + os.O_CREAT + os.O_TRUNC, 0666).