linux - Mismatch between sys.executable and sys.version in Python

Question

There are two Python interpreters installed:

[user@localhost ~]$ /usr/bin/python -V && /usr/local/bin/python -V
Python 2.4.3
Python 2.7.6

Sudo changes PATH for every command it runs as follows:

[user@localhost ~]$ env | grep PATH && sudo env | grep PATH
PATH=/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/user/bin
PATH=/usr/bin:/bin

I run a test script:

[user@localhost ~]$ cat what_python.py
#!/usr/bin/env python

import sys
print sys.executable
print sys.version
[user@localhost ~]$ sudo python what_python.py
/usr/bin/python
2.7.6 (default, Feb 27 2014, 17:05:07) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)]

and get path to Python 2.4.3 in sys.executable and version 2.7.6 reported in sys.version. Clearly sys.executable and sys.version do not match. Taking into account how sudo modifies PATH I can understand the value of sys.executable. However, why does sys.version report version 2.7.6 and not version 2.4.3, which would match usr/bin/python path reported by sys.executable?

This is a follow-up to my question Sudo changes PATH, yet executes the same binary

Answer 1

Both @Graeme

The fact that python may be unable to retrieve this suggests that it is doing its own PATH search (…)

and @twalberg

(…) it looks like sys.executable searches the current PATH instead of resolving argv[0] (or maybe because argv[0] is simpy python in this case...), (…)

were basically right. I was reluctant to believe that Python does something so simple (silly?) as using PATH to locate itself but this is true.

Python's sys module is implemented in Python/sysmodule.c file. As of version 2.7.6, sys.executable is set at line 1422 like this:

 SET_SYS_FROM_STRING("executable",
                     PyString_FromString(Py_GetProgramFullPath()));

Py_GetProgramFullPath() function is defined in file Modules/getpath.c starting from line 701:

char *
Py_GetProgramFullPath(void)
{
    if (!module_search_path)
        calculate_path();
    return progpath;
}

Function calcuate_path() is defined in the same file and contains the following comment:

/* If there is no slash in the argv0 path, then we have to
 * assume python is on the user's $PATH, since there's no
 * other way to find a directory to start the search from.  If
 * $PATH isn't exported, you lose.
 */

As can be seen in my case, one loses also when the first Python on exported $PATH is different than the Python being run.

More information on the process of calculating placement of interpreter's executable can be found at the top of getpath.c file:

Before any searches are done, the location of the executable is determined. If argv[0] has one or more slashes in it, it is used unchanged. Otherwise, it must have been invoked from the shell's path, so we search $PATH for the named executable and use that. If the executable was not found on $PATH (or there was no $PATH environment variable), the original argv[0] string is used.

Next, the executable location is examined to see if it is a symbolic link. If so, the link is chased (correctly interpreting a relative pathname if one is found) and the directory of the link target is used.

Let's make a couple of tests to verify the above:

If argv[0] has one or more slashes in it, it is used unchanged.

[user@localhost ~]$ sudo /usr/local/bin/python what_python.py
/usr/local/bin/python
2.7.6 (default, Feb 27 2014, 17:05:07) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)]

Ok.

If the executable was not found on $PATH (or there was no $PATH environment variable), the original argv[0] string is used.

[user@localhost ~]$ sudo PATH= python what_python.py
<empty line>
2.7.6 (default, Feb 27 2014, 17:05:07) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)]

Wrong. In this case statement from sys module's documentation is true – If Python is unable to retrieve the real path to its executable, sys.executable will be an empty string or None. .

Let's see if adding location of python's binary back to the PATH (after sudo had removed it) fixes the problem:

[user@localhost ~]$ sudo PATH=$PATH python what_python.py
/usr/local/bin/python
2.7.6 (default, Feb 27 2014, 17:05:07) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)]

It does.

Related:

• Python issue 7774 – sys.executable: wrong location if zeroth command argument is modified.
• Python issue 10835 – sys.executable default and altinstall
• python-dev mailing list thread – towards a stricter definition of sys.executable
• Stackoverflow question – how to find the location of the executable in C