First off, apologies for what I'm sure will be obvious is my rudimentary understanding of bash and shells and subprocesses.
I am trying to use Python to automate calls to a program called Freesurfer (actually, the subprogram I'm calling is called recon-all.)
If I were doing this directly at the command line, I'd "source" a script called mySetUpFreeSurfer.sh that does nothing but set three environment variables, and then "source" another script, FreeSurferEnv.sh. FreesurferEnv.sh doesn't seem to me to do anything but set a lot of environment variables and echo some stuff to the terminal, but it's more complicated than the other bash script, so I'm not sure of that.
Here is what I have right now:
from subprocess import Popen, PIPE, call, check_output
import os
root = "/media/foo/"
#I got this function from another Stack Overflow question.
def source(script, update=1):
pipe = Popen(". %s; env" % script, stdout=PIPE, shell=True)
data = pipe.communicate()[0]
env = dict((line.split("=", 1) for line in data.splitlines()))
if update:
os.environ.update(env)
return env
source('~/scripts/mySetUpFreeSurfer.sh')
source('/usr/local/freesurfer/FreeSurferEnv.sh')
for sub_dir in os.listdir(root):
sub = "s" + sub_dir[0:4]
anat_dir = os.path.join(root, sub_dir, "anatomical")
for directory in os.listdir(anat_dir):
time_dir = os.path.join(anat_dir, directory)
for d in os.listdir(time_dir):
dicoms_dir = os.path.join(time_dir, d, 'dicoms')
dicom_list = os.listdir(dicoms_dir)
dicom = dicom_list[0]
path = os.path.join(dicoms_dir, dicom)
cmd1 = "recon-all -i " + path + " -subjid " + sub
check_output(cmd1, shell=True)
call(cmd1, shell=True)
cmd2 = "recon-all -all -subjid " + sub,
call(cmd2, shell=True)
This is failing:
Traceback (most recent call last):
File "/home/katie/scripts/autoReconSO.py", line 28, in <module>
check_output(cmd1, shell=True)
File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command 'recon-all -i /media/foo/bar -subjid s1001' returned non-zero exit status 127
I maybe understand why this is. My "calls" later in the script are raising new subprocesses that do not inherit environment variables from the processes that are raised by invocation of the source() function. I have done a number of things to try to confirm my understanding. One example -- I put these lines:
mkdir ~/testFreeSurferEnv
export TEST_ENV_VAR=~/testFreeSurferEnv
in the FreeSurferEnv.sh script. The directory gets made just fine, but in the Python script this:
cmd = 'mkdir $TEST_ENV_VAR/test'
check_output(cmd, shell=True)
fails like this:
File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command 'mkdir $TEST_ENV_VAR/test' returned non-zero exit status 1
QUESTION:
How can I make the subprocess that runs "recon-all" inherit the environment variables it needs? Or how can I do everything I need to do -- run the scripts to set the environment variables, and call recon-all, in the same process? Or should I approach the problem another way? Or do I likely misunderstand the problem?
See Question&Answers more detail:
os