apache - Issue with imports when using WSGI in EC2 Instance to Host Flask App

Question

I am trying to run a very simple Flask app off an EC2 instance using mod_wsgi. My apache error log keeps showing

"ImportError: No module named pandas, referer: http://xxxxx"

Despite the fact that I have pandas installed. For reference, pip freeze yields me

click==6.7
Flask==0.12
itsdangerous==0.24
Jinja2==2.9.5
MarkupSafe==0.23
numpy==1.12.0
pandas==0.19.2
python-dateutil==2.6.0
pytz==2016.10
scikit-learn==0.18.1
scipy==0.18.1
six==1.10.0
sklearn==0.0
virtualenv==15.1.0
Werkzeug==0.11.15

Previously, I tried using virtualenv (pip freeze of my venv is very similar to what I posed above) and then modifying the .wsgi file to use the virtualenv with the code:

activate_this = '/home/ubuntu/sklearn-env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))

Doing this through the virtualenv gives me a flask app that just times out when I make a simple GET request with nothing useful in the apache error logs and using my local pip gives me ImportErrors with all my Python modules despite the fact that I am able to open a Python2.7 instance and import pandas, flask, etc. with no problem (Python 2.7.12 for reference). I'm completely stumped here, any advice?

EDIT: So I actually fixed the issue I was having with local pip as far as the import error, but now, both my local pip and using my virtualenv just gives me a flask app that hangs forever if I import anything besides flask. If I copy this code in

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
  return 'Hello from Flask!'

if __name__ == '__main__':
  app.run()

it works perfectly fine. I can literally just add import pandas and now my app just spins indefinitely. The only thing coming from the apache error log is

[Thu Feb 23 01:02:52.010864 2017] [wsgi:warn] [pid 11686:tid 140507506435968] mod_wsgi: Compiled for Python/2.7.11. [Thu Feb 23 01:02:52.010902 2017] [wsgi:warn] [pid 11686:tid 140507506435968] mod_wsgi: Runtime using Python/2.7.12. [Thu Feb 23 01:02:52.011483 2017] [mpm_event:notice] [pid 11686:tid 140507506435968] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.3.0 Python/2.7.12 configured -- resuming normal operations [Thu Feb 23 01:02:52.011498 2017] [core:notice] [pid 11686:tid 140507506435968] AH00094: Command line: '/usr/sbin/apache2'

which I was under the impression, wasn't actually breaking anything.

My .conf file is just modifying the default one.

<VirtualHost *:80>
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        WSGIDaemonProcess classification-poc threads=5
        WSGIScriptAlias / /var/www/html/classification-poc/server.wsgi

        <Directory classification-poc>
    WSGIProcessGroup classification-poc
    WSGIApplicationGroup %{GLOBAL}
   Order deny,allow
    Allow from all
        </Directory>

        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined


        #Include conf-available/serve-cgi-bin.conf
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

My server.wsgi file is just

import sys
sys.path.insert(0, '/var/www/html/classification-poc')
from testserver import app as application

when I am not using virtualenv. When I try it with virtualenv, I just add

activate_this = '/home/ubuntu/sklearn-env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))

to the top of the file.

Answer 1

You must set:

WSGIApplicationGroup %{GLOBAL}

in mod_wsgi configuration when using numpy and related packages. If you don't it can hang because numpy has not been implemented in a way that it will work in Python sub interpreters.

See:

• http://modwsgi.readthedocs.io/en/develop/user-guides/application-issues.html#python-simplified-gil-state-api