Disclaimer: this is not a duplicate!
I'm using requests library, monkey patched under GAE standard environment and everything works flawlessly, except the times when I'm adding the extra proxy
argument to the request call. This proxy thing implies tunneling and socket operations which are disabled by default by the standard configuration of both development and deploy environments:
NotImplementedError: HTTP CONNECT Tunnelling is not supported
https://pastebin.com/YDsG9we7
Nice, so we need to enable real sockets support under our GAE platform, therefore adding:
env_variables:
GAE_USE_SOCKETS_HTTPLIB : "True"
And even without doing the requests toolbelt monkey patch (not using urlfetch at all), we get a permission denied: https://pastebin.com/ifBFKi3K
This usually happens when the sandbox thinks that you're accessing a forbidden server location or port, but the proxy is fully functional and guess what, I've tested the very same code (without any added/removed functionality due to development or deploy server as current environment) and it works just as expected on the deploy, without any kind of Permission denied
errors, by making the request completely successfull (if we try without httplib sockets enabled, we still get that Tunnelling is not supported
error under both scenarios and this is the expected behavior).
Now, I've got a working solution on the deploy (which is what it really counts), but this is not enough for me, I just want to test these requests through proxies under my development server and environment too, so what I've tried to do is the following:
- let's import our own standard sockets library by applying these: "ImportError: No module named _ssl" with dev_appserver.py from Google App Engine
- can't monkey patch the sandbox.py file for white-listing
_sockets
and _ssl
etc., no problem, I've hard-patched that directly on disk into the original file
- replacing into memory the default sockets, httplib imports etc. with the standard pythonic ones
- ISSUE:
ValueError: select only supported on socket objects.
https://pastebin.com/gHFM6QiF ; and this is somehow expected to happen, as the expected socket object is different from the expected one due to the standard sockets
import but GAE's own select
(and by removing select from the interpreter's memory will just cause more issues or not work at all), so I want to replace the select
module (which is a built-in) too with the standard pythonic one, how can I do that?
Nevermind...
Is there any method through which this nightmare will end and still using the standard environment? If it works on the deploy server without any patches at all, why shouldn't also work with the dev_appserver tool too?
I just don't want to do any more patches, I just want to get past that Permission denied
error which is not thrown under the remote deploy server, using the very same code logic, no matter if we use requests, urlfetch, httplib, sockets etc. (just the proxy's required tunneling thing).
See Question&Answers more detail:
os