amazon web services - Fixing Squid configuration for caching proxy?

Question

I am trying to have a setup where Squid is going to act as a transparent forward proxy, with caching enabled. I am leaning on a setup like here: https://aws.amazon.com/blogs/security/how-to-add-dns-filtering-to-your-nat-instance-with-squid/

The issue I have is, when I configure everything, the test server that is going through proxy is not actually caching anything, while if I try a test from the proxy itself (using squidclient) it does. So when the test server goes for a picture I have stored somewhere in the cloud, the squid access log shows "TCP_TUNNEL/200". But when I try from the proxy itself with squidclient tool, I get "TCP_MEM_HIT/200" (the first time it was miss, before it was cached), so caching works properly.

I have the rerouting added to IP tables, source and destination check is disabled (AWS setup), and overall, traffic is going as it should. I assume I need to make some changes in the configuration, as that part is where I had to copy most of the stuff, and have least experience with. I've generated the certificate as per the doc, and my config is mostly the same as the one there:

visible_hostname squid
cache_dir ufs /squid/cache 10000 16 256

# Handle HTTP requests
http_port 3128
http_port 3129 intercept
acl allowed_http_sites dstdomain .amazonaws.com
http_access allow allowed_http_sites

# Handle HTTPS requests
https_port 3130 cert=/etc/squid/ssl/squid.pem ssl-bump intercept
acl SSL_port port 443
http_access allow SSL_port
acl allowed_https_sites ssl::server_name .amazonaws.com
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump peek step2 allowed_https_sites
ssl_bump splice step3 allowed_https_sites
ssl_bump terminate step3 all

http_access deny all

The whitelisting works, mostly everything else works too, so the only thing missing is squid not caching things that are requested by the test server and is instead only passing it through. Any idea what configuration changes I need in order to fix this?

question from:https://stackoverflow.com/questions/65883294/fixing-squid-configuration-for-caching-proxy

Answer 1

The reason for this not working when accessing the proxy via the test server, is that you are tunneling the HTTP request. When you tunnel the HTTP request, ie using the CONNECT method (which will automatically happen when you access the image via https), all caching is bypassed. You cannot cache a tunnelled request as Squid can't see the actual data being passed through it - it proxies the return data straight back to the client as TCP packets.