ssl - NGINX Proxy not working with HTTP/2 and gRPC

Question

I am trying to use NGINX as an "API Gateway" into my gRPC services - all within a Kubernetes Cluster. A Typescript React App is just making calls via the grpc-web module to an Envoy proxy, then to the API NGINX Proxy. (I have tested that end of the stack - and I'm 100% sure that envoy works fine).

NOTE: I may be making a mistake NOT using TLS with the Envoy Proxy (Which is the 'client' to NGINX) - so please comment if that's the mistake I'm making

For this to work with my gRPC endpoints, I need to enable HTTP/2 proxying (this is required for gRPC to work - it must be over HTTP/2). And so, following the official NGINX Documentation which is here: https://www.nginx.com/blog/nginx-1-13-10-grpc/ , my nginx.conf file looks like:

worker_processes auto;

events {}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent"';

    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''        close;
    }

    server {
        listen 1449 ssl http2;

        ssl_protocols TLSv1.2;
        ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
        ssl_prefer_server_ciphers on;

        ssl_certificate ./server.crt;
        ssl_certificate_key ./server.key;

        location /com.example.grpcService {
            grpc_pass grpcs://api-grpc-server:9090;

            proxy_buffer_size          512k;
            proxy_buffers              4 256k;
            proxy_busy_buffers_size    512k;
            grpc_set_header Upgrade $http_upgrade;
            grpc_set_header Connection "Upgrade";
            grpc_set_header Connection keep-alive;
            grpc_set_header Host $host:$server_port;
            grpc_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            grpc_set_header X-Forwarded-Proto $scheme;
        }
    }
}

I also heard from another forum that you MUST use TLS/SSL with HTTP/2 or it won't work, so I first tried it without - it didn't work. Then I tried it with the generated SSL certificates and it looks like I'm still getting a 400 error from the proxied service. The log looks like:

172.17.0.17 - - [05/Jan/2021:18:16:23 +0000] "PRI * HTTP/2.0" 400 157 "-" "-"

I have used OpenSSL for the certificates which resulted in .crt and .key files being generated - which I then used for BOTH my Spring Boot gRPC Server & NGINX Proxy. My OpenSSL version is OpenSSL 1.1.1c 28 May 2019.

I am using those same certificates on the actual gRPC Server itself, this looks like:

@Component
public class GrpcServerRunner implements CommandLineRunner, DisposableBean {

    private final ConfigurableApplicationContext applicationContext;
    private Server server;

    public GrpcServerRunner(@Autowired ConfigurableApplicationContext applicationContext) {
        this.applicationContext = applicationContext;
    }

    @Override
    public void run(String... args) throws Exception {

        File cert = new File("~/etc/ssl/server.crt");
        File key = new File("~/etc/ssl/server.key");

        BindableService service = applicationContext.getBean("grpcService", BindableService.class);
        server = ServerBuilder.forPort(9090).useTransportSecurity(cert, key).addService(service).build();

        runSever();
    }

    private void runSever() {
        Thread thread = new Thread(() -> {
            try {
                server.awaitTermination();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        });
        thread.setDaemon(false);
        thread.start();
    }

    @Override
    public void destroy() {
        server.shutdown();
    }

}

I'd really appreciate any help, questions, feedback or solutions to this problem - so thanks in advance.

Answer 1

等待大神答复