0

I have an existing setup using two cascaded nginx instances:

  1. the outer nginx is just doing TLS termination and forward to inner nginx
  2. the inner nginx is routing the requests to different applications (REST APIs)

Until now we were only using HTML pages and REST APIs. The routing of the outer nginx is pretty simple, it is mainly serving static HTML, the other traffic is forwarded to the inner nginx.

Now we want to support also grpc APIs. The problem with grpc is that the path of the requests are defined by the grpc interface proto, e.g. "acme.myapi/MyMethod". The routing of the requests of the outer nginx shall not depend on that interface definitions because the outer nginx shall not need to be changed when new grpc interfaces are added and there can even be an overlap in the url path between REST API and grpc, e.g both could begin with "/acme/". Also we have no control over the name of the proto package names and therefore the url path.

The grpc routing of the inner nginx is using the :authority header for the routing to the grpc applications. The configuration is updated dynamically when new applications are added.

All grpc calls shall be redirected to a specific endpoint of the inner nginx (grpc://127.0.0.1:50051), the REST API calls shall use another endpoint (https://127.0.0.1:4443/) of the inner nginx instance.

The easiest way from the perspective of the nginx configuration would be to use two different endpoints (different ports) also for the outer nginx. Unfortunately the organizational effort to open another port in the firewalls would be very high and is not accepted, so I need to use the same endpoint for all protocols.

This is the config I currently use and it works with the problems describe above:

location ~ ^acme\. { # "acme" is the grpc packagename
    grpc_pass grpc://127.0.0.1:50051;
    grpc_set_header "Host" $host;
    grpc_read_timeout 1200s;
    grpc_send_timeout 1200s;
    client_body_timeout 1200s;
    client_max_body_size 0;
}
location / {
    proxy_pass https://127.0.0.1:4443/;
    [...] # setting lots of other options
}

Here is the config I'd like to use (simplified to the minimum) but according to If is Evil… this cannot be used:

location / {
  if ($http_content_type ~ ^application/grpc)
  {
    grpc_pass grpc://127.0.0.1:50051;
    grpc_set_header "Host" $host;
    grpc_read_timeout 1200s;
    grpc_send_timeout 1200s;
    client_body_timeout 1200s;
    client_max_body_size 0;
  }

normally I would use 'else' here, but this is not supported


if ($http_content_type !~ ^application/grpc)
  {
    # also set other REST API specific config options here, left out for brevity
    proxy_pass https://127.0.0.1:4443/;
  }

I get the following error from nginx:

nginx: [emerg] "grpc_set_header" directive is not allowed here in /etc/nginx/conf.d/my.conf.locations:80

In the end I need to be able to use the $http_content_type as a selector instead (or additionally) to the url path.

Is there any good solution for this requirements?

Michael Dreher
  • 101

2 Answers2

0

I think I found a good solution and using if only for rewrite which is allowed. I need to map any conditions to a url path, so I can solely use location statement to match it. In my case I map the content-type ~ ^application/grpc header to the location /grpc and in the location section I remove the /grpc prefix.

    if ($http_content_type ~ ^application/grpc) {
        rewrite "(.*)" "/grpc$1" break; # add "/grpc" to url
    }
    location /grpc/ {
        internal; # not reachable from outside, just for the rewritten location
        rewrite "^/grpc(.*)" "$1" break; # remove "/grpc" from url
        grpc_pass grpc://127.0.0.1:4080;
        #grpc_set_header ":authority" $host; # unfortunately this doesn't work
        grpc_set_header "Host" $host; # workaround because ":authority" can't be set
        grpc_read_timeout 3153600000s;
        grpc_send_timeout 3153600000s;
        client_body_timeout 3153600000s;
        client_max_body_size 0;
    }

The location section for the REST API is not affected.

First test is working, but I need to dig deeper if I miss any side effects.

Michael Dreher
  • 101
0

I stumbled upon the same problem and found another solution:

location /somepath {
    error_page 418 = @grpc_backend;
    if ($http_content_type ~ ^application/grpc) { return 418; }

proxy_pass http://unix:/dev/shm/backend_1.socket;



}


location @grpc_backend {
    grpc_pass unix:/dev/shm/backend_2.socket;
}
alvinma
  • 1