Pomerium / nginx with google

reaching out to the community for some help here, reached a wall and unsure how to get passed it.

OS: Ubuntu 20.04 LTS, running docker with docker compose and portainer and pomerium with nginx using the recipe from here


pomerium_debug: true
address: :80
cookie_secret: cookie_secret
shared_secret: shared_secret

idp_provider: "google"
idp_provider_url: https://accounts.google.com
idp_client_id: clinet_id
idp_client_secret: client_secret
idp_service_account: idp_service_Account

insecure_server: true
#forward_auth_url: http://fwdauth.localhost.pomerium.io
forward_auth_url: http://fwdauth.domain.com
authenticate_service_url: https://authenticate.domain.com
#authenticate_service_url: https://authenticate.localhost.pomerium.io

  - from: https://verify.domain.com
# - from: https://verify.localhost.pomerium.io
    to: https://verify
#      - pomerium.com
#      - gmail.com
      - numat-tech.com
    pass_identity_headers: true

#  - from: https://verify.localhost.pomerium.io
#    to: http://verify
#    policy:
#      - allowed_domains:
#         or:
#            - email:
#                is: email
#    pass_identity_headers: true

#  - from: https://verify.localhost.pomerium.io
#    to: https://httpbin
#    allowed_domains:
#      - pomerium.com
#      - gmail.com
#    pass_identity_headers: true


version: "3"
    # to emulate nginx-ingress behavior, use openresty which comes with 'escaped_request_uri'
    # pre-compiled. Also uncomment lines marked `uncomment to emulate nginx-ingress behavior`
    # in the nginx `.conf` configuration files.
    # image: openresty/openresty
    image: nginx
    restart: unless-stopped
      - "80:80"
      - "443:443"
      - ./verify.conf:/etc/nginx/conf.d/verify.conf
      - ./pomerium.conf:/etc/nginx/conf.d/pomerium.conf

      - ./fullchain.pem:/etc/nginx/nginx.pem
      - ./privkey.pem:/etc/nginx/nginx-key.pem

#     - ./_wildcard.localhost.pomerium.io.pem:/etc/nginx/nginx.pem
#     - ./_wildcard.localhost.pomerium.io-key.pem:/etc/nginx/nginx-key.pem
      - ./proxy.conf:/etc/nginx/proxy.conf

    image: pomerium/verify
      - 80
    image: pomerium/pomerium:latest
      - ./config.yaml:/pomerium/config.yaml:ro
      - 80


# Pomerium endpoint
server {
    listen 443 ssl;
#   server_name  authenticate.localhost.pomerium.io fwdauth.localhost.pomerium.io;
    server_name  authenticate.domain.com fwdauth.domain.com;
    ssl_certificate /etc/nginx/nginx.pem;
    ssl_certificate_key /etc/nginx/nginx-key.pem;

    location / {
      proxy_pass http://pomerium;
      include /etc/nginx/proxy.conf;

# Define an upstream so that we don't need resolvers when we use variables in proxy_pass directives
# https://stackoverflow.com/questions/17685674/nginx-proxy-pass-with-remote-addr
upstream pomerium {
    server pomerium;


# Protected application
server {
  listen 80;
  listen 443 ssl http2;

# server_name verify.localhost.pomerium.io;
  server_name domain.com;
  ssl_certificate /etc/nginx/nginx.pem;
  ssl_certificate_key /etc/nginx/nginx-key.pem;

  location = /ext_authz {

    proxy_pass_request_body off;
    proxy_set_header Content-Length "";
    proxy_set_header X-Forwarded-Proto "";

    proxy_set_header Host fwdauth.domain.com;
#   proxy_set_header Host fwdauth.localhost.pomerium.io;
    proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
    proxy_set_header X-Original-Method $request_method;
    proxy_set_header X-Real-IP $remote_addr;

    proxy_set_header X-Forwarded-For $remote_addr;

    proxy_set_header X-Auth-Request-Redirect $request_uri;

    proxy_buffering off;

    proxy_buffer_size 4k;
    proxy_buffers 4 4k;
    proxy_request_buffering on;
    proxy_http_version 1.1;

    proxy_ssl_server_name on;
    proxy_pass_request_headers on;

    client_max_body_size 1m;

    # Pass the extracted client certificate to the auth provider

    set $target http://pomerium/verify?uri=$scheme://$http_host$request_uri;

    # uncomment to emulate nginx-ingress behavior
    # set $target http://pomerium/verify?uri=$scheme://$http_host$request_uri&rd=$pass_access_scheme://$http_host$escaped_request_uri;
    proxy_pass $target;

  location @authredirect {
    add_header Set-Cookie $auth_cookie;

    # uncomment to emulate nginx-ingress behavior
    # return 302 https://fwdauth.localhost.pomerium.io/?uri=$scheme://$host$request_uri&rd=$pass_access_scheme://$http_host$escaped_request_uri;

    return 302
#     https://fwdauth.localhost.pomerium.io/?uri=$scheme://$host$request_uri;

  location / {
    proxy_pass http://verify;

    include /etc/nginx/proxy.conf;
    # If we get a 401, respond with a named location
    error_page 401 = @authredirect;
    # this location requires authentication
    auth_request /ext_authz;
    auth_request_set $auth_cookie $upstream_http_set_cookie;
    add_header Set-Cookie $auth_cookie;

    # pass JWT assertion upstream
    auth_request_set $auth_resp_x_pomerium_jwt_assertion $upstream_http_x_pomerium_jwt_assertion;
    proxy_set_header x-pomerium-jwt-assertion $auth_resp_x_pomerium_jwt_assertion;

when I navigate to https://verify.domain.com, I am redirected to google to sign in, and then I am being shown this and have no idea how to fix

Hi @XxEnigmaticxX, thanks for taking the time to provide all the relevant config files.

Full disclosure: I haven’t personally done a forward-auth config yet, so take anything I say with a grain of salt. But there’s two avenues of troubleshooting I’d suggest:

  • I looked through the example config you linked as well as our guide on the topic, and I couldn’t find another example that included the lines:

    auth_request_set $auth_resp_x_pomerium_jwt_assertion $upstream_http_x_pomerium_jwt_assertion;
    proxy_set_header x-pomerium-jwt-assertion $auth_resp_x_pomerium_jwt_assertion;

    Where are you seeing these config options referenced?

  • You said on Reddit that it all works well when you use the test domain, but not when you switch it to your actual domain. This suggests a DNS issue, although the results of the verify page do not. Can you provide more context as to what you saw when using the test domain, how you spoofed the DNS, and what was changed in the config?

Anyways, that’s what I, a lowly technical writer, could come up with. I’ll wrangle up a subject matter expert to weigh in with a little more authority.

@XxEnigmaticxX are you missing the signing key? Settings | Pomerium

I think without it, the JWKS endpoint won’t have any keys so it can’t validate the assertion header.

thank you for your help, i made some changes on my end from the original reddit post to this post here. in regards to your question, I found the config options here, while trying to get the verify instance to work with nginx using the recipe from above

forgive me for my ignorance here, and thank you for all your help,but now sure how to set that? do I run that command in the nginx container, and the output is what I use as the value for the signing key? where would I set the value for signing key?

# Generates an P-256 (ES256) signing key
openssl ecparam  -genkey  -name prime256v1  -noout  -out ec_private.pem
# careful! this will output your private key in terminal
cat ec_private.pem | base64

alright so i think i figured it out.

i went into the running nginx container and ran the code below

# Generates an P-256 (ES256) signing key
openssl ecparam  -genkey  -name prime256v1  -noout  -out ec_private.pem
# careful! this will output your private key in terminal
cat ec_private.pem | base64

that gave me back a long ass string.
i then added

signing_key: long ass string

to my config.yaml file, now when I head to verify.domain.com my identity can be validated

1 Like

Excellent! That’s my bad, I saw couldn't get json web key and thought “token” instead of key.

While we’re here, it’s worth asking: What’s the use case for using forward-auth to Nginx? Pomerium is a first class proxy / ingress controller on its own. Do you have a use-case that requires this setup?

this may not be the correct answer, but have no idea if pomerium can also act as a web-server. my use case is we have people (external + internal) that need to access our internal resources. I’m pretty familiar with NGINX and saw yall had a basic setup for pomerium to leverage NGINX so decided to stand it up and take it for a spin

Pomerium itself does not act as a web server, so if the service you’re protecting needs an independent web server to run Nginx is a fine choice. But the setup you’ve tested is more complicated than it has to be. The verify test app, for example, has its own web server built in so you can point a route in Pomerium directly at the docker container.

Another example is our Grafana integration guide. Since the application already serves content without an additional web server, you need only tell it where to look for the JWT and JWK in the headers and authenticate endpoint, respectively.

I appreciate having this discussion. It confirms an internal idea we had to clarify this point on all our documentation relating to forward-auth. We can make it more clear that it’s possible to set up Pomerium for an existing scenario where Nginx is handling authentication, but unless you have a specific use case for it you’re probably better off using Pomerium in its default model.

more complicated than it has to be.

story of my life, lol.

1 Like

is there any way i can mark my own support threads as closed?

Good point. This category needs the “Mark solution” option turned on. I’ll get on that.

EDIT: Done.