Pomerium Auth0/GitHub IdP Grafana JWT does not work

Running the latest version of Pomerium and Grafana (Community stack), Auth0 or GitHub are tested IdP. Grafana is setup to use JWT.

What happened?

Post authentication on the IdP portal, redirection to grafana is not working as expected. Below error is thrown,

{
“message”: “invalid API key”
}

What did you expect to happen?

Post auth it should redirect to Grafana Page

What’s your environment like?

  • Pomerium version (retrieve with pomerium --version): latest (helm installer)
  • Server Operating System/Architecture/Cloud: Ubuntu 20.04, K3s, traefik ingress

What’s your config.yaml?

routes:
    - from: https://grafana.qehnelo.xyz
      to: http://main-grafana.prometheus
      policy:
        - allow:
            or:
              - domain:
                  is: gmail.com
      tls_skip_verify: true 

ingress:
  annotations:
    kubernetes.io/ingress.class: "traefik"
    cert-manager.io/cluster-issuer: letsencrypt-stg
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
    #    ingress.pomerium.io/pass_identity_headers: "true"
  secretName: le-wc-qehnelo.xyz

What did you see in the logs?

message": "invalid API key"

Additional context

Tried both the Git and auth0 IdP and looks same. tried different version and problem seems same

invalid API key is not Pomerium error message.

Also, I notice you’re using forward-auth configuration with Ingress managed by Traefik.
Pomerium provides a first class Ingress controller Ingress Controller | Pomerium