Programatic Access JWT expires too early

Support
1

I’ve implemented a programmatic access workflow as described here. After receiving the session callback, I capture the JWT from the URL and add it to subsequent API requests in an auth header like Authorization: Pomerium {my-jwt}. It works successfully at first, but starts to get 401’s after 1h. Is this the expected behavior? Is there a config change, or a refresh token workflow, that we should be looking at in order to keep a JWT alive for longer?

What did you expect to happen?

The JWT would continue working for 14h

How’d it happen?

  1. Obtained JWT with programmatic access
  2. Used JWT in Authorization header
  3. Repeatedly used JWT in a test script to connect to API behind the Pomerium proxy - script started receiving 401’s after 1h

What’s your environment like?

  • pomerium/ingress-controller:sha-cdc389c
  • Deployed in kubernetes

What’s your config.yaml?

We’re using the Kubernetes Pomerium CRD so don’t have an explicit config.yaml but I can pick out interesting parts of that spec if needed!

What did you see in the logs?

pod_name="pomerium-7649fd9746-bsbqf" node_name="my-k8s-node" {"level":"error","service":"identity_manager","error":"identity/oidc: user info endpoint: 401 Unauthorized: ","user_id":"00uaFOO","session_id":"ID.BAR","time":"2022-12-19T22:55:43Z","message":"failed to update user info, deleting session"}

Additional context

We’ve tried explicitly setting a expiration with the following, but outcome is the same

spec:
  secrets: pomerium/bootstrap
  authenticate:
    url: https://authenticate.my-domain.io
  identityProvider:
    provider: okta
    url: "https://my-instance.okta.com"
    scopes:
      - openid
      - email
      - profile
      - groups
    secret: pomerium/idp
  cookie:
    expire: "12h"
1 Like
2

The log hints at session expiring on Okta side, please check whether there is a max 1hr session restriction configured in Okta.

Do you experience the same when using a normal login flow?

3

Our Okta session lifetime should last 12h. This does happen with normal login as well.

We also noticed that in the /.pomerium/ “User Details” the “Expires At” timestamp is the expected expiration, but there is an “Exp” field that shows one hour from the time of authorization. The data generating that page seems to have the shorter expiration associated with session.oauthToken.expiresAt, would that likely point us to a particular place we need to configure?

4

I believe Expires At is Pomerium session max time to live.
exp comes from Okta token. Pomerium continuously refreshes user session and user info and apparently it runs into an error with Okta trying to do that:

pod_name="pomerium-7649fd9746-bsbqf" node_name="my-k8s-node" {"level":"error","service":"identity_manager","error":"identity/oidc: user info endpoint: 401 Unauthorized: ","user_id":"00uaFOO","session_id":"ID.BAR","time":"2022-12-19T22:55:43Z","message":"failed to update user info, deleting session"}

when that happens, the Pomerium session cannot be continued and is deleted (as noted in the log message).

You need investigate why the supplied credentials are rejected by Okta while session refresh.

5

I have the same issue with the Google provider @mchaines did you find out why Okta can’t refresh the session?

6

Hi @abelaytzev ,

Do you have an error message from Pomerium?

7

Hi @calebdoxsey I have this error messages in my pomerium v0.18

{"level":"error","config_file_source":"/etc/pomerium/config.yaml","bootstrap":true,"service":"identity_manager","error":"identity/oidc: refresh failed: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\n  \"error\": \"invalid_client\",\n  \"error_description\": \"The OAuth client was not found.\"\n}","user_id":"XXXXXXXXXXXXXX","session_id":"XXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXX","time":"2024-00-00T00:00:00Z","message":"failed to refresh oauth2 token, deleting session"}
{"level":"error","config_file_source":"/etc/pomerium/config.yaml","bootstrap":true,"service":"identity_manager","error":"identity/oidc: user info endpoint: 401 Unauthorized: {\"error\":\"invalid_request\",\"error_description\":\"Invalid Credentials\"}","user_id":"XXXXXXXXXXXXX","session_id":"XXXXX-XXXXX-XXXX-XXXX-XXXXXXXXX","time":"2024-00-00T00:00:00Z","message":"failed to update user info, deleting session"}
8

I think this error means that the idp_client_id is not valid. Are you sure it’s entered correctly?

1 Like
9

Thank you, @calebdoxsey You are right. idp_client_id is not set for the databroker pod, and it fails the refresh token process.