Forward to DOMAIN/.pomerium/

Support
1

Hi there,

What happened?

When I want to access my domain I am always forwarded to the Pomerium user dashboard on DOMAIN/.pomerium/ but when I add some other path to DOMAIN/path I am forwarded to the right content

What did you expect to happen?

I should be forwarded to the landing page of the proxied site.

How’d it happen?

  1. Ran https://www.DOMAIN
  2. Hit enter
  3. Saw pomerium user dashboard on DOMAIN/.pomerium/

What’s your environment like?

  • Pomerium version current version on docker compose
  • Server Operating System/Architecture/Cloud:
    ubuntu 22.04 server

What’s your config.yaml?

routes:
  - from: https://xyz.com
    to: http://localhost:3000
    policy:
      - allow:
          or:
            - domain:
                is: xyz
            - domain:
                is: xyz
    pass_identity_headers: true
1 Like
2
  1. is your IdP login working properly? are you able to authenticate - i.e. (/.pomerium shows you user info?)
  2. just to rule out IdP interference try to see if the route works with public_unauthenticated_access: true?
  3. we would need some logs; note that every response carries an x-request-id in the response header and that request ID is attached to the relevant Pomerium log lines.
3
  1. is your IdP login working properly? are you able to authenticate - i.e. (/.pomerium shows you user info?)

Yeah azure AD as IdP works fine, User Info is shown properly

  1. just to rule out IdP interference try to see if the route works with public_unauthenticated_access: true?

Pretty much the same behaviour. When I access the base domain it forwards to IdP-Provider and then forwards to pomerium dashboard.
When I access non base domaon path e.g. DOMAIN/docs it forwards directly to the in this case not protected site.

  1. we would need some logs; note that every response carries an x-request-id in the response header and that request ID is attached to the relevant Pomerium log lines.

this is the logs that I get on compose:
pomerium-pomerium-1 | {“level”:“info”,“service”:“envoy”,“upstream-cluster”:“pomerium-control-plane-http”,“method”:“GET”,“authority”:“DOMAIN”,“path”:“/”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36”,“referer”:“”,“forwarded-for”:“93.241.69.183”,“request-id”:“8eeeedcc-c9ae-4517-a3f7-58c8e267d7c3”,“duration”:0.896459,“size”:34,“response-code”:302,“response-code-details”:“via_upstream”,“time”:“2023-01-18T10:38:32Z”,“message”:“http-request”}
pomerium-pomerium-1 | {“level”:“info”,“service”:“envoy”,“upstream-cluster”:“pomerium-control-plane-http”,“method”:“GET”,“authority”:“DOMAIN”,“path”:“/.pomerium/”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36”,“referer”:“”,“forwarded-for”:“93.241.69.183”,“request-id”:“f047dca3-b935-4589-8f60-33408ff83942”,“duration”:12.001113,“size”:4320,“response-code”:200,“response-code-details”:“via_upstream”,“time”:“2023-01-18T10:38:32Z”,“message”:“http-request”}

4

Can anyone help? It is probably just a stupid mistake…

I would have to switch to Oauth_proxy or caddy-security instead.

5

this is where the redirect is sent from, however you did not include the adjacent log lines from i.e. authorize service to understand why the redirect was necessary.

When you say current version do you refer to pomerium/pomerium:latest or pomerium/pomerium:main ?

you may always join our public Slack for more interactive troubleshooting.

6

I used latest. Just switched to main and showed the same error.
Do these lines tell you something more?

pomerium-pomerium-1  | {"level":"info","X-Forwarded-For":["93.241.69.183"],"X-Forwarded-Proto":["https"],"ip":"127.0.0.1","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36","request-id":"e5729d20-0d34-4a42-a65e-d95c77751982","error":"Bad Request: internal/sessions: session is not found","idp_id":"IDP_ID_REMOVED","time":"2023-02-07T16:07:40Z","message":"authenticate: session load error"}
pomerium-pomerium-1  | {"level":"info","service":"envoy","upstream-cluster":"pomerium-control-plane-http","method":"GET","authority":"DOMAIN","path":"/","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36","referer":"","forwarded-for":"IP_ADDRESS","request-id":"e748ee77-4f44-4e3d-9902-24124ab67c0f","duration":1.86653,"size":34,"response-code":302,"response-code-details":"via_upstream","time":"2023-02-07T16:07:40Z","message":"http-request"}
pomerium-pomerium-1  | {"level":"info","service":"envoy","upstream-cluster":"pomerium-control-plane-http","method":"GET","authority":"DOMAIN","path":"/.pomerium/","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36","referer":"","forwarded-for":"IP_ADDRESS","request-id":"e5729d20-0d34-4a42-a65e-d95c77751982","duration":91.223814,"size":420,"response-code":302,"response-code-details":"via_upstream","time":"2023-02-07T16:07:40Z","message":"http-request"}
7

What is your authenticate_service_url set to? I wonder if its a separate sub-domain such as authenticate.DOMAIN.

Your authentication domain cannot be reused for any of the routes.

Also, having full set of logs is important to understand what is going on. i.e. the below is annotated log of a first request:


*** USER NAVIGATES TO ROUTE ***

{"level":"info","ip":"127.0.0.1","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36","request-id":"b77577db-4e03-491a-8f8a-903a1b038d24","error":"Bad Request: internal/sessions: session is not found","idp_id":"","time":"2023-02-07T13:02:30-05:00","message":"authenticate: session load error"}

*** NO SESSION, AUTHORIZE WOULD REDIRECT TO SIGN-IN ***

{"level":"info","service":"envoy","upstream-cluster":"pomerium-control-plane-http","method":"GET","authority":"authenticate.localhost.pomerium.io","path":"/.pomerium/sign_in","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36","referer":"","forwarded-for":"192.168.2.75","request-id":"b77577db-4e03-491a-8f8a-903a1b038d24","duration":9.977875,"size":871,"response-code":302,"response-code-details":"via_upstream","time":"2023-02-07T13:02:30-05:00","message":"http-request"}

*** IDP LOGIN SCREEN, FOLLOWED BY REDIRECT TO AUTHENTICATE CALLBACK *** 

{"level":"info","service":"envoy","upstream-cluster":"pomerium-control-plane-http","method":"GET","authority":"authenticate.localhost.pomerium.io","path":"/oauth2/callback","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36","referer":"https://accounts.google.com/","forwarded-for":"192.168.2.75","request-id":"4c2246dd-3d7f-42c6-b58b-3255152aff46","duration":364.053916,"size":394,"response-code":302,"response-code-details":"via_upstream","time":"2023-02-07T13:02:38-05:00","message":"http-request"}

*** CREATING SESSION *** 

{"level":"info","type":"type.googleapis.com/session.Session","id":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0","time":"2023-02-07T13:02:38-05:00","message":"get"}
{"level":"info","type":"type.googleapis.com/user.User","id":"106998556907638105504","time":"2023-02-07T13:02:38-05:00","message":"get"}
{"level":"info","record-count":2,"record-type":"type.googleapis.com/user.User","time":"2023-02-07T13:02:38-05:00","message":"put"}
{"level":"info","type":"type.googleapis.com/session.Session","query":"","offset":0,"limit":1,"filter":{"$or":[{"id":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0"},{"$index":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0"}]},"time":"2023-02-07T13:02:38-05:00","message":"query"}
{"level":"info","type":"type.googleapis.com/session.Session","query":"","offset":0,"limit":1,"filter":{"$or":[{"id":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0"},{"$index":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0"}]},"time":"2023-02-07T13:02:38-05:00","message":"query"}
{"level":"info","type":"type.googleapis.com/user.User","query":"","offset":0,"limit":1,"filter":{"$or":[{"id":"106998556907638105504"},{"$index":"106998556907638105504"}]},"time":"2023-02-07T13:02:38-05:00","message":"query"}
{"level":"info","type":"type.googleapis.com/user.ServiceAccount","query":"","offset":0,"limit":1,"filter":{"$or":[{"id":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0"},{"$index":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0"}]},"time":"2023-02-07T13:02:38-05:00","message":"query"}
{"level":"info","type":"pomerium.io/DirectoryUser","query":"","offset":0,"limit":1,"filter":{"$or":[{"id":"106998556907638105504"},{"$index":"106998556907638105504"}]},"time":"2023-02-07T13:02:38-05:00","message":"query"}

*** CHECK REQUEST TO ROUTE, OK NOW ***

{"level":"info","service":"authorize","request-id":"9c67f0ae-423f-4aa2-b766-780aee0eeca7","check-request-id":"9c67f0ae-423f-4aa2-b766-780aee0eeca7","method":"GET","path":"/","host":"httpbin1.localhost.pomerium.io","query":"","ip":"127.0.0.1","session-id":"debb7c57-9c3c-4a9a-b074-6a9403ec9ce0","allow":true,"allow-why-true":["user-ok"],"deny":false,"deny-why-false":["valid-client-certificate-or-none-required"],"user":"106998556907638105504","email":"XXXXX@gmail.com","time":"2023-02-07T13:02:38-05:00","message":"authorize check"}
8

It is set to the same Domain https://subdomain.domain.toplevel the same as in the from-route. I am trying to get through the documentation but it does not really help me a lot. I mean everything works except for this forward https://subdomain.domain.toplevel → https://subdomain.domain.toplevel/.pomerium/ that I don’t know where it comes from. I will try to capture the logs better …

9

authenticate service URL must be a distinct domain name and cannot be reused for routes.

i.e.

authenticate_service_url: https://authenticate.domain.toplevel

routes: 
  - from: https://www.domain.toplevel
     to: http://localhost:3000
     allow_any_authenticated_user: true
10

Alright. Many thanks - that worked. I am sorry but it was not clear to me that route and service url need to be distinct but I makes sense.