View in #general on Slack
@Edgard_Gomez: @Alex hey there just joined from your suggestion on the support board
@Alex: Hi @Edgard_Gomez - so yea, I’d love to know more about the tech structure in your org and what you’re trying to solve for, so I can maybe offer a suggestion for a simpler deployment. If you don’t mind, I’ll set up a group chat with you, me, and an SME who’s out for the holidays, but can read the scrollback when they get back.
@Edgard_Gomez: For sure.
So I was recently hired on to a small startup. They use google as their idp. They have this auth system called auth that they use with nginx and leverage our google groups to grant / deny access on internal resources.
The auth system is flaky at best. They been trying to switch to pomerium but the guy who I replaced couldn’t get it going so it’s been a task assigned for me to solve
I have a basic proof of concept up and running, but have some issues with routes and granting / denying access as well as routes sending me to the wrong spot like internal.domain.com sends me to diborane.domain.com
@Alex: Sounds like the classic lead-in to Pomerium. What I wanna talk more about is what services you’re securing behind this setup, so I can built a model for what kind of infrastructure you need.
@Edgard_Gomez: They are all custom made apps. So nothing off the shelf. We have an internal site (internal.domain.com) that’s just an index page that loads a markdown file that has hyper links to internal resources like plc’s
@Alex: i.e. “I’m securing web services A and B along with database C, which runs in docker containers on in-house bare-metal”, or… reading above… “we’re running an internal index site on nginx (I assume) that links to internal resources on an internal VM”
So your internal index site, I assume you want anyone in your company to be able to access it, but only specified users or groups to access the internal resources it links to?
@Edgard_Gomez: Yeah we’re fully dockerized here. So our bare metal server runs Ubuntu, which loads a docker-swarm and all our containers talk to the pics
Exactly. The internal anyone with a @domain.com word mail should have access. And then based on google group membership they are either granted or denied access to the internal resources
@Alex: OK - and right now that main index points to PLCs at their internal domain… which I’m guessing is going to be a source of the issues.
@Edgard_Gomez: kinda.
http://internal.domain.com is just a static page that loads a markdown file. and we have static urls for each plc, each plc has a hardcoded IP that loads up a webpage for that PLC.
internal.domain.com —> index.html–>>172.1.18.5 (ip for docker container for plc)
@Alex: OK. Do I assume correctly that you don’t want everyone who has access to the static page and knows the IP of a PLC to be able to access it directly by IP, right?
@Edgard_Gomez: i figured out how to add in conf files for nginx to make sure each plc is protected by pomerium, but at the moment, i have 3 routes in my config.yaml verify, diborane, internal.
verify works like a champ, no issue whatsoever. but the other 2 routes diborane and internal both send me to diborane.domain.com
@Alex: So this comes back to the cruxt of the original question: Why are you using an Nginx server for this?
@Edgard_Gomez: the plcs are logically seperated on our network with a vlan, so users cant access them directly
we use nginx because people are working both from home and from the office, so we have to be able to let people who are not physically in the building reach those resources and this org is very much against using VPN’s
@Alex: So Pomerium solves for that without the need for Nginx
I’ve been diagramming this as we talk about it, to try to better understand the model.
So as I see it, if the links in the index were all pointed to http://plc1.domain.com|plc1.domain.com, http://plc2.domain.com|plc2.domain.com, etc, then:
• everyone could access the index,
• when a user clicked a link on the index they would open a new connection through Pomerium
• Pomerium would grant or deny access to that service based on that service’s route.
So Pomerium listens at the edge of your network, and is routed directly to each PLC
(aside: I put Pomerium outside the swarm to make the diagram easier, but there’s no reason it couldn’t be in the swarm as a container or set of containers)
@Edgard_Gomez: So your saying that on my external dns. I can set my domain to resolve to my external ip. Then forward those request directly to pomerium. And pomerium will do all the necessary routing ?
And I don’t need to add Cnames to my external dns to get them to resolve?
@Alex: That’s what I’m saying. Pomerium can be run as an all-in-one binary which can hide this fact, but it’s actually several sub-services that make up the product. One of them is the Proxy service, which handles all the routing from the rest of the internet to your internal service, restricted by the Authorize service of course. This is discussed in detail here.
Regarding DNS, What I typically do is route one subdomain to the IP Pomerium will use, and then CNAME all sub-domains of that namespace to it. For example:
http://employees.domain.com|employees.domain.com -> A -> $IP
*.http://employees.domain.com|employees.domain.com -> CNAME -> http://employees.domain.com|employees.domain.com
Since you’re already running a swarm and have a large user base, you’d probably want each service in its own container anyway for better scalability.
But back to the DNS question; there’s no reason why you can’t instead do that at the main domain level. If, say, you had a public website for your customers that is accessed at http://domain.com|domain.com, you would just create a route for that too and give it public access.
@Edgard_Gomez: Huh, that’s really freaking interesting. So I can just stand up pomerium without nginx. Add in a wildcard dns record and pomerium takes care of the rest? Just add in routes to my endpoints?
You really weren’t lying when you said I made it more complicated than it needed to be lol
@Alex: “and pomerium takes care of the rest”
yes*
*with one small caveat, and it’s a big one.
@Edgard_Gomez: Lay it on me Alex
@Alex: So one thing we haven’t yet talked about is certificates. Since Pomerium requires TLS for all connections between end users and the proxies, you need a certificate for each domain or subdomain that gets routed.
In my personal setup, I oscillate between solutions. The easiest one is autocert - With the autocert setting enabled in the Pomerium config, Pomerium handles talks to Let’s Encrypt on your behalf and gets a certificate for each new route when it’s brought online.
If LE certs provisioned automatically for you is a viable solution, then we need go no further. If not, tell me your certificate situation and I’ll suggest a config solution.
@Edgard_Gomez: Yeah we use letsencrypt with certbot at the moment to renew our cera
We have a very with the domain and wildcard for the domain
@Alex: “We have a very with the…” <— wat?
If you’re saying that you already have a wildcard certificate for the domain you want to use, and you want to continue to manage it outside of Pomerium, that’s also an easy config: You just turn off autocert in Pomerium and feed it the wildcard cert and key, and it will apply it to all the routes.
ipso facto columbo oreo
@Edgard_Gomez: Yeah sorry. Typing on my phone. We use let’s encrypt with cert or to renew our cert which is essentially just a wildcard domain. But if pomerium can handle all that for me, I’m all for it.
Cert bot*
@Alex: Bam - auto cert uses certbot under the hood, but it will also handle your renewals and whatnot.
@Edgard_Gomez: That’s amazing
@Alex: If you’re not using the existing wildcard cert for anything that Pomerium won’t manage, then it sounds perfect. I suggest you set up Pomerium next to your current solution with autocert in staging mode, set up all your routes, and then when they’re working you can switch over the routing of your main domain (or for those ports to the same IP) to Pomerium and turn off staging mode.
@Edgard_Gomez: Gonna give this configuration a whirl and see how it goes. Really appreciate you Alex
Pomerium will handle everything so this seems like a much cleaner solution overall
@Alex: Happy to help! Since this convo didn’t end up getting into anything secret to your company, and I’m pretty sure there are a lof of people who thought as you did about what Pomerium can and can’t do, I’m gonna copy this whole thread back into your Discuss post for others to benefit from. Cool?
@Edgard_Gomez: For sure
