# Groups changes in 0.20.0 **URL:** \ **Category:** Support\ **Created:** [December 12, 2022, 8:51pm UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192 "2022-12-12T20:51:55Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![alexrudd](https://avatars.discourse-cdn.com/v4/letter/a/bbce88/32.png) [@alexrudd](https://discuss.pomerium.com/u/alexrudd)\ **Post date:** [December 12, 2022, 8:51pm UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/1 "2022-12-12T20:51:55Z")
I noticed `0.20.0` is [released](https://github.com/pomerium/pomerium/releases/tag/v0.20.0) and am considering deploying (we’re currently on `git-3b2cc672`) soon. However, it’s unclear from the release notes what the group changes are: Listed as **new** (i.e. not Breaking) > - move directory providers [#3633](https://github.com/pomerium/pomerium/pull/3633) This PR links to an internal issue which isn’t available to the public and has the following description. > Move support for groups from all identity providers. [GitHub - pomerium/datasource: external data source contrib](https://github.com/pomerium/datasource) > > ## User Explanation > > Group claims are no longer supported. This isn’t particularly helpful IMO. - should it be “Remove” support for groups instead of “move”? - What the heck is `pomerium/datasource`? The docs only have dead links ([example](https://www.pomerium.com/docs/enterprise/external-data/ip-ranges#install)) - **What, exactly, is no longer supported?** I do access control like this in `config.yaml`. Will it break? ```yaml jwt_claims_headers: - name - email - picture - groups - given_name - family_name routes: - from: https://verify.pomerium.com to: http://pomerium_verify pass_identity_headers: true allowed_groups: - user@pomerium.com ``` ---
**Author:** ![bobby](https://yyz1.discourse-cdn.com/flex031/user_avatar/discuss.pomerium.com/bobby/32/1_2.png) [@bobby](https://discuss.pomerium.com/u/bobby)\ **Post date:** [December 13, 2022, 9:18pm UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/2 "2022-12-13T21:18:04Z")
Hi @alexrudd , Apologies, this isn’t the upgrade path we were shooting for with v0.20. We’ve held off on the `v0.20.0` announcement in part because our documentation site is currently _not_ the state we’d like it to be after a major re-org and switching hosting. Unfortunately, that has meant broken links and a staggered release. - The upgrading doc covers what you are looking for. [Upgrading | Pomerium](https://www.pomerium.com/docs/releases/upgrading#idp-groups-policy) - You are correct, that should have been marked as a breaking change. I’ve updated the github changelog and crosslink-ed the upgrade guide to make it more obvious. - Assuming you are getting groups from jwt claims (as your config suggests) this will not break for you. A longer explanation on _why_ these changes were required will come with the announcement. Hope that helps! ---
**Author:** ![alexrudd](https://avatars.discourse-cdn.com/v4/letter/a/bbce88/32.png) [@alexrudd](https://discuss.pomerium.com/u/alexrudd)\ **Post date:** [December 13, 2022, 10:46pm UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/3 "2022-12-13T22:46:12Z")
Hi @bobby, thanks for the prompt reply. I hope your hosting switchover wasn’t too painful. > Assuming you are getting groups from jwt claims (as your config suggests) this will not break for you. Hmmm, I’m nearly certain our deploy will break. The Upgrade Guide answers a fair number of questions, but not all. (1) The JWT claims are added by pomerium for upstream applications, but we had been using `allowed_groups` in the `routes` definition. This has been deprecated. (Mild bummer, it was useful shorthand for simple cases.) The upgrade guide appears at first glance to have the necessary changes, not bad overall… (2) But wait! The upgrade guide says to use `claims: group/xxxx` which won’t exist for Google, right? ```auto routes: - from: https://verify.pomerium.com to: http://pomerium_verify pass_identity_headers: true -- allowed_groups: -- - users@pomerium.com ++ policy: ++ allow: ++ and: ++ - groups: users // Google ++ - claims: group/users // non-Google ``` (3) `idp_service_account`, which we were also using per instructions in prior versions, will totally break for [Google](https://www.pomerium.com/docs/identity-providers/google). The [docs](https://www.pomerium.com/docs/identity-providers/google#setting-up-directory-sync) appear to have been updated helpfully, here. But they remain unclear as to what the “plugin” actually is, so it looks like groups on Google has been moved to Enterprise only.\*\* > Unforunately, Google does not yet support getting groups data using a custom claim. Groups must be loaded by using a plugin to fetch directory information (see Enterprise’s Directory Sync). (4) It’s unclear to me whether the move to `datasource` breaks anything. I’ll wait until the docs settle down, as the [webpage](https://www.pomerium.com/docs/integrations) is just a marketing blurb and [`README.md`](https://github.com/pomerium/datasource/blob/main/README.md) is empty. \*\* Note: we have an Enterprise license, but have been using the open-source Core. It seems now a switch is required. @XxEnigmaticxX ---
**Author:** ![alexrudd](https://avatars.discourse-cdn.com/v4/letter/a/bbce88/32.png) [@alexrudd](https://discuss.pomerium.com/u/alexrudd)\ **Post date:** [January 6, 2023, 7:53pm UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/4 "2023-01-06T19:53:45Z")
Thanks for updating the changelog with breaking changes. Can you clarify the upgrade path for `allowed_groups`? As I’ve discussed above, I’m nearly certain an upgrade would break are deploy but it’s unclear how to migrate. ---
**Author:** ![bobby](https://yyz1.discourse-cdn.com/flex031/user_avatar/discuss.pomerium.com/bobby/32/1_2.png) [@bobby](https://discuss.pomerium.com/u/bobby)\ **Post date:** [January 6, 2023, 11:01pm UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/5 "2023-01-06T23:01:46Z")
Yes. Will respond back shortly here in a few hours. Thanks for the poke @alexrudd ! ---
**Author:** ![bobby](https://yyz1.discourse-cdn.com/flex031/user_avatar/discuss.pomerium.com/bobby/32/1_2.png) [@bobby](https://discuss.pomerium.com/u/bobby)\ **Post date:** [January 7, 2023, 2:09am UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/6 "2023-01-07T02:09:12Z")
Upgrade path is to use ppl syntax for groups, and to update directory sync to use external datasources. ### Yaml version ```yaml allow: or: - email: is: bdd@pomerium.com - groups: has: 00tyjcwt2dgurje ``` ### PPL Builder version ![Screenshot 2023-01-06 at 6.07.52 PM](https://canada1.discourse-cdn.com/flex031/uploads/pomerium/original/1X/e58ad2f912a26f3770c80cb845d96cd6f91fd749.png) - [https://www.pomerium.com/docs/releases/enterprise/upgrading#0200](https://www.pomerium.com/docs/releases/enterprise/upgrading#0200) - For Google Specifically: [Google Workspace (formerly known as G Suite) | Pomerium](https://www.pomerium.com/docs/identity-providers/google#setting-up-directory-sync) Hope that helps; if you hit any snags or it could be made clearer please let us know. ---
**Author:** ![alexrudd](https://avatars.discourse-cdn.com/v4/letter/a/bbce88/32.png) [@alexrudd](https://discuss.pomerium.com/u/alexrudd)\ **Post date:** [January 14, 2023, 12:03am UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/7 "2023-01-14T00:03:09Z")
Well, I got reasonably far. I managed to attach an Enterprise console (0.20.0) to my existing Core (`v0.19.0-28-g3b2cc672 + 3b2cc672`). I imported the existing service account, which appears to have worked since it’s adding entries in the databroker PSQL table `records` ![Screenshot 2023-01-13 at 5.36.54 PM](https://canada1.discourse-cdn.com/flex031/uploads/pomerium/original/1X/a689aa65db72c04a09c773a6574aaa4d31b89bae.png). (I think these are new since they are namespaced as `pomerium.io` which I haven’t seen before. There are also old(?) records of type: `type.googleapis.com/user.User` and `type.googleapis.com/directory.Group` but I think (?) those are from the 0.19 databroker) I tried using PPL: ```auto - from: https://blah to: blah allow: or: groups: has: 02pta16n1mjcsij # engineering@domain.com ``` I am a member of this group ID (yuck, can I use names?), and it matches the PSQL screenshot. I can also see it on the Groups Info tab at `./pomerium`. I can also see it via Verify. EDIT: Below problems were caused by my `config.yaml` missing a `policy:` block above `allow:` I got it working!! although I’ll need to disable the old service account config to be sure it’s actually pulling from the External Data Store. === No dice, sadly. ![Screenshot 2023-01-13 at 5.50.20 PM](https://canada1.discourse-cdn.com/flex031/uploads/pomerium/original/1X/3d6372998986ca9626f5543b8ed41c635492a410.png) ```auto "level": "info", "service": "authorize", "allow": false, "allow-why-false": [ "non-pomerium-route" ], "deny": false, "deny-why-false": [ "valid-client-certificate-or-none-required" ], ``` I wondered if I should use `record` instead of `group` somehow. I tried ```auto allow: or: record: has: 02pta16n1mjcsij # engineering@domain.com ``` and ```auto allow: or: record: 02pta16n1mjcsij # engineering@domain.com ``` and ```auto allow: or: record: group_ids: 02pta16n1mjcsij # engineering@domain.com ``` before giving up. Sadly, the [documentation](https://www.pomerium.com/docs/integrations) on External Data Sources remains useless so I’m out of ideas. ---
**Author:** ![alexrudd](https://avatars.discourse-cdn.com/v4/letter/a/bbce88/32.png) [@alexrudd](https://discuss.pomerium.com/u/alexrudd)\ **Post date:** [January 14, 2023, 12:43am UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/8 "2023-01-14T00:43:48Z")
I’m now pretty confident I got it working using a service account as an external data source. In fact it does work with names and not ids (thank goodness). ```auto - from: https://blah to: blah policy: allow: or: groups: has: engineering@domain.com ``` To confirm I (1) Commented out the existing `idp_service_account` in `config.yaml` (2) Deleted the record `type.googleapis.com/directory.Group` corresponding to `engineering@domain.com` (3) Attempted to access `https://blah` and was allowed. (via the new method) (4) Reverted to using `allowed_groups` with the `idp_service_account` disabled. (5) Attempted to access `https://blah` and was denied (the old method) (6) Re-enabled `idp_service_account` and waited several mins for a sync (7) Attempted to access `https://blah` and was allowed (old method) ---
**Author:** ![alexrudd](https://avatars.discourse-cdn.com/v4/letter/a/bbce88/32.png) [@alexrudd](https://discuss.pomerium.com/u/alexrudd)\ **Post date:** [January 19, 2023, 1:51am UTC](https://discuss.pomerium.com/t/groups-changes-in-0-20-0/192/9 "2023-01-19T01:51:13Z")
A refinement of the above… `groups` should be an array item. Otherwise, if you try to do multiple ones the others will get dropped. ```auto - from: https://blah to: blah policy: deny: or: groups: #FAIL has: engineering@domain.com groups: has: it@domain.com allow: or: - groups: #OK has: engineering@domain.com - groups: has: it@domain.com ```