# Example Docker Compose Fails to Start Any Containers **URL:** https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162 **Category:** Support **Tags:** docker **Created:** [October 3, 2022, 7:03am UTC](https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162 "2022-10-03T07:03:42Z") **Posts on this page:** 6 **Page:** 1
### Author: ![aadhi0319](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@aadhi0319](https://discuss.pomerium.com/u/aadhi0319) #### Post date: [October 3, 2022, 7:03am UTC](https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162/1 "2022-10-03T07:03:42Z")
## What happened? I am new to Pomerium and I was following the docker [startup guide](https://www.pomerium.com/docs/install/quickstart). Both containers failed to start. There seems to be an issue with allocating memory for at least one of the two containers. ## What did you expect to happen? I expected both containers to execute properly so that I could continue to experiment with Pomerium. ## How’d it happen? `sudo docker compose up` docker-compose.yml: ```auto version: "3" services: pomerium: image: pomerium/pomerium:latest volumes: ## Mount your domain's certificates : https://www.pomerium.com/docs/reference/certificates - ../certs/_wildcard.localhost.pomerium.io.pem:/pomerium/cert.pem:ro - ../certs/_wildcard.localhost.pomerium.io-key.pem:/pomerium/privkey.pem:ro ## Mount your config file : https://www.pomerium.com/docs/reference/ - ../pomerium/config.yaml:/pomerium/config.yaml:ro ports: - 443:443 ## https://verify.localhost.pomerium.io --> Pomerium --> http://verify verify: image: pomerium/verify:latest expose: - 8000 ``` ## What’s your environment like? `Linux raspberrypi 5.15.61-v8+ #1579 SMP PREEMPT Fri Aug 26 11:16:44 BST 2022 aarch64 GNU/Linux` - Raspberry Pi 3 running Raspbian Lite OS. 64 bit arm. - TLS certificates self signed using the method depicted in the tutorial. - Pomerium docker container with “latest” tag. ## What’s your config.yaml? ```auto # See detailed configuration settings : https://www.pomerium.com/docs/reference/ # this is the domain the identity provider will callback after a user authenticates authenticate_service_url: https://authenticate.localhost.pomerium.io #################################################################################### # Certificate settings: https://www.pomerium.com/docs/reference/certificates.html # # The example below assumes a certificate and key file will be mounted to a volume # # available to the Docker image. # #################################################################################### certificate_file: /pomerium/cert.pem certificate_key_file: /pomerium/privkey.pem ################################################################################## # Identity provider settings : https://www.pomerium.com/docs/identity-providers/ # # The keys required in this section vary depending on your IdP. See the # # appropriate docs for your IdP to configure Pomerium accordingly. # ################################################################################## idp_provider: "auth0" idp_provider_url: "https://.eu.auth0.com" idp_client_id: "" idp_client_secret: "" idp_service_account: "" # Generate 256 bit random keys e.g. `head -c32 /dev/urandom | base64` cookie_secret: # https://pomerium.com/reference/#routes routes: - from: https://verify.localhost.pomerium.io to: http://verify:8000 policy: - allow: or: - email: is: @gmail.com pass_identity_headers: true ``` ## What did you see in the logs? ```auto [+] Running 2/0 ⠿ Container docker-verify-1 Created 0.0s ⠿ Container docker-pomerium-1 Created 0.0s Attaching to docker-pomerium-1, docker-verify-1 docker-verify-1 | exec /bin/verify: exec format error docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","service":"all","config":"local","checksum":"247981595a7e0bd9","time":"2022-10-03T06:37:30Z","message":"config: updated config"} docker-pomerium-1 | {"level":"debug","watch_file":"/pomerium/config.yaml","time":"2022-10-03T06:37:30Z","message":"filemgr: watching file for changes"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"envoy_version":"1.23.1+10f39007f8ed9d9db6ab93aed67de64f79ac9b40e5e0392054ef86114887b3fe","version":"0.19.1-1662666141+c0a88707","time":"2022-10-03T06:37:30Z","message":"cmd/pomerium"} docker-pomerium-1 | {"level":"info","address":"127.0.0.1:38325","time":"2022-10-03T06:37:30Z","message":"grpc: dialing"} docker-pomerium-1 | {"level":"info","outbound_port":"38325","databroker_urls":["http://127.0.0.1:5443"],"time":"2022-10-03T06:37:30Z","message":"config: starting databroker config source syncer"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"service":"all","config":"databroker","checksum":"247981595a7e0bd9","time":"2022-10-03T06:37:30Z","message":"config: updated config"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"service":"metrics_manager","time":"2022-10-03T06:37:30Z","message":"metrics: http server disabled"} docker-pomerium-1 | {"level":"error","domain":"*","time":"2022-10-03T06:37:30Z","message":"cryptutil: no TLS certificate found for domain, using self-signed certificate"} docker-pomerium-1 | {"level":"error","domain":"*","time":"2022-10-03T06:37:32Z","message":"cryptutil: no TLS certificate found for domain, using self-signed certificate"} docker-verify-1 exited with code 1 docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"grpc-port":"35993","http-port":"38299","outbound-port":"38325","metrics-port":"36119","debug-port":"43675","acme-tls-alpn-port":"41119","time":"2022-10-03T06:37:34Z","message":"server started"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"time":"2022-10-03T06:37:37Z","message":"envoy: starting envoy process"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"path":"/tmp/pomerium-envoy3253722158/envoy","checksum":"10f39007f8ed9d9db6ab93aed67de64f79ac9b40e5e0392054ef86114887b3fe","time":"2022-10-03T06:37:37Z","message":"running envoy"} docker-pomerium-1 | {"level":"info","pid":13,"time":"2022-10-03T06:37:37Z","message":"envoy: start monitoring subprocess"} docker-pomerium-1 | {"level":"info","address":"127.0.0.1:38325","time":"2022-10-03T06:37:37Z","message":"grpc: dialing"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"time":"2022-10-03T06:37:37Z","message":"enabled authenticate service"} docker-pomerium-1 | {"level":"info","Algorithm":"ES256","KeyID":"935ee5538ec72615c44e36643d857e7260046a71f6d5217183c8fd2adb3433eb","Public Key":{"use":"sig","kty":"EC","kid":"935ee5538ec72615c44e36643d857e7260046a71f6d5217183c8fd2adb3433eb","crv":"P-256","alg":"ES256","x":"XeXKVXlXGvgGjA_-gwt3ZhykAJMnkYGTf09yZSnynok","y":"PKRMJGPgaMGrDqy97Q3V-CfdfMOWcrDWC--LtL-QWbI"},"time":"2022-10-03T06:37:37Z","message":"authorize: signing key"} docker-pomerium-1 | {"service":"envoy","name":"envoy","time":"2022-10-03T06:37:37Z","message":"external/com_github_google_tcmalloc/tcmalloc/system-alloc.cc:631] MmapAligned() failed - unable to allocate with tag (hint, size, alignment) - is something limiting address placement? 0x563840000000 1073741824 1073741824 @ 0x556801fa88 0x556801b6a0 0x556801af48 0x5568002f4c 0x5568018010 0x5568017e2c 0x5567ff78a8 0x5567f36c00 0x5567f32298 0x5567ebf484 0x5567fedbd0 0x7fbeae41c0"} docker-pomerium-1 | {"service":"envoy","name":"envoy","time":"2022-10-03T06:37:37Z","message":"external/com_github_google_tcmalloc/tcmalloc/arena.cc:58] FATAL ERROR: Out of memory trying to allocate internal tcmalloc data (bytes, object-size); is something preventing mmap from succeeding (sandbox, VSS limitations)? 131072 600 @ 0x556801fde4 0x5568002fdc 0x5568018010 0x5568017e2c 0x5567ff78a8 0x5567f36c00 0x5567f32298 0x5567ebf484 0x5567fedbd0 0x7fbeae41c0"} docker-pomerium-1 | {"level":"info","address":"127.0.0.1:38325","time":"2022-10-03T06:37:38Z","message":"grpc: dialing"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"time":"2022-10-03T06:37:38Z","message":"enabled authorize service"} docker-pomerium-1 | {"level":"info","Algorithm":"ES256","KeyID":"6b88a3adf9fd432aa5941d5b3e9b0fa6f11e00ba4441ca2a33415d6e032efb62","Public Key":{"use":"sig","kty":"EC","kid":"6b88a3adf9fd432aa5941d5b3e9b0fa6f11e00ba4441ca2a33415d6e032efb62","crv":"P-256","alg":"ES256","x":"kBKOD3Z52mvu4uk_oRCW2tjbCYPT3b6FQjuyT0TwiXk","y":"cDz5P8DtbC5FCufpHIckBXfDL2uGbzNjjfn47o8C6Hk"},"time":"2022-10-03T06:37:38Z","message":"authorize: signing key"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"time":"2022-10-03T06:37:38Z","message":"enabled databroker service"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"address":"127.0.0.1:38325","time":"2022-10-03T06:37:38Z","message":"grpc: dialing"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"time":"2022-10-03T06:37:38Z","message":"enabled proxy service"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"addr":"127.0.0.1:35993","time":"2022-10-03T06:37:38Z","message":"starting control-plane gRPC server"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"addr":"127.0.0.1:38299","time":"2022-10-03T06:37:38Z","message":"starting control-plane http server"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"addr":"127.0.0.1:43675","time":"2022-10-03T06:37:38Z","message":"starting control-plane debug server"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"addr":"127.0.0.1:36119","time":"2022-10-03T06:37:38Z","message":"starting control-plane metrics server"} docker-pomerium-1 | {"level":"info","name":"identity_manager","duration":30000,"time":"2022-10-03T06:37:38Z","message":"acquire lease"} docker-pomerium-1 | {"level":"info","time":"2022-10-03T06:37:38Z","message":"using in-memory store"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"service":"identity_manager","syncer_id":"identity_manager","syncer_type":"","time":"2022-10-03T06:37:38Z","message":"initial sync"} docker-pomerium-1 | {"level":"info","type":"","time":"2022-10-03T06:37:38Z","message":"sync latest"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"service":"identity_manager","directory_groups":0,"directory_users":0,"sessions":0,"users":0,"time":"2022-10-03T06:37:38Z","message":"initial sync complete"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"service":"identity_manager","time":"2022-10-03T06:37:38Z","message":"refreshing directory users"} docker-pomerium-1 | {"level":"info","config_file_source":"/pomerium/config.yaml","bootstrap":true,"service":"identity_manager","syncer_id":"identity_manager","syncer_type":"","time":"2022-10-03T06:37:38Z","message":"listening for updates"} docker-pomerium-1 | {"level":"info","server_version":355086907169193799,"record_version":0,"time":"2022-10-03T06:37:38Z","message":"sync"} docker-pomerium-1 | {"level":"fatal","pid":13,"time":"2022-10-03T06:37:38Z","message":"envoy: subprocess exited"} docker-pomerium-1 exited with code 1 ``` ## Additional context > > > \*\*Is your feature request related to a problem? Please describe.\*\* > > Due to the… way envoy is distributed, we will be losing ARM support on the new release. This issue is to track re-adding it. > > \*\*Describe the solution you'd like\*\* > > ARM64 at least. ARM32 as well, ideally. This issue ndicates that the container should work fine on Arm as long as the architecture is 64 bit. Also, note that both the verify and the pomerium containers fail for different reasons. Thanks in advance for your help! ---
### Author: ![denis](https://avatars.discourse-cdn.com/v4/letter/d/5e9695/32.png) [@denis](https://discuss.pomerium.com/u/denis) #### Post date: [October 3, 2022, 2:53pm UTC](https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162/2 "2022-10-03T14:53:31Z")
`pomerium/verify` fails as docker images are only supplied for amd64 architecture. `pomerium` bundles Envoy that cannot start indeed due to `mmap` failure. That sounds like a restriction on your system. Please try to run a binary release file without docker. ---
### Author: ![aadhi0319](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@aadhi0319](https://discuss.pomerium.com/u/aadhi0319) #### Post date: [October 3, 2022, 6:59pm UTC](https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162/3 "2022-10-03T18:59:58Z")
Thank you for the speedy response. You are right, mmap is failing for some reason. From some google searching, I found that Pomerium or a Pomerium dependency (likely Envoy) is calling Google’s TCMalloc. There are known [issues](https://github.com/google/tcmalloc/issues/33) with TCMalloc on arm64. At this point do you think it would be best to just move to an x86 system? Using a Raspberry Pi for this would have been nice, but I’m not sure how long it will take to get this fixed. Also, I’m curious as to how the other people who got Pomerium to work on Raspberry Pis did it. If you did it, please advise! ---
### Author: ![denis](https://avatars.discourse-cdn.com/v4/letter/d/5e9695/32.png) [@denis](https://discuss.pomerium.com/u/denis) #### Post date: [October 4, 2022, 1:06am UTC](https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162/4 "2022-10-04T01:06:05Z")
I tried Pomerium v19.1 with google cloud ARM64 VM and it works. ```auto Linux arm-test 5.18.0-0.deb11.4-cloud-arm64 #1 SMP Debian 5.18.16-1~bpo11+1 (2022-08-12) aarch64 GNU/Linux ``` Could you please try if the below works. if it does not, please open a ticket with Envoy. `docker run envoyproxy/envoy:v1.23.1` ---
### Author: ![aadhi0319](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@aadhi0319](https://discuss.pomerium.com/u/aadhi0319) #### Post date: [October 4, 2022, 1:41am UTC](https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162/5 "2022-10-04T01:41:49Z")
I get the same error when I run that container. I checked the Envoy Proxy Github and it turns out that this is a known issue (as of 3 days ago). Thanks for your help! > > > Envoy crashes on RP 4 due to \`tcmalloc\` allocation error > =====================…====================== > > I am trying to run Envoy on RP4 and 64bit Raspberry Pi OS. > > Envoy \`1.20\` crashes with: > > \`\`\` > external/com\_github\_google\_tcmalloc/tcmalloc/system-alloc.cc:550\] MmapAligned() failed (size, alignment) 1073741824 1073741824 @ 0x559342edc4 0x55934211a4 0x5593420be4 0x559340a4c0 > external/com\_github\_google\_tcmalloc/tcmalloc/arena.cc:34\] FATAL ERROR: Out of memory trying to allocate internal tcmalloc data (bytes, object-size) 131072 48 @ 0x559342f0d4 0x559340a534 > \`\`\` > > Envoy \`1.23\` crashes with: > > \`\`\` > external/com\_github\_google\_tcmalloc/tcmalloc/system-alloc.cc:631\] MmapAligned() failed - unable to allocate with tag (hint, size, alignment) - is something limiting address placement? 0x151880000000 1073741824 1073741824 @ 0x55941cfa88 0x55941cb6a0 0x55941caf48 0x55941b2f4c 0x55941c8010 0x55941c7e2c 0x55941a78a8 0x55940e6c00 0x55940e2298 0x559406f484 0x559419dbd0 0x7f834774ac > external/com\_github\_google\_tcmalloc/tcmalloc/arena.cc:58\] FATAL ERROR: Out of memory trying to allocate internal tcmalloc data (bytes, object-size); is something preventing mmap from succeeding (sandbox, VSS limitations)? 131072 600 @ 0x55941cfde4 0x55941b2fdc 0x55941c8010 0x55941c7e2c 0x55941a78a8 0x55940e6c00 0x55940e2298 0x559406f484 0x559419dbd0 0x7f834774ac > \`\`\` > > Tested also on envoy \`1.17\`, \`1.18\`, \`1.19\`, and \`1.22\` (it seems that \`tcmalloc\` was introduced in \`1.17\`). Same behaviour. > > How to reproduce: > > In Docker: > > \`\`\`shell > docker run --rm envoyproxy/envoy:v1.24-latest > \`\`\` > > On a host machine: > > \`\`\`shell > envoy\_location=$(docker run --rm --entrypoint which docker.io/envoyproxy/envoy:v1.23-latest envoy) > container\_id=$(docker create which envoyproxy/envoy:v1.23-latest) > docker cp "${container\_id}":"${envoy\_location}" ./envoy > docker rm -v "${container\_id}" > ./envoy > \`\`\` > > I am using RP4 8GB. ---
### Author: ![aadhi0319](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@aadhi0319](https://discuss.pomerium.com/u/aadhi0319) #### Post date: [October 16, 2022, 2:06am UTC](https://discuss.pomerium.com/t/example-docker-compose-fails-to-start-any-containers/162/6 "2022-10-16T02:06:10Z")
Using a 64 bit ARM OS that isn’t Raspbian fixes this issue. In my case, I used Ubuntu Server LTS for the Raspberry Pi.