Running Docker Containers as Current Host User

[jtreminio]

https://jtreminio.com/blog/running-docker-containers-as-current-host-user/

[edulan]

Thanks for your detailed explanation @jtreminio, was really helpful 👏 👏

[dcaballero]

Insightful and helpful, however I have a question, we are looking to use something like this approach to dynamically setup user(s) inside the container to match host user ids, you know, to deal with permissions for volumes and so on, however, we want to be able to build 1 image that we can then run in different environments, with some kind of parameters that pass the actual user id from that particular environment, we would like to avoid having to build a different image per environment, any insight into how to resolve current user and pass to compose yml? Preferably without using previous external shell steps, because docker restart policies would not be able to run that in the event of a container restart, or can it?

[lephleg]

Great article! I've always got halfway to it. It's nice to use this as a reference.

[rogerpence]

Juan--What a great a article and clever solution. I struggled with this for months and can't for the life of me understand why your info isn't part of the Docker docs. I'm now running Docker in my container as my local user and it's great (and it worked the first damn time! Nothing ever does that for me!). Next time you make a trip down from Dallas to San Antonio I'll buy lunch! Thank you, again.

[jtreminio]

@dcaballero Unfortunately the ID reassignment needs to happen at the pre-boot stage.

If you simply run a script in a running container that does all the work, it may not have your expected results. Any users currently running a process in the container may experience issues since their user ID would technically no longer exist.

[jtreminio]

@edulan @lephleg @rogerpence Thank you all for the kind words! I am glad you may have found some use for this!

[mangalaman93]

I was wondering if only mapping the user IDs would work. If the current user belongs to root group, things will work out just fine -

• All the host directories can be modified by the current user from the host
• All the container directories can be accessed and modified by the mapped user inside container

Any problems with this solution?

[Xerkus]

I tried all kind of approaches, but when you hop projects you can't really expect to have anything readily available in place. Custom scripting takes time to write and as a contractor you are expected to jump right in from pretty much day one.
So, I ended up using lebokus/bindfs docker plugin to get access to UID/GID remap provided by fuse fs. It proved to be the most flexible approach.
sudo docker plugin install lebokus/bindfs

Here is an excerpt from one of the docker-compose.yml I used for local microservices development environment where some microservices were bound to local development forks:
localdev/docker-compose.yml

version: "3.4"

services:
  clients-service:
    build: ../clients-service
    environment:
      SERVICE_DATABASE_HOST: clients-db
      SERVICE_DATABASE_USER: clients
      SERVICE_DATABASE_PASSWORD:
    ports:
      - "8085:80"
    volumes:
      - clients-service:/app
volumes:
  clients-service:
    driver: lebokus/bindfs:latest
    driver_opts:
      sourcePath: "${PWD}/../clients-service"
      map: "${UID:-1000}/33:@${UID:-1000}/@33"  

Best part of this approach is that I do not need to modify images or containers in any way, I can spin up already built image and bind to local files. I can bind any local user to any containerized user id, all at the same time.
As everything else with docker, there are downsides. This plugin uses sshfs, which adds some overhead and might not work for windows or mac. This setup is pretty static as it depends on volume provider. Named volume with mapping options must be created first, before it can be used with docker run -v. However, there is new --mount syntax that should make usage more convenient, since it will allow to create temporary volumes with docker run --mount directly.

[akorz]

Hello,

Thank you for great article. I try to modify redis image

FROM redis:5.0.1

ARG USER_ID
ARG GROUP_ID

RUN if [ ${USER_ID:-0} -ne 0 ] && [ ${GROUP_ID:-0} -ne 0 ]; then \
    userdel -f redis && \
    if getent group redis ; then groupdel redis; fi && \
    groupadd -g ${GROUP_ID} redis && \
    useradd -l -u ${USER_ID} -g redis redis && \
    install -d -m 0755 -o redis -g redis /home/redis && \
    chown -R \
         ${USER_ID}:${GROUP_ID} \
        /home/redis \
        /data \
;fi
        
USER redis

part of my composer file

redis:
    build:
      context: ../redis
      dockerfile: Dockerfile
      args:
        USER_ID: $USER_ID
        GROUP_ID: $GROUP_ID
    working_dir: /app/redis
    container_name: mrktplc_redis
    command: redis-server --appendonly yes
    volumes:
      - ../data/redis:/data

I start it by shell script

export USER_ID=$(id -u)
export GROUP_ID=$(id -g)

docker-compose -p mrktplc -f $DIR/../docker/docker-compose.dev.yml up -d --build --remove-orphans

Then I check /data folder on my host

root@alexei-Z170-HD3:~/PhpstormProjects/web/data# ls -al
итого 20
drwxr-xr-x  5 alexei alexei 4096 июн 18 18:00 .
drwxr-xr-x 11 alexei alexei 4096 июн 18 17:36 ..
drwxr-xr-x  7     27 sudo   4096 июн 18 18:00 mysql
drwxr-xr-x  5 alexei alexei 4096 июн 18 18:00 rabbitmq
drwxr-xr-x  2 root   root   4096 июн 18 18:00 redis

Why do I get redis under root? My current user is 1000:1000

[jtreminio]

@akorz If you share a directory with a container, and the directory did not exist before running the docker command, docker creates the directory as the user running the daemon (usually root).

[Carsak]

@jtreminio
Hi Juan.
I just translated your article to russian with my some improvements. Do you mind?
link to article https://almat.su/docker-zapusk-protsessa-iz-pod-tekushhego-polzovatelya/

[jtreminio]

@jtreminio
Hi Juan.
I just translated your article to russian with my some improvements. Do you mind?
link to article https://almat.su/docker-zapusk-protsessa-iz-pod-tekushhego-polzovatelya/

Awesome!

[musiro77]

Hello thanks a lot
unfortuanatly i still have th eproble after running the compose, the mount volume remains by root 0:0

[musiro77]

Hi, thanks for this apporach and clarification , however I still see a problem if we mount a volume
.
the folder /home/sel/features is always owner by root 0:0 after mounting this volum from host.
see in composer file : ./temp:/home/sel/features
when I remove this line (without mounting), then the folder /home/sel/features is owned by 1000:1000

here my compose file:

tests:
container_name: tests
build:
context: /.
dockerfile: Dockerfile
args:
USER_ID: ${USER_ID:-0}
GROUP_ID: ${GROUP_ID:-0}
volumes:
- ./temp:/home/sel/features

in Dockerfile, I am using your approach
the entry point is just ls -Ian
ENTRYPOINT pwd && ls -lan && ls -lan features

/home/sel
tests | total 183292
tests | drwxr-xr-x 1 1000 1000 4096 Nov 18 09:02 .
tests | drwxr-xr-x 1 0 0 4096 Nov 18 08:53 ..
tests | -rw-rw-rw- 1 1000 1000 1096 Nov 18 09:02 Dockerfile
tests | drwxr-xr-x 2 0 0 4096 Nov 17 13:56 features
tests | drwxrwxrwx 1 1000 1000 4096 Nov 18 07:05 huhu
tests | -rw-rw-rw- 1 1000 1000 8733 Nov 18 07:05 pom.xml

[carlfischerjba]

Thanks for a comprehensive write-up of the issues and solutions.

It seems that for your solution to work the id for the internal user account defined at build time needs to match the id for the host account used at run time. This is fine if you're building and running on the same machine but won't necessarily work if you're using different machines, e.g. building the image on a CI server but running on a developer's workstation, possibly in a different company or with a different Linux distribution. Would you agree that this is a concern or have I missed something?

[sourcecodemage]

I'm going to use this for strategy for a docker container I'm building to run ansible playbooks that have a lot of dependencies. The problems to solved were keeping ssh keys secure and developing the container on Mac but running it on Linux in production. Having Ansible run as the current user and using ssh keys on the host machines saves me having to make a service user and giving it ssh keys and access to our servers.

I'll need to do that later once I move manual processes into pipelines ( like Jenkins ) , but that's a problem to be solved another day :)

[rajasyedtw]

@jtreminio - You saved our day!! let me know if you are visiting Tamil Nadu, India. I'll buy Briyani for you!!! :)

After so many attempts/google searches, your blog gave us proper guidelines on how to solve the issue.

[prismplex]

Hi,
thank you for this great article with a straight solution. Unfortunately I cannot get it to work and I really do not know what the problem could be. It seems to be a permission problem, spinning up the default image(/container) without modifications works.

I'm getting the following error, when I start the container:

Traceback (most recent call last):
  File "/sbin/my_init", line 414, in <module>
    main(args)
  File "/sbin/my_init", line 330, in main
    import_envvars(False, False)
  File "/sbin/my_init", line 90, in import_envvars
    for envfile in listdir("/etc/container_environment"):
  File "/sbin/my_init", line 74, in listdir
    return sorted(os.listdir(path))
PermissionError: [Errno 13] Permission denied: '/etc/container_environment'

My Dockerfile:

FROM jtreminio/php:7.3

ARG USER_ID
ARG GROUP_ID

RUN if [ ${USER_ID:-0} -ne 0 ] && [ ${GROUP_ID:-0} -ne 0 ]; then \
    userdel -f www-data &&\
    if getent group www-data ; then groupdel www-data; fi &&\
    groupadd -g ${GROUP_ID} www-data &&\
    useradd -l -u ${USER_ID} -g www-data www-data &&\
    install -d -m 0755 -o www-data -g www-data /home/www-data &&\
    chown --changes --silent --no-dereference --recursive \
          --from=33:33 ${USER_ID}:${GROUP_ID} \
        /home/www-data \
        /.composer \
        /var/run/php-fpm \
        /var/lib/php/sessions \
;fi
        
USER www-data

My docker-compose.yml:

version: "3.1"
services:

    memcached:
      image: memcached:alpine
      container_name: php-memcached

    redis:
      image: redis:alpine
      container_name: php-redis

    php-fpm:
      build:
        context: .
        dockerfile: Dockerfile
        args:
          USER_ID: 1000
          GROUP_ID: 985
      container_name: php-73
      volumes:
        - ${USERDIR}/nginx:/var/www
        - ${USERDIR}/.composer:/.composer
        - ./php-ini-overrides.ini:/etc/php/php-custom.ini
      environment:
        - PHP_INI_SCAN_DIR=:/p/apcu:/p/geoip:/p/imagick:/p/memcached:/p/redis
      ports:
       - "9003:9000"

${USERDIR} is defined in /etc/environment.

[ljay79]

Works great for me,
just one issue i has with the code snippet.

error: Creating mailbox file: File exists

I was able to solve it by change the userdel command to

userdel -f -r www-data

[zzeroo]

Great article, thank you so much. Very complete and so clear nice work.

[jeff47]

Very helpful! I'm trying to get this to work for Tdarr. For my model, I need to change your example from www-data to Tdarr, but otherwise it should be quite similar.

Dockerfile

FROM haveagitgat/tdarr:latest

ARG PUID
ARG PGID

RUN userdel -f Tdarr && \
  groupdel Tdarr && \
  groupadd -g ${PGID} Tdarr && \
  useradd -l -u ${PUID} -g Tdarr Tdarr && \
  chown -R ${PUID}:${PGID} /home/Tdarr/Tdarr

Build command

> docker image build --build-arg PUID=1005 --build-arg PGID=1006 -t jeffrice/tdarr-plus:1.0 .

However, when I try to build the new image, I get an error when the userdel command runs. It seems like a process is already running under the user Tdarr, but why would that be true during the build process?

userdel: user Tdarr is currently used by process 1
userdel: Permission denied.
userdel: cannot lock /etc/passwd; try again later.

Any tips?

[gpiffaretti]

@carlfischerjba I have that same problem. I can't bind the image to a specific user in build time, I need to distribute an image and make it work in other environments that I don't own. Still doing research, if anybody finds a solution for that please share :)

[zzeroo]

@gpiffaretti Did you notice the article on the very first post of this issue? All basic information are in that excellent article and in some of the commends in this github issue. Try to understand everything and ask specific questions here.

The article was made to solve exactly your problem I think.

[zzeroo]

@jeff47 You are trying to override the user Tdar. The Image haveagitgat/tdarr:latest use this user internally for doing all its stuff. See: https://hub.docker.com/r/haveagitgat/tdarr/dockerfile

If that worked nothing from the image haveagitgat/tdarr:latest would work. Try to look for an other route, this one can't work.

[carlfischerjba]

@gpiffaretti wrote:

I have that same problem. I can't bind the image to a specific user in build time, I need to distribute an image and make it work in other environments that I don't own.

You can specify the user in your docker-compose.yml with a line like this: user: ${DOCKER_UID:-0}:${DOCKER_GID:-0}. You then need to run with DOCKER_UID=$(id -u) DOCKER_GID=$(id -g) docker-compose up. This will ensure that the main process in your container runs with the same uid/gid as the user running the stack on the host.

The advantage is that you're not running as root and that any files written into bind mounts will be owned by you rather than root or some other arbitrary uid that may not match a user account on the host. You also do not need to match the image to the host or agree on a particular uid to be present everywhere the app might be deployed.

You can't enforce running as non-root in the image itself since it's dependent on the right steps being taken at runtime but you could combine it with the suggestions in the article and create another non-root user to use as the default (replace the 0 with 1000 or whatever in the compose file and set USER 1000 in the Dockerfile).

Depending on how your application is designed, there can be some issues around file permissions. You shouldn't be writing anything inside the container as a best practice but if you do there may be issues since the non-root user may not be allowed to write in certain locations. Writing to a volume or bind mount is ok but again it may not be writable by the new user, especially if you don't know what that user is going to be until runtime. In some images, I've had to set very relaxed permissions or certain directories so that when a volume is mounted there it gets permissions that allow any user to write. Not ideal but the best I've come up with so far. The ideal situation is if you never write anything to disk and only interact with services over the network, then permissions are not an issue and you can run your container as any user.

[carlfischerjba]

@jeff47 I guess the base image sets USER tdarr which means that user is being used to run the build itself. Switching user first might work (it should at least allow the build to proceed, I don't know whether it will cause problems at runtime). Try something like this.

FROM haveagitgat/tdarr:latest

ARG PUID
ARG PGID

USER root

RUN userdel -f Tdarr && \
  groupdel Tdarr && \
  groupadd -g ${PGID} Tdarr && \
  useradd -l -u ${PUID} -g Tdarr Tdarr && \
  chown -R ${PUID}:${PGID} /home/Tdarr/Tdarr

USER ${PUID}:${PGID}

[carlfischerjba]

@jeff47 I guess the base image sets USER tdarr which means that user is being used to run the build itself. Switching user first might work (it should at least allow the build to proceed, I don't know whether it will cause problems at runtime).

Looking at the Dockerfile for the base image I see one problem for the tdarr image is that it runs the main process with sudo. This only works because the Tdarr user is added to the sudoers file earlier in the build. You won't be able to use sudo as another user unless you add them too.

In addition to setting the user it installs things in that user's home directory /home/Tdarr. This may not be accessible to any other non-root user unless you change the permissions on that directory or rebuild your own image. Even if permissions are correct, there may be problems because files are expected to exist in the user's home directory. Setting HOME or somehow pointing the new user's home directory to to the existing /home/Tdarr might help, or maybe you could keep the Tdarr user and just change its id in /etc/passwd.

I'm not familiar with the software in question but my understanding is that using sudo in Docker containers is generally discouraged. Also running multiple services in the same container (in this case mongodb and node) is considered bad practice. Splitting into separate containers and not using sudo might be possible but we're getting way off topic for this thread.

[evs-xsarus]

@jtreminio how did you get this working? I can't get the user ID and group ID dynamically in the command, docker returns an error. See this longer issue report by me: docker-library/php#1067

Update: got it working, ARG's need to be added below FROM too if you need them during the build process.

[schollii]

I split the Docker run into multiple steps:

1. run container in detached mode
2. exec a chmod command in container to fix the ownership
3. exec in container for a shell, or as many times as desired for command commands.

See https://devops.sentianse.com/blog/docker-run-mirror-host-user for details.

[nilsnolde]

still running strong, this one:)

one important caveat: if the directory, which one bind-mounts to the container, does not exist yet on the host machine, that folder will be created by docker and, this took me several hours to understand, that folder will have root owner with 755 permissions. if the container (and its internal non-root user) then tries to create files in that directory it will consequently fail with permission denied.

it doesn't matter what one does in the image instructions, the problem is the bind mount. it works with named (docker managed) volumes etc. and there seems to be absolutely no way to work around that since it happens at container creation time and there's no way hooking into that. once it's created you also can't chown that folder in an entrypoint script since that'd require root permissions and the whole point of this exercise is to not run a container with root permissions.

if anyone finds a proper solution I'm all ears. in the meantime I simply check in the entrypoint script if the directory inside the container has the proper owner and permissions and fail the run if it doesn't, notifying the user to create that directory before starting the container. that's the best UX I could come up with..

[davola]

Excellent post @jtreminio 👏👏👏
This solved all my problems!

Here I leave the Alpine version of your fix, but all the kudos are for you!

FROM php:8-alpine

ARG USER_ID=1000
ARG GROUP_ID=1000

RUN deluser --remove-home www-data &&\
    addgroup -S -g ${GROUP_ID} www-data &&\
    adduser -S -u ${USER_ID} -D www-data www-data &&\
    chown www-data:www-data -R /app

[akblissweb]

Last item on the "dockerise my world" todo list... make my containers run as non-root users. Aha ... and here I find the absolute Bible on the subject. Well done :-)

[grimpows]

hey guys, what about this :
FROM whatever_image_you_want!!!!

ARG USER_ID=1000
ARG GROUP_ID=1000

RUN find / -not -path "*/proc/*" -group www-data -exec chgrp -h ${GROUP_ID} {} ;
RUN find / -not -path "*/proc/*" -user www-data -exec chown -h ${USER_ID} {} ;

RUN usermod -u ${USER_ID} www-data
RUN groupmod -g ${GROUP_ID} www-data

i'm excluding */proc/* cose otherwise this trigger an error while building (dont know why, but this should refer to some docker internal configuration)
here we change the id and gid from www-data but could be any other user

i'm actually trying this possibility, but not sure if that work well yet ! :)

[schollii]

WARNING: Do NOT use chown -R in a docker build, unless there are only a few files (like 10 max). The overlay filesystem used by docker makes recursive chown is very expensive command. Like 500 files will take several minutes. There are posts of people seeing that particular step of the build increase by 10 minutes when they added the chown -R.

If you think about it, chown -R is not often needed, because most files only need to be readable. A very few need to be executable (such as nodejs modules, etc) and they are often in just one or two folders; and almost never do files need to be writable, only a few folders so that the application can create temporary files like log output, __ENV.js etc depending on your tech stack.

Why give your process the abiilty to overwrite files that it should never overwrite? This is increasing the attack surface for hackers.

So most likely you can get away with chown on just a couple of folders, with no -R.

Alternatively use COPY --chown but based on the above I doubt that you really want this.

Follow principle of least privileges: Focus on making all necessary files readable, only make a few folders writable (/tmp etc).

[grimpows]

the fact that i'm using -R (using find /) is that i'm not always aware of with folder where created/used during the build process of a docker image i didnt builded myself.
for apache dockerImage builded the conf.d path may differ from image to other(ex :/etc/apache or /usr/etc/local/apache ...) , but ofc instead of using find /, if you are aware of wich folder need to be exaclty chown, its better to be more explicit thant just pointing the root folder.

also note that i'm filtering all folder/file with a specific -group and -user to filter only file that are already owned by www-data so i'm not applying a chown on ALL file, but only previous file that was owned by www-data with older ID, and i apply the new ID to those folders/files :)

for php-apache and another container with php and composer, the build time for those command are instant-like ...

also note that using this solution make the remove of www-data (or any other user/group) not necessary anymore, and also let you modify permission without be aware of wich file was created previously by this user during the build process of the image you are importing FROM.

to be more clear, i'm not using a chmod -R at all, i'm just applying new user_id/group_id for file owned by www-data then i change the uid/gid of www-data to match this specific user_id/group_id.

in fact when i do a exec sh with docker inside my container root owned file are still owned by root but for exemple var/html are owned by www-data with the proper user_id/group_id defined with ARG.
finally i'm getting exactly same result as the solution presented here, but without have to deep inside dockerfiles from other image to know wich file need to be changed with chown...

edit : i guess you missreading what my command do ... read it again cose "Focus on making all necessary files readable, only make a few folders writable (/tmp etc)." is exactly what those command do actually.

[grimpows]

here is a exemple for composer with php-fpm

dockerfile :
FROM composer:$COMPOSER_VERSION as selectedcomposer

FROM php:${PHP_VERSION}-fpm

ARG USER_ID=1000
ARG GROUP_ID=1000

RUN apt-get -qqy update;
apt-get -qqy upgrade;

RUN apt-get -qqy install libicu-dev libpng-dev git zip unzip

RUN docker-php-ext-configure intl && docker-php-ext-install intl

RUN docker-php-ext-install mysqli pdo pdo_mysql && docker-php-ext-enable pdo_mysql

RUN docker-php-ext-install gd

COPY --from=selectedcomposer /usr/bin/composer /usr/bin/composer

WORKDIR /var/www/app/

RUN find / -not -path "*/proc/*" -group www-data -exec chgrp -h ${GROUP_ID} {} ;
RUN find / -not -path "*/proc/*" -user www-data -exec chown -h ${USER_ID} {} ;

RUN usermod -u ${USER_ID} www-data
RUN groupmod -g ${GROUP_ID} www-data

RUN chown www-data:www-data /var/www
RUN chown -R www-data:www-data /var/www

USER www-data

RUN mkdir /var/www/.composer

docker-composer.yml
version: '3.8'
services:
composer:
build:
context : ../${DOCKERFILE_CONTEXT_PATH}/dev-tools/composer
args:
COMPOSER_VERSION: ${COMPOSER_VERSION}
PHP_VERSION: ${PHP_VERSION}
USER_ID: ${USER_ID}
GROUP_ID: ${GROUP_ID}
volumes:
- "../${HOST_SRC_PATH}${COMPOSER_SRC}:/var/www/app"
- "${COMPOSER_CACHE_DIR}:/var/www/.composer"
restart: "no"
environment:
VIRTUAL_HOST: ${VIRTUAL_HOST}
DATABASE_NAME: ${MYSQL_DATABASE}
DATABASE_HOST: ${DATABASE_HOST_NAME}
DATABASE_USER: ${MYSQL_USER}
DATABASE_PASSWORD: ${MYSQL_PASSWORD}

networks:
backend:

and the output of ls -l from /var
drwxr-xr-x 2 root root 4096 Mar 19 13:46 backups
drwxr-xr-x 1 root root 4096 Mar 28 00:00 cache
drwxr-xr-x 1 root root 4096 Apr 2 14:27 lib
drwxrwsr-x 2 root staff 4096 Mar 19 13:46 local
lrwxrwxrwx 1 root root 9 Mar 28 00:00 lock -> /run/lock
drwxr-xr-x 1 root root 4096 Mar 29 01:15 log
drwxrwsr-x 2 root mail 4096 Mar 28 00:00 mail
drwxr-xr-x 2 root root 4096 Mar 28 00:00 opt
lrwxrwxrwx 1 root root 4 Mar 28 00:00 run -> /run
drwxr-xr-x 2 root root 4096 Mar 28 00:00 spool
drwxrwxrwt 2 root root 4096 Mar 19 13:46 tmp
drwxr-xr-x 1 www-data www-data 4096 Apr 2 15:19 www

outpout of /etc/passwd :
www-data:x:1000:1000:www-data:/var/www:/usr/sbin/nologin

everything work perfectly :)
now my composer container from docker-compose is able to handle cache with uid/gid based on WSL2 uid/guid for windows ^^

[vladanghene]

Hi @jtreminio, while this is a fascinating and insightful read, from whence I learned a lot even on topics outside the main focus, I must ultimately defer to standard practices and NOT use your quote unquote solutions because frankly, they are really unsecure workarounds that bring more hassle that they fix.
Yes, you made some valid points, yes this is still a problem with docker. But it's an unfixed problem for a reason.

[jhyland9]

You mentioned that this is not necessary when running docker on windows. when I cli into the running container, items are owned by root and the abc user. If i look in the passwd file, I don't see the host user from the windows machine.

[dss010101]

great article - i learned a lot here. some questions:

1. what does this line do: install -d -m 0755 -o www-data -g www-data /home/www-data &&\
2. does the user name matter here, or just the ids need to match the host user id and group?
3. ive read an option where mounting /etc/passwd and group is solution. any thoughts on how to handle cases where user/groups are managed by a directory service (where in that cases users may not exist in /etc/passwd for instance)?