4 Easy Steps to Set Up a Private Docker Registry on Ubuntu

Once you start working with the docker, you will eventually find that you want a bit more control over the images you want to deploy your container on. Here Docker Registry plays an important role, it helps you to centralize your container images and also reduce the build time for you and your team. Benefits of registries don’t stop here, you can integrate it with your continuous integration/continuous deployment (CI/CD) pipelines, by automating the image push process to a private docker registry, which helps to update the production or development environment on the go.

Docker does provide a free publicly available registry known as Docker Hub, that hosts custom build Docker images. But this is not ideal when you are working on proprietary software or web application, as it contains all the necessary code to run an application. Hence here comes the Private Docker Registry to rescue.

This tutorial will help you to set up and secure your own private Docker Registry. Below are the mentioned prerequisites before we begin 4 step guide:

1. We need 2 Ubuntu 18.04 servers with sudo privileges. First will act as a client server, and second will be a private Docker Registry.
2. Both systems should have Docker and Docker Compose.
3. Static IP to point your domain.
4. A domain name or a sub-domain (whichever you prefer) that point/resolve registry server.
5. SSL for the private registry server. We will use Let’s Encrypt with Nginx.

So let’s begin the guide…

Step 1: Setting Up the Docker Registry

The Docker command line is perfect for managing 2 or 3 containers. But if we speak of deploying a full application deployment, which often requires few other components running parallely, then it can be a bit overwhelming.

With Docker Compose, we can create a .yml file that helps us to set up each container’s configuration and establish communications between them. Now, the Docker registry as an application consists of multiple components, so we are going to use Docker Compose to manage configuration.

Just in case if private Docker Registry Server doesn’t have docker-compose yet, then follow the below-mentioned steps.

Installing Docker Compose on Linux Systems

First, you need to download the Docker Compose binary from the Compose repository, then make the binary executable:

$ sudo curl -L "https://github.com/docker/compose/releases/download/1.25.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose \
&& sudo chmod +x /usr/local/bin/docker-compose

Now test the installation:

$ docker-compose --version
docker-compose version 1.25.0, build 4667896b

In case your installation fails, then you might need to create an extra symbolic link:

$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

Note that the above-mentioned steps are strictly for Linux based systems. If you are on another OS, follow the instructions mentioned on this link to install docker-compose.

Setting Up Private Docker Registry

On the server follow the below-mentioned steps to create your own private Docker Registry. First, we are going to create a directory, move into it and then create a sub-directory to store our data:

$ mkdir /your/preferred/path/my-pdr && cd $_
$ mkdir main

Create the docker-compose.yml configuration file for our registry in the my-pdr directory, and open it in a preferred text editor:

$ touch docker-compose.yml && nano docker-compose.yml

Add the below described basic configuration for Docker registry:

version: '3.7'
 
services:
 registry:
   restart: always
   image: registry:2
   ports:
   - "5000:5000"
   Environment:
     REGISTRY_AUTH: htpasswd
     REGISTRY_AUTH_HTPASSWD_REALM: Registry
     REGISTRY_AUTH_HTPASSWD_PATH: /a2auth/registry.password
     REGISTRY_HTTP_SECRET: SomeRandomStringToUse
     REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /main
     REGISTRY_STORAGE_DELETE_ENABLED: ‘true’
   volumes:
     - ./main:/main
     - ./a2auth:/a2auth

Let’s break-down the above configuration,

Restart Policy:

We need to ensure that if our system is forcefully stopped or a planned system reboot is required, then registry server restarts as the system boots. Hence we are going to set restart:always.

Image Section:

In the image section, you have to use Docker’s official image https://hub.docker.com/_/registry with tag 2. You can use any other or latest image as per your requirement, but for the sake of this tutorial, we are going to stick with registry:2 image.

Port Section:

In the port section, we had used the default port number as used in registry image which is 5000, it tells Docker to map the port 5000 on the server to port 5000 in the running container. In case if 5000 port is already occupied, then you can use any other port as per your wish.

Environment Section:

In the environment section, we had set a couple of environment variables in the Docker Registry container, we are going to break this into two parts as mentioned below:

Storage:

1. REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY with the path /main, as application detects the variable during startup, it will start saving data to the defined directory, i.e. /main.
2. REGISTRY_STORAGE_DELETE_ENABLED it is set to true, otherwise, Docker Registry container will not support deleting images.

Authentication:

You can use a basic authentication mechanism to manage the access to your private Docker Registry. For this, you can create an authentication file with htpasswd and add users to it.

You can install a htpasswd package by running the following command:

$ sudo apt install apache2-utils

Create a directory to store the credentials:

$ mkdir /your/preferred/path/my-pdr/a2auth && cd $_

To create the first user, you can use below command. It will prompt you to enter the password, as you enter the password make a sure you copy it somewhere safe. As this will be used to login into your private registry:

$ htpasswd -Bc registry.password yourusername

You can use flag -B to specify bcrypt, which is more secure than default encryption and -c to create a new user.

1. REGISTRYAUTH, by this we have specified htpasswd, which is the authentication schema we used.
2. REGISTRYAUTHHTPASSWDPATH, it points to the authentication file which we had created for the user.
3. REGISTRYAUTHHTPASSWDREALM, it implies the name of the htpasswd realm.
4. REGISTRYHTTPSECRET, you can use any long-string format random string into the variable, or you can generate one from your system itself by running this command: cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 32

Volume Section:

You had also mentioned volumes section in the configuration file, which helps Docker to map the /main directory inside the container to /main on registry server. So by the end of the day, all data which is sent to registry container will get stored in /your/preferred/path/my-pdr on the registry server.

Now it’s time to put our configuration to test. You can do this by running the following command:

$ docker-compose up

You will see the below mentioned output:

Creating network "registry_default" with the default driver
Pulling registry (registry:2)...
2: Pulling from library/registry
c87736221ed0: Pull complete
1cc8e0bb44df: Pull complete
54d33bcb37f5: Pull complete
e8afc091c171: Pull complete
b4541f6d3db6: Pull complete
Digest: sha256:8004747f1e8cd820a148fb7499d71a76d45ff66bac6a29129bfdbfdc0154d146
Status: Downloaded newer image for registry:2
Creating registry_registry_1 ... done
Attaching to registry_registry_1
registry_1  | time="2019-12-31T09:21:37.028548595Z" level=info msg="redis not configured" go.version=go1.11.2 instance.id=3a07596d-d918-4b9f-ac80-e0b3f9ae1e1a service=registry version=v2.7.1 
registry_1  | time="2019-12-31T09:21:37.028661776Z" level=info msg="Starting upload purge in 55m0s" go.version=go1.11.2 instance.id=3a07596d-d918-4b9f-ac80-e0b3f9ae1e1a service=registry version=v2.7.1 
registry_1  | time="2019-12-31T09:21:37.040850986Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.11.2 instance.id=3a07596d-d918-4b9f-ac80-e0b3f9ae1e1a service=registry version=v2.7.1 
registry_1  | time="2019-12-31T09:21:37.041642152Z" level=info msg="listening on [::]:5000" go.version=go1.11.2 instance.id=3a07596d-d918-4b9f-ac80-e0b3f9ae1e1a service=registry version=v2.7.1

Voila… The output indicates that our container is starting and running on port 5000. For now, hit the CTRL+C to shut down your private Docker Registry.

Step 2: Configuring Nginx for Port Forwarding

Now you have to set up port forwarding via Nginx to container’s port which is running on port 5000. Once this step is complete, you can access the private registry at your defined domain or subdomain.

This guide is assuming you already have set up Nginx with Let’s Encrypt. Now you need to create a server configuration file –

$ cd /etc/nginx/conf.d && nano yourdomainname.com.conf

Here you have to forward traffic to port 5000, on which your existing Docker Registry container is going to run. You need to append the additional information from the server to registry for each request and response in the header.

The Server configuration file should look something like below. Note that you need to run certbot to install the SSL for the domain.

http {
  upstream docker-registry {
    server registry:5000;
  }
map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
    '' 'registry/2.0';
}
server {
   listen 80;
   listen [::]:80;    
   server_name yourdomainname.com;
   return 301 https://$host$request_uri;
}
 
server {
   listen 443 ssl http2;
   listen [::]:443 ssl http2; 
 
       server_name yourdomainname.com;
       ssl_certificate      /etc/letsencrypt/live/yourdomainname.com/fullchain.pem;
       ssl_certificate_key  /etc/letsencrypt/live/yourdomainname.com/privkey.pem;
 
       location /v2/ {
       if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
         return 404;
       }
      add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
       proxy_pass                          http://docker-registry;
       proxy_set_header  Host              $http_host;
       proxy_set_header  X-Real-IP         $remote_addr;
       proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
       proxy_set_header  X-Forwarded-Proto $scheme;
       proxy_read_timeout                  900;
       }
}

The $http_user_agent block verifies that whether Docker version of the client is above 1.5 or not because here we are using version 2.0 of the registry. UserAgent ensures that it is not a Go application. You can find more information on Nginx header configuration here.

Now it’s time to test the server configuration file:

$ sudo nginx -t

You will see the following output:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

In case your test fails, then you can go through the Nginx error log which usually resides in /var/log/nginx/ direcorty, it will help you to understand what went wrong.

Next, you need to change the Nginx’s default file upload limit which happens to be the only 1MB. As Docker splits large image uploads into separate layers, sometimes they can get as big as 1GB, so you need to ensure that the registry can handle large file uploads. To do so, you need to tweak client_max_body_size limit in /etc/nginx/nginx.conf file.

Open the file and find http section:

... 
http {
	client_max_body_size 3000M;
	...
}
...

Again, test your changes and restart the Nginx.

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
 
$ sudo systemctl restart nginx

Here you can see that the test is successful and you had changed the max upload size to 3GB.

To confirm that Nginx is forwarding traffic to port 5000, open a browser window and enter your URL:

https://yourdomainname.com/v2/

You will see a prompt in your browser, as the authentication process is in the play. Enter the username and password you created earlier, and you will see an empty JSON object:

{}

Whereas, if you go to your terminal, you will find the output similar to the following:

registry_1  | time="2019-12-31T09:21:37.041642152Z" level=info msg="response completed" go.version=go1.7.6 http.request.host=cornellappdev.com http.request.id=a8f5984e-15e3-4946-9c40-d71f8557652f http.request.method=GET http.request.remoteaddr=128.84.125.58 http.request.uri="/v2/" http.request.useragent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.125995ms http.response.status=200 http.response.written=2 instance.id=3093e5ab-5715-42bc-808e-73f310848860 version=v2.6.2
registry_1  | 172.17.0.2 - - [31/Dec/2019:09:21:38+0000] "GET /v2/ HTTP/1.0" 200 2 "" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7"

200 Response code in the last line indicates that the container handled the request successfully.

Step 3: Publishing Image to Private Docker Registry

For the sake of this tutorial, we will create a simple image based on the alpine image from Docker Hub.

Going to your client-server, run the following command:

$ docker pull alpine
edge: Pulling from library/alpine
d95bb1b66adb: Pull complete 
Digest: sha256:2e8c50cbe65693cdf3e6c3822f23ee3e07a7d92fd891d0a5ed9710aedd05ee19
Status: Downloaded newer image for alpine
docker.io/library/alpine
 
$ docker run -it --name test-alpine alpine /bin/sh

Flag -it gives you interactive shell access into the container. Through which you acan create a file to test the publishing process:

root@dc59bfcd63b3:/# touch file-from-client-server.txt

Exit from the Docker container:

root@dc59bfcd63b3:/# exit

By doing so, you have created a new image, based on the image already running plus the changes you performed. Now you have to commit the changes:

$ docker commit $(docker ps -lq) your-test-image
sha256:83b7ae6c35a534d81ea0ad13fce007c2c6da39dfaf13a35faacdb358d7e12eb6

You can verify if the above command runs successfully, by entering the following command:

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
your-test-image     latest              83b7ae6c35a5        7 seconds ago       6MB

You have successfully created the image, but at this point, it only resides in the client server. Now it’s time to push your newly created image to your private Docker Registry. You will be prompted to enter the username and password:

$ docker login https://yourdomainname.com

It is always a good thing to add tags to your image as it certainly helps in the long run and helps you to identify images and push the tagged image to the registry:

$ docker tag v1 yourdomainname.com/your-test-image
$ docker push yourdomainname.com/your-test-image

Your output will look similar to the following:

The push refers to a repository [yourdomainname.com/test-image]
83b7ae6c35a5: Pushed
dec4bff59bf4: Pushed
7363c5bcdbce: Pushed
982d147b635c: Pushed
... 

Step 4: Pulling for Private Docker Registry

You have already pushed the image successfully, now you have to pull an image from the remote server into your client server. If you wish, you can also test it from another machine.

So now you are going to log in with the username password you created earlier:

$ docker login https://yourdomainname.com

Pull the image from your private Docker registry. If successful you will see the following output in your terminal:

$ docker pull yourdomainname.com/your-test-image
v1: Pulling from v2/your-test-image
d95bb1b66adb: Pull complete 
Digest: sha256:2e8c50cbe65693cdf3e6c3822f23ee3e07a7d92fd891d0a5ed9710aedd05ee19
Status: Downloaded newer image for alpine:edge
yourdomainname.com/v2/your-test-image:v1

To confirm that pull was successful, you can always run the container with the following command. You can use --rm flag so that the container will destroy itself once you exit the interactive shell:

$ docker run --rm -it yourdomainname.com/your-test-image /bin/sh

To verify the changes you made to your image, use the ls command:

$ ls
bin    dev    etc  file-from-client-server.txt  home   lib    media  mnt    opt    proc   root   run    sbin   srv    sys    tmp    usr    var

You will see the file we created previously with name file-from-client-server.txt. You can now confirm that you have set up a secure private Docker Registry through which anyone with the credentials can push and pull custom images.