Using the Docker command-line

../_images/docker-registry.gif

In this section you’ll learn to make use of docker images, created by others, by using the command line.

Docker has a very active community and most everything you can think of and is publicly available has been build into a docker image by someone already. It makes trying out new things very easy.

At the end of this section you should:

  • Be more familiar with most commonly used docker commands, like: run, exec, ps, rm, rmi, attach and probably some more

  • Know how to start and stop a container in different ways and know when to use which

  • Have working knowledge of docker processes

  • Have learned a few best practices

  • Know how to cleanup your docker environment

  • Be familiar with terms like: registry, image, container

Tip

Lazy programmers (like me) might want to clone this git repository as some files you are asked to use or create will already be provided here:

git clone https://github.com/IvoNet/docker-from-scratch.git

Hello, world!

Attention

Commandline commands have only been tested with bash/(z)sh (mac/linux) and cmd (Windows). Not with PowerShell! PowerShell will probably work just fine but some commands may need something different. This is not taken into account in this workshop! Use at your own risk :-)

Let’s start by doing a Hello, world!. It might comfort you to know that this time honoured tradition will not be forsaken in this Hands-on Lab 😄.

Exercise

Open a terminal and run

docker run ivonet/hello-world

Your terminal should show something like:

Unable to find image 'ivonet/hello-world:latest' locally
latest: Pulling from ivonet/hello-world
9b18e9b68314: Already exists
3f715686f3c8: Already exists
Digest: sha256:970514d359b523145d8a9cbedcef62cfa9a58560a477970dca87e07b53b5141d
Status: Downloaded newer image for ivonet/hello-world:latest
 _   _      _ _                             _     _ _
| | | | ___| | | ___    __      _____  _ __| | __| | |
| |_| |/ _ \ | |/ _ \   \ \ /\ / / _ \| '__| |/ _` | |
|  _  |  __/ | | (_) |   \ V  V / (_) | |  | | (_| |_|
|_| |_|\___|_|_|\___( )   \_/\_/ \___/|_|  |_|\__,_(_)
                    |/

Because it was the first time you ran the command and the image was not yet present on your host machine it had to pull it from the registry (hub.docker.com) before it could be run. Docker will do this automatically for you.

Essentially the run command did this:

docker pull ivonet/hello-world
docker run ivonet/hello-world

If you run the command again, only the Hello, world! message will be displayed because the image has already been pulled. It will therefore execute much faster.

Note

The docker run command uses this basic syntax:

docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

Where all the [OPTIONS] are to configure the IMAGE and where the [COMMAND] [ARG ... ] are performed on the IMAGE.

to get more help you can look at the commandline reference or try:

docker run --help

This can be done for any docker command.

This docker container will only run long enough to print the Hello, world! message to the stdout and then stop. A container is made from the Docker image, as the GIF on the top of this page illustrates. Every time you run the same “docker run” command, a new container with its own name will be created.

In this case that is not very efficient because, except for logging, this container does not build up state and it’s behavior does not vary. Keeping the container therefore has no added value.

Exercise

What would you need to change to the above run command to run the container, but not keep the container when done?

--- Solution ---

Extra Credit question

Exercise

Try to let the ivonet/hello-world image give another figlet output

Hint

That would be the [COMMAND] [ARG ... ] part. For those who don’t know the figlet command just use:

figlet "Ok, but first Coffee"
  ___  _        _           _      __ _          _
 / _ \| | __   | |__  _   _| |_   / _(_)_ __ ___| |_
| | | | |/ /   | '_ \| | | | __| | |_| | '__/ __| __|
| |_| |   < _  | |_) | |_| | |_  |  _| | |  \__ \ |_
 \___/|_|\_( ) |_.__/ \__,_|\__| |_| |_|_|  |___/\__|
           |/
  ____       __  __
 / ___|___  / _|/ _| ___  ___
| |   / _ \| |_| |_ / _ \/ _ \
| |__| (_) |  _|  _|  __/  __/
 \____\___/|_| |_|  \___|\___|
--- Solution ---

Registry

Many images can be found in the official docker registry, also called the docker hub. You can search for an image with a specific functionality (e.g. an image with MySQL installed) by going to the docker hub website and search for it there, but you can also search through the command line:

docker search mysql

Exercise

Try out this command yourself.

Note

Sometimes you see names like user/image-name and sometimes you only see image-name. The images with only a single name are so called Official images.

Docker Official Images are a curated set of Docker open source and “drop-in” solution repositories.

Processes

../_images/docker-image-container.gif

Note

A container can be seen as an instance of an image, but it is not exactly the same kind of instance as a programming language (e.g. java / python). In java an instance is always “running” and that is not the case with Docker. A docker container (instance) can either be running or stopped.

In the hello, world! exercise you have probably created a couple of containers as you will see in the next exercise.

View Processes

Let’s find the hello-world example in the docker processes.

Hint

Many docker sub-commands have the same “command” as their equivalent in linux.

e.g. looking at running processes in linux is done with the ps command, so if you want to see running docker processes that would translate to docker ps [options].

Exercise

With what command can you see stopped containers?

If you used the correct command you should see something like the listing below with some data like the CONTAINER ID and NAMES different. Depending on how many times you ran the Hello, world! example you might see less or more entries in the listing.

CONTAINER ID        IMAGE                 COMMAND                  CREATED              STATUS                          PORTS                    NAMES
b8a8a56c1a5a        ivonet/hello-world    "figlet -w 1000 'Hel…"   About a minute ago   Exited (0) About a minute ago                            cocky_mestorf
a028c2e6921c        ivonet/hello-world    "figlet -w 1000 'Hel…"   12 minutes ago       Exited (0) About a minute ago                            beautiful_tereshkova

You see that the CONTAINER ID’s and the NAMES are all different. This shows that for each new run a new container was created.

But why would we want this? The answer in this case is that we don’t. It does illustrate nicely though that we have now created multiple containers based in 1 image and all these containers are now in the stopped state.

--- Solution ---

Start container

Exercise

What happens when you start an existing ivonet/hello-world container with docker start <CONTAINER ID>?

Tip

You can start a stopped container with the docker start [ID|NAME] command. The ID can be just a unique part from the start of the CONTAINER ID or the given NAME.

e.g. docker start b8 would already be unique in the above mentioned list of processes and therefore enough for docker to identify the container.

Exercise

Did you see the Hello, world! message again? Why (not) do you think? Do you think the message was printed? Where would that be? Can we still see the output?

Hint

Where does a container writes its output to? how can you see this log?

--- Solution ---

Stopped containers when started again will always run in daemon mode, also called detached mode. That means that it is runs as a background process and no output is written to the active console. It is possible though to attach to a running process…

Foreground / Background process

Let’s start a foreground process.

docker run --name shell -it busybox /bin/sh

Exercise

Explain what this command does with all it’s parameters (the -it option could also have been written like this: -i -t).

--- Solution ---

You should see a new prompt. This is the /bin/sh prompt provided by the busybox container. A shell is a long running process, so the container keeps on running. You can see that by opening a new terminal and performing the docker ps command.

Note

If you want to exit (stop) this container from the provided prompt you can do this by ctrl-d or exit as that will exit the shell. For other containers running in interactive TTY mode this might be something else.

Exercise

How would you stop this process with a docker command? (from another terminal window)

If all went well you can see that the container is not running anymore in the first terminal.

--- Solution ---

If you stop a container it will still exist as a stopped container unless the --rm option was provided in the run command. You should not see this container with the docker ps command as that will only show running processes, but should be visible with the docker ps -a command.

Exercise

  • Try these commands yourself.

  • start the existing container called ‘shell’ again.

If all went well you only got shell as feedback in the terminal. You should also be able to see the container in the running processes.

The trouble is that we want to access the shell again but now it is running in detached mode.

Exercise

Try to attach to the running container called shell.

If all went according to plan you should see the shell prompt from the container. Commands you perform at this point are performed within the container.

--- Solutions ---

The next logical step is to remove a container.

Exercise

  • Remove the container called ‘shell’

  • Remove the container(s) from the hello-world example

Tip

If the container is still running you might want to stop it first or force the removal.

Warning

If you --name a container yourself and try to run the same command again to create a new container it will fail with a message like:

docker: Error response from daemon: Conflict. The container name "/shell" is already in use by container "5f9da09c00b8d5e3c7690a978127f5bf2fdd328fec0d2ed44e4b44df0d8fcc88". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.

Names of containers must be unique on the host machine. If no name is provided docker will generate a unique name itself. They can be funny sometimes.

--- Solution ---

So now you should be able to:

  • search for images in the registry

  • pull an image from the registry

  • run a basic container in the background (detached) of in the foreground

  • use ps [-a] to view containers either running or stopped

  • stop and start an existing container

  • attach to a detached running container

  • remove (rm) a container

Ports

Running Linux commands inside a Docker container is fun, but let’s do something more useful.

Until now you have created your own long running process by providing one on the commandline to an image with no special function of its own. The whole goal of docker is to have specialized containers, with everything in it, doing a specific thing.

Let’s look at such a specialized docker container next and learn about exposing functionality.

A basic webserver

Exercise

  • Pull the ivonet/nginx-example Docker image from the Docker hub (registry). This image uses the nginx webserver to serve a static HTML website.

  • Start a new container from the ivonet/nginx-example image that publishes port 80 from the container to port 3000 on your host.

  • Go in a browser to http://localhost:3000

  • If you did it right you should see a nice green message 😄

  • If you started the container as a foreground process you will now also see logging done by nginx to the console.

  • Stop the container

  • Start the container again and look at the message again (should work)

  • Show the logs with a docker command (tail them)

  • Remove the container

--- Solution ---

Mapping ports between your host machine and your containers can be a bit confusing. Docker does application level isolation, which means that you have no access to the ports exposed in a container unless you explicitly make them available when creating the container.

The trick is to remember that the host port always goes on the left, and the container port always goes on the right.

Think of it from the perspective of an incoming request, which would first go through port 3000 on your host and then be forwarded to port 80 on your container.

Tip

If you want to detach from a foreground process without stopping the container you can do that by pressing ctrl+p and ctrl+q after each other.

Extra Credit question

Run the above server again with docker run --name coolness -d -p 3000:80 ivonet/nginx-example and prove that it works.

Now you have a docker container running in the background called coolness, serving the nice message.

Exercise

  • How can you execute other commands on this container while it is running?

  • can you start a /bin/sh shell and execute commands?

    • try ps -ef

    • What number (PID) does the main process have?

--- Solution ---

Volumes

Many containers out there need to save data. For example if you have a MySQL docker container and you are creating a database with tables and data, it needs to be saved somewhere. The state of a container will be saved, so you can save your data inside the container. Though, this is generally a bad idea:

  • Your software (MySQL) and your data are now coupled (no separation of concerns)

  • What if you don’t need the mysql software for a while but don’t want to lose your data?

  • If you remove your container you lose your data too.

  • Upgrading your MySQL software becomes almost impossible without losing your data.

  • Backing up your data is much more difficult.

All good reasons not to save data in the same container as where the software runs. To save your data in Docker, you can (and should) use volumes.

Volumes are a way of sharing data either with the host machine or in a volume container.

Host Volumes

First let’s take a look at sharing data with the host machine

Exercise

  • Create a directory in your project directory called data

  • Create a file called host.txt in the data folder

  • Run the ivonet/nginx-example image with the following options:

    • as an interactive tty foreground process

    • opening /bin/sh

    • mount the local data volume to the container /data folder

  • Show what is in the /data directory of the container

Tip

You can use the option -v <host/path>:<container/path> to mount a data volume to a container. Note that to mount a directory from the host machine you need to provide a fully qualified path. As with ports the host path is on the left and the container path on the right.

Exercise

  • While in the container create a new file called /data/container.txt (you can create a file using the touch command)

  • Exit the container and see what is now in the <host>/data directory

Warning

If you do not use fully qualified paths you are actually using the volume container option.

--- Solution ---

On the web page of nginx on docker hub you get a very nice description what it can do and how you can execute commands. The ivonet/nginx-example image is derived from that image.

One of the things you can do is mount a volume where nginx is actually watching for files.

# Linux / Mac
docker run --rm -d -p 3001:80 -v $(pwd)/data:/usr/share/nginx/html ivonet/nginx-example

# Windows
docker run --rm -d -p 3001:80 -v "%CD%\data":/usr/share/nginx/html ivonet/nginx-example

# Windows Powershell
docker run --rm -d -p 3001:80 -v "$((pwd).Path)\data:/usr/share/nginx/html" ivonet/nginx-example

Exercise

  • What happens if you goto http://localhost:3001?

  • and what happens if you create a file in the data directory called index.html with the following content and refresh the page?

index.html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>hello</title>
</head>
<body>
   <h1>Hello, NGINX!</h1>
</body>
</html>
--- Solution ---

Now you have nginx running in a docker container listening on its port 80 in the /usr/share/nginx/html directory. That directory is mounted to a host machine directory and you can edit files which are picked up immediately by the container. This feature is very handy during development. You do not need to regenerate a docker image every time you make a change, but can live see the changes you make and when the time comes that you are happy with the result you can build a real image with the data.

Very cool stuff 😄!

Think about it. If you use an Application Server like Payara to serve your thin Jakarta EE war, but you did not install Payara natively on your laptop, how do you deploy and test? By building a docker image from it and running it. But Payara also has an auto deploy folder, so what if you could just build your war and place it in a certain host folder that has been mounted to the autodeploy directory of Payara in the container? Then you suddenly have a much faster development cycle. Read more about it on this blog post (optional).

Data Volumes

The second way of saving data is in another special container, the so called Data Volume. Data volumes are container with no other function than storing data.

Docker will manage where the container is saved.

Note

Usage:    docker volume COMMAND

Manage volumes

Commands:
  create      Create a volume
  inspect     Display detailed information on one or more volumes
  ls          List volumes
  prune       Remove all unused local volumes
  rm          Remove one or more volumes

Run 'docker volume COMMAND --help' for more information on a command.

You can start the same example as before, but now with a data-volume, as follows:

docker run --rm -d -p 3001:80 -v my-data:/usr/share/nginx/html ivonet/nginx-example

Just by leaving out the path on the left side of the -v option you are creating a data volume inside a container at /my-data.

Note

  • Volumes are initialized when a container is created. If the container’s base image contains data at the specified mount point, that existing data is copied into the new volume upon volume initialization. (This does not apply when mounting a host directory).

  • Data volumes can be shared and reused among containers.

  • Changes to a data volume are made directly.

  • Changes to a data volume will not be included when you update an image.

  • Data volumes persist even if the container itself is deleted.

Exercise

  • do docker run --rm -d -p 3001:80 -v my-data:/usr/share/nginx/html ivonet/nginx-example

  • Look at the volume just created

  • How would you list the contents of the data volume? (Brain teaser!)

  • Create your own data volume with the docker volume ... command

  • How would you copy (docker cp --help) the index.html (former exercise) to this volume? (Brain teaser!)

  • Check if you actually copied something to that volume by using another container.

--- Solution ---

Inspect containers

You can find any kind of information about a container by inspect -ing it. In common usage you will not often need to use this command, but it is good to know about it.

Below are some examples performed on the Virtual Machine:

- Inspect full example (show/hide)
- Inspect example with filter (show/hide)

See the Docker Inspect reference for more information.

Images

Above you explored how to start and stop containers. Containers are instances of images. Images can be viewed with the docker images command.

As the images are the base of all the containers you create, you can accumulate quite a few over time.

Layers

Docker images are like onions. Not because they stink or make you cry but because they have layers. We will go into layers in the Creating Docker images section, but here is a short introduction:

For example if you compare the history of the nginx image with the history of the ivonet/docker-from-scratch` image you will see something like:

$ docker history nginx:1.17.3-alpine
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
41c8c3458a93        11 days ago         /bin/sh -c #(nop)  CMD ["nginx" "-g" "daemon…   0B
<missing>           11 days ago         /bin/sh -c #(nop)  STOPSIGNAL SIGTERM           0B
<missing>           11 days ago         /bin/sh -c #(nop)  EXPOSE 80                    0B
<missing>           11 days ago         /bin/sh -c set -x     && addgroup -g 101 -S …   15.7MB
<missing>           11 days ago         /bin/sh -c #(nop)  ENV PKG_RELEASE=1            0B
<missing>           11 days ago         /bin/sh -c #(nop)  ENV NJS_VERSION=0.3.5        0B
<missing>           11 days ago         /bin/sh -c #(nop)  ENV NGINX_VERSION=1.17.3     0B
<missing>           11 days ago         /bin/sh -c #(nop)  LABEL maintainer=NGINX Do…   0B
<missing>           11 days ago         /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B
<missing>           11 days ago         /bin/sh -c #(nop) ADD file:fe64057fbb83dccb9…   5.58MB

$ docker history ivonet/docker-from-scratch
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
6a77a4bebb74        29 hours ago        /bin/sh -c #(nop) COPY dir:e1baa3f8487c01720…   69.7MB
<missing>           11 days ago         /bin/sh -c #(nop)  CMD ["nginx" "-g" "daemon…   0B
<missing>           11 days ago         /bin/sh -c #(nop)  STOPSIGNAL SIGTERM           0B
<missing>           11 days ago         /bin/sh -c #(nop)  EXPOSE 80                    0B
<missing>           11 days ago         /bin/sh -c set -x     && addgroup -g 101 -S …   15.7MB
<missing>           11 days ago         /bin/sh -c #(nop)  ENV PKG_RELEASE=1            0B
<missing>           11 days ago         /bin/sh -c #(nop)  ENV NJS_VERSION=0.3.5        0B
<missing>           11 days ago         /bin/sh -c #(nop)  ENV NGINX_VERSION=1.17.3     0B
<missing>           11 days ago         /bin/sh -c #(nop)  LABEL maintainer=NGINX Do…   0B
<missing>           11 days ago         /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B
<missing>           11 days ago         /bin/sh -c #(nop) ADD file:fe64057fbb83dccb9…   5.58MB

You can see that the ivonet/docker-from-scratch is the same image as the nginx image, but with an extra layer. You could say that the ivonet/docker-from-scratch image inherits FROM the nginx image.

Exercise

Take a look at histories of some of the other images on your machine.

At this point you should be:

  • More comfortable with the docker cli ;-)

  • Publish ports (-p).

  • Use (data-)volumes (-v)

  • Inspect containers (docker inspect ...)

  • Understand the image layer system and see its history

Next you will start creating your own docker images…