Using the Docker command-line¶
In this section you’ll learn to make use of docker images, created by others, by using the command line.
Docker has a very active community and most everything you can think of and is publicly available has been build into a docker image by someone already. It makes trying out new things very easy.
At the end of this section you should:
Be more familiar with most commonly used docker commands, like: run, exec, ps, rm, rmi, attach and probably some more
Know how to start and stop a container in different ways and know when to use which
Have working knowledge of docker processes
Have learned a few best practices
Know how to cleanup your docker environment
Be familiar with terms like: registry, image, container
Tip
Lazy programmers (like me) might want to clone this git repository as some files you are asked to use or create will already be provided here:
git clone https://github.com/IvoNet/docker-from-scratch.git
Hello, world!¶
Attention
Commandline commands have only been tested with bash/(z)sh
(mac/linux) and cmd
(Windows).
Not with PowerShell!
PowerShell will probably work just fine but some commands may need something different. This is not taken
into account in this workshop! Use at your own risk :-)
Let’s start by doing a Hello, world!. It might comfort you to know that this time honoured tradition will not be forsaken in this Hands-on Lab 😄.
Exercise
Open a terminal and run
docker run ivonet/hello-world
Your terminal should show something like:
Unable to find image 'ivonet/hello-world:latest' locally
latest: Pulling from ivonet/hello-world
9b18e9b68314: Already exists
3f715686f3c8: Already exists
Digest: sha256:970514d359b523145d8a9cbedcef62cfa9a58560a477970dca87e07b53b5141d
Status: Downloaded newer image for ivonet/hello-world:latest
_ _ _ _ _ _ _
| | | | ___| | | ___ __ _____ _ __| | __| | |
| |_| |/ _ \ | |/ _ \ \ \ /\ / / _ \| '__| |/ _` | |
| _ | __/ | | (_) | \ V V / (_) | | | | (_| |_|
|_| |_|\___|_|_|\___( ) \_/\_/ \___/|_| |_|\__,_(_)
|/
Because it was the first time you ran the command and the image was not yet present on your host machine it had to pull
it from the registry (hub.docker.com) before it could be run.
Docker will do this automatically for you.
Essentially the run command did this:
docker pull ivonet/hello-world
docker run ivonet/hello-world
If you run the command again, only the Hello, world! message will be displayed because the image has already been pulled. It will therefore execute much faster.
Note
The docker run
command uses this basic syntax:
docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Where all the [OPTIONS]
are to configure the IMAGE
and where
the [COMMAND] [ARG ... ]
are performed on the IMAGE
.
to get more help you can look at the commandline reference or try:
docker run --help
This can be done for any docker command.
This docker container will only run long enough to print the Hello, world! message to the stdout and then stop. A container is made from the Docker image, as the GIF on the top of this page illustrates. Every time you run the same “docker run” command, a new container with its own name will be created.
In this case that is not very efficient because, except for logging, this container does not build up state and it’s behavior does not vary. Keeping the container therefore has no added value.
Exercise
What would you need to change to the above run command to run the container, but not keep the container when done?
Extra Credit question¶
Exercise
Try to let the ivonet/hello-world
image give another figlet
output
Hint
That would be the [COMMAND] [ARG ... ]
part.
For those who don’t know the figlet
command just use:
figlet "Ok, but first Coffee"
___ _ _ _ __ _ _
/ _ \| | __ | |__ _ _| |_ / _(_)_ __ ___| |_
| | | | |/ / | '_ \| | | | __| | |_| | '__/ __| __|
| |_| | < _ | |_) | |_| | |_ | _| | | \__ \ |_
\___/|_|\_( ) |_.__/ \__,_|\__| |_| |_|_| |___/\__|
|/
____ __ __
/ ___|___ / _|/ _| ___ ___
| | / _ \| |_| |_ / _ \/ _ \
| |__| (_) | _| _| __/ __/
\____\___/|_| |_| \___|\___|
Registry¶
Many images can be found in the official docker registry, also called the docker hub. You can search for an image with a specific functionality (e.g. an image with MySQL installed) by going to the docker hub website and search for it there, but you can also search through the command line:
docker search mysql
Exercise
Try out this command yourself.
Note
Sometimes you see names like user/image-name
and sometimes you only see image-name
.
The images with only a single name are so called Official images.
Docker Official Images are a curated set of Docker open source and “drop-in” solution repositories.
Processes¶
Note
A container can be seen as an instance of an image, but it is not exactly the same kind of instance as a programming language (e.g. java / python). In java an instance is always “running” and that is not the case with Docker. A docker container (instance) can either be running or stopped.
In the hello, world! exercise you have probably created a couple of containers as you will see in the next exercise.
View Processes¶
Let’s find the hello-world example in the docker processes.
Hint
Many docker sub-commands have the same “command” as their equivalent in linux.
e.g. looking at running processes in linux is done with the ps
command,
so if you want to see running docker processes that would translate to docker ps [options]
.
Exercise
With what command can you see stopped containers?
If you used the correct command you should see something like the listing below with some data like the CONTAINER ID and NAMES different. Depending on how many times you ran the Hello, world! example you might see less or more entries in the listing.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b8a8a56c1a5a ivonet/hello-world "figlet -w 1000 'Hel…" About a minute ago Exited (0) About a minute ago cocky_mestorf
a028c2e6921c ivonet/hello-world "figlet -w 1000 'Hel…" 12 minutes ago Exited (0) About a minute ago beautiful_tereshkova
You see that the CONTAINER ID’s and the NAMES are all different. This shows that for each new run a new container was created.
But why would we want this? The answer in this case is that we don’t. It does illustrate nicely though that we have now created multiple containers based in 1 image and all these containers are now in the stopped
state.
Start container¶
Exercise
What happens when you start an existing ivonet/hello-world
container with docker start <CONTAINER ID>
?
Tip
You can start a stopped container with the docker start [ID|NAME]
command.
The ID can be just a unique part from the start of the CONTAINER ID or the given NAME.
e.g. docker start b8
would already be unique in the above mentioned list of processes and
therefore enough for docker to identify the container.
Exercise
Did you see the Hello, world! message again? Why (not) do you think? Do you think the message was printed? Where would that be? Can we still see the output?
Hint
Where does a container writes its output to? how can you see this log?
Stopped containers when started again will always run in daemon mode, also called detached mode.
That means that it is runs as a background process and no output is written to the active console.
It is possible though to attach
to a running process…
Foreground / Background process¶
Let’s start a foreground process.
docker run --name shell -it busybox /bin/sh
Exercise
Explain what this command does with all it’s parameters (the -it
option could also have been written like this: -i -t
).
You should see a new prompt. This is the /bin/sh
prompt provided by the busybox container. A shell is
a long running process, so the container keeps on running. You can see that by opening
a new terminal and performing the docker ps
command.
Note
If you want to exit (stop) this container from the provided prompt you can do this by ctrl-d or exit as that will exit the shell. For other containers running in interactive TTY mode this might be something else.
Exercise
How would you stop
this process with a docker command? (from another terminal window)
If all went well you can see that the container is not running anymore in the first terminal.
--- Solution ---If you stop a container it will still exist as a stopped container unless the --rm
option was provided
in the run
command. You should not see this container with the docker ps
command as that will only
show running processes, but should be visible with the docker ps -a
command.
Exercise
Try these commands yourself.
start
the existing container called ‘shell’ again.
If all went well you only got shell
as feedback in the terminal.
You should also be able to see the container in the running processes.
The trouble is that we want to access the shell again but now it is running in detached mode.
Exercise
Try to attach
to the running container called shell
.
If all went according to plan you should see the shell prompt from the container. Commands you perform at this point are performed within the container.
--- Solutions ---The next logical step is to remove a container.
Exercise
Remove the container called ‘shell’
Remove the container(s) from the hello-world example
Tip
If the container is still running you might want to stop it first or force the removal.
Warning
If you --name
a container yourself and try to run the same command again to create a new container it will
fail with a message like:
docker: Error response from daemon: Conflict. The container name "/shell" is already in use by container "5f9da09c00b8d5e3c7690a978127f5bf2fdd328fec0d2ed44e4b44df0d8fcc88". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
Names of containers must be unique on the host machine. If no name is provided docker will generate a unique name itself. They can be funny sometimes.
So now you should be able to:
search
for images in the registrypull
an image from the registryrun
a basic container in the background (detached) of in the foregrounduse
ps [-a]
to view containers either running or stoppedstop
andstart
an existing containerattach
to a detached running containerremove (
rm
) a container
Ports¶
Running Linux commands inside a Docker container is fun, but let’s do something more useful.
Until now you have created your own long running process by providing one on the commandline to an image with no special function of its own. The whole goal of docker is to have specialized containers, with everything in it, doing a specific thing.
Let’s look at such a specialized docker container next and learn about exposing functionality.
A basic webserver¶
Exercise
Pull the
ivonet/nginx-example
Docker image from the Docker hub (registry). This image uses the nginx webserver to serve a static HTML website.Start a new container from the
ivonet/nginx-example
image that publishes port 80 from the container to port 3000 on your host.Go in a browser to http://localhost:3000
If you did it right you should see a nice green message 😄
If you started the container as a foreground process you will now also see logging done by nginx to the console.
Stop the container
Start the container again and look at the message again (should work)
Show the
logs
with a docker command (tail them)Remove the container
Mapping ports between your host machine and your containers can be a bit confusing. Docker does application level isolation, which means that you have no access to the ports exposed in a container unless you explicitly make them available when creating the container.
The trick is to remember that the host port always goes on the left, and the container port always goes on the right.
Think of it from the perspective of an incoming request, which would first go through port 3000 on your host and then be forwarded to port 80 on your container.
Tip
If you want to detach from a foreground process without stopping the container you can do that by pressing ctrl+p and ctrl+q after each other.
Extra Credit question¶
Run the above server again with docker run --name coolness -d -p 3000:80 ivonet/nginx-example
and prove that it works.
Now you have a docker container running in the background called coolness, serving the nice message.
Exercise
How can you execute other commands on this container while it is running?
can you start a
/bin/sh
shell and execute commands?try
ps -ef
What number (PID) does the main process have?
Volumes¶
Many containers out there need to save data. For example if you have a MySQL docker container and you are creating a database with tables and data, it needs to be saved somewhere. The state of a container will be saved, so you can save your data inside the container. Though, this is generally a bad idea:
Your software (MySQL) and your data are now coupled (no separation of concerns)
What if you don’t need the mysql software for a while but don’t want to lose your data?
If you remove your container you lose your data too.
Upgrading your MySQL software becomes almost impossible without losing your data.
Backing up your data is much more difficult.
All good reasons not to save data in the same container as where the software runs. To save your data in Docker, you can (and should) use volumes.
Volumes are a way of sharing data either with the host machine or in a volume container.
Host Volumes¶
First let’s take a look at sharing data with the host machine
Exercise
Create a directory in your project directory called data
Create a file called host.txt in the data folder
Run the
ivonet/nginx-example
image with the following options:as an interactive tty foreground process
opening
/bin/sh
mount the local data volume to the container
/data
folder
Show what is in the /data directory of the container
Tip
You can use the option -v <host/path>:<container/path>
to mount a data volume to a container.
Note that to mount a directory from the host machine you need to provide a fully qualified path.
As with ports the host path is on the left and the container path on the right.
Exercise
While in the container create a new file called
/data/container.txt
(you can create a file using thetouch
command)Exit the container and see what is now in the <host>/data directory
Warning
If you do not use fully qualified paths you are actually using the volume container option.
On the web page of nginx on docker hub you get a very nice description what it can do and how you can execute commands.
The ivonet/nginx-example
image is derived from that image.
One of the things you can do is mount a volume where nginx is actually watching for files.
# Linux / Mac
docker run --rm -d -p 3001:80 -v $(pwd)/data:/usr/share/nginx/html ivonet/nginx-example
# Windows
docker run --rm -d -p 3001:80 -v "%CD%\data":/usr/share/nginx/html ivonet/nginx-example
# Windows Powershell
docker run --rm -d -p 3001:80 -v "$((pwd).Path)\data:/usr/share/nginx/html" ivonet/nginx-example
Exercise
What happens if you goto http://localhost:3001?
and what happens if you create a file in the data directory called index.html with the following content and refresh the page?
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>hello</title>
</head>
<body>
<h1>Hello, NGINX!</h1>
</body>
</html>
Now you have nginx running in a docker container listening on its port 80 in the /usr/share/nginx/html
directory.
That directory is mounted to a host machine directory and you can edit files which are picked up immediately by the container.
This feature is very handy during development. You do not need to regenerate a docker image every time you make a change, but can live
see the changes you make and when the time comes that you are happy with the result you can build a real image with the data.
Very cool stuff 😄!
Think about it. If you use an Application Server like Payara to serve your thin Jakarta EE war, but you did not install Payara natively on your laptop, how do you deploy and test? By building a docker image from it and running it. But Payara also has an auto deploy folder, so what if you could just build your war and place it in a certain host folder that has been mounted to the autodeploy directory of Payara in the container? Then you suddenly have a much faster development cycle. Read more about it on this blog post (optional).
Data Volumes¶
The second way of saving data is in another special container, the so called Data Volume. Data volumes are container with no other function than storing data.
Docker will manage where the container is saved.
Note
Usage: docker volume COMMAND
Manage volumes
Commands:
create Create a volume
inspect Display detailed information on one or more volumes
ls List volumes
prune Remove all unused local volumes
rm Remove one or more volumes
Run 'docker volume COMMAND --help' for more information on a command.
You can start the same example as before, but now with a data-volume, as follows:
docker run --rm -d -p 3001:80 -v my-data:/usr/share/nginx/html ivonet/nginx-example
Just by leaving out the path on the left side of the -v
option you are creating a data volume inside a container at /my-data.
Note
Volumes are initialized when a container is created. If the container’s base image contains data at the specified mount point, that existing data is copied into the new volume upon volume initialization. (This does not apply when mounting a host directory).
Data volumes can be shared and reused among containers.
Changes to a data volume are made directly.
Changes to a data volume will not be included when you update an image.
Data volumes persist even if the container itself is deleted.
Exercise
do
docker run --rm -d -p 3001:80 -v my-data:/usr/share/nginx/html ivonet/nginx-example
Look at the volume just created
How would you list the contents of the data volume? (Brain teaser!)
Create your own data volume with the
docker volume ...
commandHow would you copy (
docker cp --help
) the index.html (former exercise) to this volume? (Brain teaser!)Check if you actually copied something to that volume by using another container.
Inspect containers¶
You can find any kind of information about a container by inspect
-ing it. In common usage you will not often
need to use this command, but it is good to know about it.
Below are some examples performed on the Virtual Machine:
- Inspect full example (show/hide)- Inspect example with filter (show/hide)
See the Docker Inspect reference for more information.
Images¶
Above you explored how to start and stop containers. Containers are instances of images.
Images can be viewed with the docker images
command.
As the images are the base of all the containers you create, you can accumulate quite a few over time.
Layers¶
Docker images are like onions. Not because they stink or make you cry but because they have layers. We will go into layers in the Creating Docker images section, but here is a short introduction:
For example if you compare the history of the nginx
image with the history of the ivonet/docker-from-scratch`
image
you will see something like:
$ docker history nginx:1.17.3-alpine
IMAGE CREATED CREATED BY SIZE COMMENT
41c8c3458a93 11 days ago /bin/sh -c #(nop) CMD ["nginx" "-g" "daemon… 0B
<missing> 11 days ago /bin/sh -c #(nop) STOPSIGNAL SIGTERM 0B
<missing> 11 days ago /bin/sh -c #(nop) EXPOSE 80 0B
<missing> 11 days ago /bin/sh -c set -x && addgroup -g 101 -S … 15.7MB
<missing> 11 days ago /bin/sh -c #(nop) ENV PKG_RELEASE=1 0B
<missing> 11 days ago /bin/sh -c #(nop) ENV NJS_VERSION=0.3.5 0B
<missing> 11 days ago /bin/sh -c #(nop) ENV NGINX_VERSION=1.17.3 0B
<missing> 11 days ago /bin/sh -c #(nop) LABEL maintainer=NGINX Do… 0B
<missing> 11 days ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
<missing> 11 days ago /bin/sh -c #(nop) ADD file:fe64057fbb83dccb9… 5.58MB
$ docker history ivonet/docker-from-scratch
IMAGE CREATED CREATED BY SIZE COMMENT
6a77a4bebb74 29 hours ago /bin/sh -c #(nop) COPY dir:e1baa3f8487c01720… 69.7MB
<missing> 11 days ago /bin/sh -c #(nop) CMD ["nginx" "-g" "daemon… 0B
<missing> 11 days ago /bin/sh -c #(nop) STOPSIGNAL SIGTERM 0B
<missing> 11 days ago /bin/sh -c #(nop) EXPOSE 80 0B
<missing> 11 days ago /bin/sh -c set -x && addgroup -g 101 -S … 15.7MB
<missing> 11 days ago /bin/sh -c #(nop) ENV PKG_RELEASE=1 0B
<missing> 11 days ago /bin/sh -c #(nop) ENV NJS_VERSION=0.3.5 0B
<missing> 11 days ago /bin/sh -c #(nop) ENV NGINX_VERSION=1.17.3 0B
<missing> 11 days ago /bin/sh -c #(nop) LABEL maintainer=NGINX Do… 0B
<missing> 11 days ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
<missing> 11 days ago /bin/sh -c #(nop) ADD file:fe64057fbb83dccb9… 5.58MB
You can see that the ivonet/docker-from-scratch is the same image as the nginx image,
but with an extra layer. You could say that the ivonet/docker-from-scratch
image
inherits FROM
the nginx
image.
Exercise
Take a look at histories of some of the other images on your machine.
At this point you should be:
More comfortable with the docker cli ;-)
Publish ports (
-p
).Use (data-)volumes (
-v
)Inspect containers (
docker inspect ...
)Understand the image layer system and see its
history
Next you will start creating your own docker images…