Compare commits
2 Commits
06193bafba
...
ad77de3670
Author | SHA1 | Date |
---|---|---|
Aleksey Zubakov | ad77de3670 | 2 years ago |
Aleksey Zubakov | 3ff9dee64a | 2 years ago |
@ -0,0 +1,2 @@ |
||||
a = {k: k * 379 for k in range(10)} |
||||
print(a) |
@ -0,0 +1,9 @@ |
||||
els = ["a", "b", "c"] |
||||
|
||||
|
||||
def _enum(els): |
||||
return zip(range(len(els)), els) |
||||
|
||||
|
||||
for idx, el in _enum(els): |
||||
print(idx, el) |
@ -0,0 +1,36 @@ |
||||
class IterableStack: |
||||
def __init__(self): |
||||
self._lst = [1, 2, 3, 4, 5] |
||||
|
||||
def __iter__(self): |
||||
return ReverseIterator(self._lst) |
||||
|
||||
|
||||
class ReverseIterator: |
||||
def __init__(self, _lst): |
||||
self._lst = _lst |
||||
self.position = len(_lst) |
||||
|
||||
def __next__(self): |
||||
self.position -= 1 |
||||
if self.position < 0: |
||||
raise StopIteration() |
||||
|
||||
return self._lst[self.position] |
||||
|
||||
|
||||
# container = IterableStack() |
||||
# for v in container: |
||||
# print(v) |
||||
|
||||
container = IterableStack() |
||||
it_container = iter(container) # container.__iter__() |
||||
while True: |
||||
try: |
||||
v = next(it_container) # it_container.__next__() |
||||
|
||||
# тело фора |
||||
print(v) |
||||
# конец тела фора |
||||
except StopIteration: |
||||
break |
@ -0,0 +1,9 @@ |
||||
def local_gen(): |
||||
n = 0 |
||||
|
||||
def next(): |
||||
nonlocal n |
||||
n += 1 |
||||
return n |
||||
|
||||
return next |
@ -0,0 +1,17 @@ |
||||
def _range(start, stop): |
||||
assert start < stop |
||||
|
||||
def inside_gen(): |
||||
nonlocal start |
||||
while start < stop: |
||||
print("Going to return: ", start) |
||||
yield start |
||||
start += 1 |
||||
|
||||
print("here") |
||||
|
||||
return inside_gen() |
||||
|
||||
|
||||
# for el in _range(0, 10): |
||||
# print(el) |
@ -0,0 +1,20 @@ |
||||
from typing import List |
||||
|
||||
|
||||
def load_files(paths: List[str]): |
||||
for path in paths: |
||||
with open(path) as f: |
||||
yield f.readlines() |
||||
|
||||
|
||||
paths = [ |
||||
"dict_gen.py", |
||||
"enum.py", |
||||
"for_loop.py", |
||||
"gen.py", |
||||
"lzy_load.py", |
||||
] |
||||
|
||||
for cont in load_files(paths): |
||||
# code working with content |
||||
print(cont) |
@ -0,0 +1,13 @@ |
||||
# Docker Tutorial |
||||
|
||||
|
||||
* [Prerequisites](prerequisites) |
||||
* [Useful external links](links) |
||||
* [Summary of commands](summary) |
||||
* [Lesson 1: Docker basics and running a container](lesson01) |
||||
* [Lesson 2: Introduction to Docker image builds](lesson02) |
||||
* [Lesson 3: Image layers](lesson03) |
||||
* [Lesson 4: Persisting data](lesson04) |
||||
* [Lesson 5: Network access](lesson05) |
||||
* [Lesson 6: Environment variables and configuration](lesson06) |
||||
* [Good Docker practices](practices) |
@ -0,0 +1,100 @@ |
||||
# Lesson 1: Docker basics and running a container |
||||
|
||||
1. Download the `busybox` Docker image from Docker Hub: |
||||
|
||||
$ docker images |
||||
$ docker pull busybox |
||||
$ docker images |
||||
|
||||
1. What do the columns mean? The first two are `REPOSITORY` and |
||||
`TAG`. Think of these as a way to name-space docker images. The |
||||
`REPOSITORY` is the name for a group of related repositories. For the case |
||||
of `busybox` the repository name is `busybox`. The second part of the |
||||
namespace is `TAG` and is separated from `REPOSITORY` with a `:` |
||||
(colon). If not explictly given, the tag defaults to `latest`. |
||||
|
||||
1. We will discuss tagging and the other columns later. |
||||
|
||||
1. Let's run busybox. |
||||
|
||||
$ docker run busybox /bin/sh -c "echo 'Hello' | md5sum" |
||||
09f7e02f1290be211da707a266f153b3 - |
||||
|
||||
1. What _is_ a docker container? |
||||
|
||||
> A container is a standard unit of software that packages up code and all |
||||
> its dependencies so the application runs quickly and reliably from one |
||||
> computing environment to another. (From https://www.docker.com) |
||||
|
||||
1. At heart a Docker container is a set of processes running in a |
||||
["namespace"](https://en.wikipedia.org/wiki/Linux_namespaces). These |
||||
namespaces isolate the processes from the other processes running on the |
||||
server. You can think of all this as a light-weight virtual machine. |
||||
|
||||
1. List the namespace of a running docker container (`lsns` is a Linux |
||||
command): |
||||
|
||||
$ docker run busybox /bin/sh -c "sleep 1000" & |
||||
root> lsns (must run as root to see the namespaces) |
||||
|
||||
1. Because Docker containers are just processes running on an existing |
||||
server inside of a namespace, Docker images use the server's kernel. Thus, |
||||
only functionality supported by the underlying kernel will work in a |
||||
Docker container. |
||||
|
||||
1. Docker containers also use ["control |
||||
groups"](https://en.wikipedia.org/wiki/Cgroups) which allow the host |
||||
operating system to put limits on the resources used by the running Docker |
||||
container. Limits can be placed on CPU, memory use, and I/O. |
||||
|
||||
# Limit docker to 10MB an use up all the memory |
||||
# (idea from https://unix.stackexchange.com/questions/99334/how-to-fill-90-of-the-free-memory) |
||||
$ docker run -m=10m busybox /bin/sh -c "cat /dev/zero | head -c 1m | tail" |
||||
$ docker run -m=10m busybox /bin/sh -c "cat /dev/zero | head -c 20m | tail" |
||||
|
||||
1. Unless you use an extra option the containers that you run will stick |
||||
around. To see this, use the `docker ps` command: |
||||
|
||||
$ docker ps --all |
||||
$ docker ps -a # (-a is the same as --all) |
||||
|
||||
1. Note that the names of the containers are random words. To give your |
||||
container a name, use the `--name` command: |
||||
|
||||
$ docker run --name=fuzzle busybox /bin/sh -c "echo 'Hello' | md5sum" |
||||
$ docker ps -a | grep fuzzle |
||||
|
||||
1. To remove one of these left over containers use `docker rm`: |
||||
|
||||
$ docker ps -a | grep fuzzle |
||||
$ docker rm fuzzle |
||||
$ docker ps -a | grep fuzzle |
||||
|
||||
1. To remove all stopped containers use `docker container prune`: |
||||
|
||||
$ docker ps -a |
||||
$ docker container prune |
||||
$ docker ps -a |
||||
|
||||
1. To avoid the whole stopped container messiness, tell Docker to remove |
||||
the container once it exits with teh `--rm` option: |
||||
|
||||
$ docker run --rm --name=fuzzle busybox /bin/sh -c "echo 'Hello' | md5sum" |
||||
$ docker ps -a | grep fuzzle |
||||
|
||||
1. You can "login" to a running docker container: |
||||
|
||||
$ docker run --rm --name=fuzzle busybox /bin/sh -c "sleep 10000" & |
||||
$ docker ps -a | grep fuzzle |
||||
$ docker exec -ti fuzzle /bin/sh |
||||
/ # # You are "inside" the running container; run some commands |
||||
/ # ps -eaf |
||||
/ # df -h |
||||
|
||||
1. The `-ti` options tell Docker that you want to allocate a pseudo-TTY |
||||
and use "interactive mode". *Warning:* logging into a running container is |
||||
not exactly like ssh'ing into a server: some commands that depend on the |
||||
terminal type may not work like you expect (e.g., editors, pagers, etc.) |
||||
|
||||
1. Being able to login to a running container is **very** useful when debugging |
||||
your Docker builds. |
@ -0,0 +1,7 @@ |
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
RUN chmod a+x /root/run.sh |
||||
CMD /root/run.sh |
@ -0,0 +1,97 @@ |
||||
# Lesson 2: Introduction to Docker image builds |
||||
|
||||
1. Clone this git repository. |
||||
|
||||
1. Change directory into `lesson02`: |
||||
|
||||
$ cd lesson02 |
||||
|
||||
1. Why are containers useful? What are their advantages over a |
||||
traditional server? |
||||
|
||||
- containers are light |
||||
- containers are portable |
||||
- containers are isolated |
||||
- containers can be run "immutably" |
||||
- containers are built hierarchically |
||||
- developers can create applications without a full server-stack |
||||
|
||||
1. What are some limitations of containers? |
||||
|
||||
- interaction is more difficult for multiple containers than for |
||||
multiple server process (although Kubernetes helps) |
||||
- some overhead so not quite as fast as "bare-metal" processes |
||||
- there are several decades worth of server administration best practices |
||||
and tools but only a few years for containers |
||||
- not good for large tightly-integrated applications (e.g., Oracle database) |
||||
|
||||
1. Question: what is the difference between an "image" and a "container"? |
||||
(See also [this Stackoverflow |
||||
question](https://stackoverflow.com/questions/23735149/what-is-the-difference-between-a-docker-image-and-a-container)). |
||||
|
||||
1. Most Docker images are build on top of existing "base" images. These |
||||
base containers are usually hosted in Docker Hub. For example, all Debian |
||||
releases come as Docker images; see https://hub.docker.com/_/debian for a |
||||
list of the base Debian Docker images. |
||||
|
||||
1. Let's build a "Hello, world." Docker image. We will build it on a |
||||
Debian buster base. First, pull the Docker image: |
||||
|
||||
$ docker pull debian:buster-slim |
||||
# We pull the "slim" image to save disk space |
||||
|
||||
1. Here is an application that echos "Hello, world." and then exits (this |
||||
file is also in the current directory). |
||||
|
||||
#!/bin/sh |
||||
echo "Hello, world." |
||||
exit 0 |
||||
|
||||
1. Now we create a "Dockerfile" that tells the build process how to create |
||||
the image. We use `debian:buster-slim` as the base and "add" the command |
||||
`run.sh`. The first argument of `ADD` is the *local* copy of the file and |
||||
the second argument is where we want the file to be in the image. |
||||
|
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
|
||||
|
||||
1. We want to make sure that the script will run, so make it executable. |
||||
|
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
RUN chmod a+x /root/run.sh |
||||
|
||||
1. Docker containers must be told which command to run. We do this with |
||||
the `CMD` directive |
||||
|
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
RUN chmod a+x /root/run.sh |
||||
CMD /root/run.sh |
||||
|
||||
1. We are now ready to build the image: |
||||
|
||||
$ docker build . -t hello-world |
||||
$ docker images | grep hello-world |
||||
|
||||
1. Question: what is the purpose of `.` (dot) in the above `docker build` |
||||
command? |
||||
|
||||
1. Note that the tag is `latest` (the default). |
||||
|
||||
1. Let's run this image in a container: |
||||
|
||||
$ docker run --rm hello-world |
||||
|
||||
1. Did you see what you expected? |
||||
|
@ -0,0 +1,3 @@ |
||||
#!/bin/sh |
||||
echo "Hello, world." |
||||
exit 0 |
@ -0,0 +1,7 @@ |
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
RUN chmod a+x /root/run.sh |
||||
CMD /root/run.sh |
@ -0,0 +1,7 @@ |
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
RUN chmod a+rx /root/run.sh |
||||
CMD /root/run.sh |
@ -0,0 +1,211 @@ |
||||
# Lesson 3: Image layers |
||||
|
||||
1. Change directory into `lesson03`. |
||||
|
||||
1. Build the `hello-world` image with the tag `v1`: |
||||
|
||||
$ docker build . -t hello-world:v1 |
||||
|
||||
1. Each Docker image consists of a sequence of file system _layers_ with the |
||||
later layers overwriting earlier ones. Each layer corresponds to an |
||||
instruction in the `Dockerfile`. Let's look at the layers. (Note: in this and |
||||
subsequent displays I leave off the CREATED column to save space). |
||||
|
||||
$ docker history hello-world:v1 |
||||
IMAGE CREATED BY |
||||
f38d648975d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo... |
||||
09b895731b10 /bin/sh -c chmod a+x /root/run.sh |
||||
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786â¦... |
||||
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s... |
||||
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"] |
||||
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869... |
||||
|
||||
$ docker history debian:buster-slim |
||||
IMAGE CREATED BY |
||||
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"] |
||||
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869... |
||||
|
||||
1. Notice that the `hello-world:v1` image's first two layers are the same |
||||
as `debian:buster-slim`'s layers. This reflects the fact that |
||||
`hello-world:v1` starts FROM the `debian:buster-slim` image. |
||||
|
||||
1. The subsequent layers of the `hello-world:v1` image each correspond to |
||||
the commands in the `Dockerfile`. Here is the Dockerfile again as a |
||||
reminder: |
||||
|
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
RUN chmod a+x /root/run.sh |
||||
CMD /root/run.sh |
||||
|
||||
1. When you create a running Docker container using `docker run` the |
||||
Docker system loads your image with each layer being _read-only_ with a |
||||
final, new layer being added. This last layer is writable. Let's try it. |
||||
|
||||
$ docker run --rm --name=fuzzle busybox /bin/sh -c "sleep 10000" & |
||||
$ docker exec -ti fuzzle /bin/sh (we "login" to the running container) |
||||
/ # cd /root; ls |
||||
~ # date > date.txt; ls -l |
||||
-rw-r--r-- 1 root root 0 Jan 16 03:56 date.txt |
||||
~ # cat /root/date.txt |
||||
Sat Jan 16 03:57:15 UTC 2021 |
||||
~ # exit |
||||
|
||||
1. The container is still running. Let's make sure the file we created is |
||||
still there. |
||||
|
||||
$ docker exec -ti fuzzle /bin/sh |
||||
~ # cat /root/date.txt |
||||
Sat Jan 16 03:57:15 UTC 2021 |
||||
~ # exit |
||||
|
||||
1. However, once the container exits, that final writable layer is thrown away. |
||||
**It does not persist**. Let's see. |
||||
|
||||
$ docker rm fuzzle |
||||
# You should get an error here. Why? |
||||
|
||||
1. To stop a running container from the outside use the `docker kill` command. |
||||
|
||||
$ docker kill fuzzle |
||||
$ docker rm fuzzle |
||||
# This last command returns an error. Why? |
||||
|
||||
1. To see that the file `/root/date.txt` is really gone, let's start the |
||||
container again and look. |
||||
|
||||
$ docker run --rm --name=fuzzle busybox /bin/sh -c "sleep 10000" & |
||||
$ docker exec fuzzle ls -l /root |
||||
|
||||
1. Notice that we did not login to the container to do an `ls`, rather, we used |
||||
the `exec` command. |
||||
|
||||
1. To reiterate, changes made to the containers top writable layer do not |
||||
persist. If you want the container to make persistent changes to files |
||||
another mechanism is needed such as mounting an external volume or writing |
||||
to some other persistent data store; more on that in a later lesson. |
||||
|
||||
1. Back to layers. |
||||
|
||||
1. When we looked at the history of the `hello-world:v1` image we saw this: |
||||
|
||||
$ docker history hello-world:v1 |
||||
IMAGE CREATED BY |
||||
f38d648975d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo... |
||||
09b895731b10 /bin/sh -c chmod a+x /root/run.sh |
||||
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786â¦... |
||||
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s... |
||||
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"] |
||||
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869... |
||||
|
||||
1. Under the `IMAGE` column are hex strings. Those correspond to the |
||||
SHA256 hash of the new layer's content. (More precisely, the SHA256 hash |
||||
of the layers configuration object.) |
||||
|
||||
1. Think of these hashes (roughly) corresponding to git commit hashes. The |
||||
layers and their hashes are useful for they tell Docker when a layer has |
||||
changed. We use a copy of `Dockerfile` with one small change: |
||||
|
||||
# Dockerfile-v2 |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD run.sh /root/run.sh |
||||
RUN chmod a+rx /root/run.sh # This is the changed line. |
||||
CMD /root/run.sh |
||||
|
||||
1. Build the image and look at its history: |
||||
|
||||
$ docker build . -f Dockerfile-v2 -t hello-world:v2 |
||||
$ docker history hello-world:v2 |
||||
IMAGE CREATED BY |
||||
0dd6cfe9bb51 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo... |
||||
39085b76a64f /bin/sh -c chmod a+rx /root/run.sh |
||||
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786... |
||||
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s... |
||||
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"] |
||||
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869... |
||||
|
||||
$ docker history hello-world:v1 |
||||
IMAGE CREATED BY |
||||
f38d648975d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo... |
||||
09b895731b10 /bin/sh -c chmod a+x /root/run.sh |
||||
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786â¦... |
||||
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s... |
||||
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"] |
||||
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869... |
||||
|
||||
1. Note that the layers have the same image ID except for the layers |
||||
starting with the one we changed. The top layers have different id's |
||||
since even though they are the same Dockerfile command they derive from |
||||
the layer that changed (again, think of a git commit). |
||||
|
||||
1. This has the pleasant result that if the first N Dockerfile commands do |
||||
not change but the N+1'st command _does_ change, then `docker build` will |
||||
use the cached layers for the first N layers and only rebuild from layer |
||||
N+1 onwards. This means rebuilds can be quite fast. |
||||
|
||||
1. **Important.** The image changes whenever one or more of the Dockerfile |
||||
`ADD`, `COPY`, or `RUN` command lines change. Even adding whitespace to a |
||||
line that has no real effect can trigger a new layer. |
||||
|
||||
1. What does it mean for a Dockerfile line to change? Either the line in |
||||
the Dockerfile itself changes (as we saw above) **or**, for commands that |
||||
add files from the local environment, the file being added changes. For |
||||
example, if we added a comment line to `run.sh` this counts as a change to |
||||
the `ADD run.sh /root/run.sh` line and would trigger a layer rebuild from |
||||
that layer forward. |
||||
|
||||
1. Edit the file `run.sh` by adding a comment and then rebuild: |
||||
|
||||
#!/bin/sh |
||||
# An extra comment |
||||
echo "Hello, world." |
||||
exit 0 |
||||
|
||||
$ docker build . -t hello-world:v3 |
||||
$ docker history hello-world:v1 |
||||
IMAGE CREATED BY |
||||
2131b85a91d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo... |
||||
30a489d1e5b0 /bin/sh -c chmod a+x /root/run.sh |
||||
94667dcd1388 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786... |
||||
ebca792f7949 /bin/sh -c #(nop) LABEL maintainer=adamhl@s... |
||||
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"] |
||||
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869... |
||||
$ docker history hello-world:v3 |
||||
IMAGE CREATED BY |
||||
acc54a5d1b49 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo... |
||||
82b9cde2c517 /bin/sh -c chmod a+x /root/run.sh |
||||
2a19083a38b0 /bin/sh -c #(nop) ADD file:997845d6d2fdc9492... |
||||
ebca792f7949 /bin/sh -c #(nop) LABEL maintainer=adamhl@s... |
||||
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"] |
||||
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869... |
||||
|
||||
1. Notice that the `ADD` line has changed its image id. |
||||
|
||||
1. A gotcha: your Dockerfile installs a package downloaded from an |
||||
external repository (like Debian). You build the image. A little while |
||||
later the package is updated in the Debian repository. You rebuild the |
||||
container thinking the rebuild will catch this change. But is does |
||||
NOT. The docker build only notices changes to the Dockerfile itself, not |
||||
to sources external to the build environment. |
||||
|
||||
1. A way around this gotcha. If you want to be sure that every time you |
||||
build your container it rebuilds each layer regardless of any changes to |
||||
the Dockerfile use the `--no-cache` option. This has the advantage of |
||||
reliability but the disadvantage that it can take significantly more time. |
||||
|
||||
$ docker build . -t hello-world:v3 (this will be fast) |
||||
$ docker build . --no-cache -t hello-world:v3 (this will slower) |
||||
|
||||
1. Another advantage of Docker caching layers is that different images |
||||
built off the same layer help save space. Recall that images layers are |
||||
loaded read-only, so if two different running containers are built off the |
||||
same layer, both containers can point to the same physical file. This is |
||||
why spawning a second container running on the same image as the first can |
||||
be so quick: the base layer is already loaded into memory so all that has |
||||
to be done is add the final writable layer that is distinct for each |
||||
container. |
@ -0,0 +1,4 @@ |
||||
#!/bin/sh |
||||
# Hello |
||||
echo "Hello, world." |
||||
exit 0 |
@ -0,0 +1,7 @@ |
||||
# Dockerfile (version 1) |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD date.sh /root/date.sh |
||||
RUN chmod a+x /root/date.sh |
||||
CMD /root/date.sh |
@ -0,0 +1,100 @@ |
||||
# Lesson 4: Persisting data |
||||
|
||||
1. Change directory into `lesson04`. |
||||
|
||||
1. We saw in Lesson 3 that data written to the final container layer does |
||||
not persist. How do we deal with this? One option is to have the container |
||||
write to an external system like a database system or cloud storage. There |
||||
is, however, another method: mounting a local file system or directory. |
||||
|
||||
1. Let's create a Docker container that saves the current date to a file. Here is |
||||
our first attempt: |
||||
|
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD date.sh /root/date.sh |
||||
RUN chmod a+x /root/date.sh |
||||
CMD /root/date.sh |
||||
|
||||
# date.sh |
||||
#!/bin/sh |
||||
echo "echoing date to /tmp/date.output" |
||||
date > /tmp/date.output |
||||
|
||||
$ docker build . -t date |
||||
$ docker run --rm --name=fuzzle date |
||||
echoing date to /tmp/date.output |
||||
|
||||
1. The above will create the file `/tmp/date.output` containing the date |
||||
but we know that the file will not persist after the container stops |
||||
running. To get around this we "mount" a local directory into the |
||||
container's `/tmp` directory. By "local directory" we mean a directory on |
||||
the computer where we run the our `docker` commands. We mount the colume |
||||
when we run the container. |
||||
|
||||
$ mkdir -p /tmp/docker |
||||
$ docker run --rm --name=fuzzle --volume=/tmp/docker:/tmp date |
||||
echoing date to /tmp/date.output |
||||
$ cat /tmp/docker/date.output |
||||
|
||||
1. You can run a container and override the `CMD` command. |
||||
|
||||
$ docker run --rm --name=fuzzle date ls -ld /etc |
||||
drwxr-xr-x 1 root root 4096 Jan 17 17:23 /etc/ |
||||
(The date.sh script did NOT run) |
||||
|
||||
1. This is especially useful when you want to login to the container and |
||||
debug the filesystem or CMD command. |
||||
|
||||
1. You can overwrite a file in the image using the same `--volume` |
||||
option. For example, let's overwrite `/etc/debian_version` with a |
||||
different file. |
||||
|
||||
$ docker run --rm --name=fuzzle date cat /etc/debian_version |
||||
10.7 |
||||
$ echo "fake-version" > /tmp/deb_ver |
||||
$ docker run --rm --name=fuzzle --volume=/tmp/deb_ver:/etc/debian_version date cat /etc/debian_version |
||||
fake-version |
||||
|
||||
1. If you mount an external directory onto a container directory |
||||
**everything** in the directory in the container is **replaced** with the |
||||
external directory. |
||||
|
||||
$ docker run --rm --name=fuzzle date ls -l /usr |
||||
total 32 |
||||
drwxr-xr-x 2 root root 4096 Jan 11 00:00 bin |
||||
drwxr-xr-x 2 root root 4096 Nov 22 12:37 games |
||||
... more ... |
||||
|
||||
# Create a directory in /tmp with a single file. |
||||
$ mkdir -p /tmp/usr; cp test.txt /tmp/usr |
||||
|
||||
# Mount /tmp/usr over /usr in the container |
||||
$ docker run --rm --name=fuzzle --volume=/tmp/usr:/usr date ls -l /usr |
||||
total 4 |
||||
-rw-r--r-- 1 52777 root 7 Jan 17 18:04 test.txt |
||||
|
||||
1. You can mount an external directory in read-only mode. This is |
||||
particularly useful when injecting secrets or configuration information |
||||
into a container. Look for the `ro` in the `--volume` option below. |
||||
|
||||
$ mkdir -p /tmp/secrets |
||||
$ echo "my-password" > /tmp/secrets/password |
||||
|
||||
$ docker run --rm --name=fuzzle --volume=/tmp/secrets:/secrets:ro date sleep 100000 & |
||||
|
||||
$ docker exec -ti fuzzle /bin/sh |
||||
# cat /secrets/password |
||||
my-password |
||||
# echo "another secret" >> /secrets/password |
||||
/bin/sh: 5: cannot create /secrets/password: Read-only file system |
||||
|
||||
$ docker kill fuzzle |
||||
|
||||
1. You can run the entire container in read-only mode. The `docker run` |
||||
option `--read-only` mounts the root filesystem (i.e., everything) in |
||||
read-only excepting any externally mounted volumes. This lets you |
||||
lock-down the filesystem except for those parts of the filesystem you |
||||
know need to be written to (e.g., `/tmp`, `/var/log`, etc.). |
@ -0,0 +1,3 @@ |
||||
#!/bin/sh |
||||
echo "echoing date to /tmp/date.output" |
||||
date > /tmp/date.output |
@ -0,0 +1 @@ |
||||
Hello. |
@ -0,0 +1,36 @@ |
||||
# Lesson 5: Network access |
||||
|
||||
1. Change directory into `lesson05`. |
||||
|
||||
1. A very popular use for containers is for serving web applications. How |
||||
do users get network access to a running container? Put simply, the host |
||||
acts as the network proxy for its containers: a network conection is made |
||||
to the host which directs the traffic into the container, and vice versa |
||||
for outbound traffic. |
||||
|
||||
1. Let's try it: |
||||
|
||||
$ docker pull httpd |
||||
$ docker run -dit --name apache --rm -p 8080:80 httpd |
||||
$ docker ps |
||||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES |
||||
23c146b78377 httpd "httpd-foreground" 20 seconds ago Up 19 seconds 0.0.0.0:8080->80/tcp apache |
||||
|
||||
1. Note that traffic sent to port 8080 on the *host* gets sent to port 80 |
||||
in the *container*. The `-d` option tells Docker to run the container in |
||||
"detached" mode, i.e., in the background. |
||||
|
||||
1. Look at the network (look for the `Containers` element): |
||||
|
||||
$ docker network inspect bridge |
||||
|
||||
1. Test the application. If running docker on your local machine put this |
||||
URL into your browser's address bar: `http://localhost:8080`. |
||||
|
||||
1. If running on a remote host run this command: |
||||
|
||||
$ wget http://localhost:8080 --quiet -O - |
||||
|
||||
1. Don't forget to kill the container: |
||||
|
||||
$ docker kill apache |
@ -0,0 +1,7 @@ |
||||
# Dockerfile |
||||
FROM debian:buster-slim |
||||
LABEL maintainer="adamhl@stanford.edu" |
||||
|
||||
ADD demo.sh /root/demo.sh |
||||
RUN chmod a+x /root/demo.sh |
||||
CMD /root/demo.sh |
@ -0,0 +1,35 @@ |
||||
# Lesson 6: Environment variables and configuration |
||||
|
||||
1. Change directory into `lesson06`. |
||||
|
||||
1. A popular way to provide configuration information to a docker container is |
||||
via environment variables. |
||||
|
||||
1. Let's try it: |
||||
|
||||
$ docker build . -t demo |
||||
$ docker run -it --rm --name fuzzle demo |
||||
|
||||
1. Does the output make sense? |
||||
|
||||
1. Let's pass in a value for the environment variable `HELLO_WORLD`: |
||||
|
||||
$ docker run -it --rm --name fuzzle -e HELLO_WORLD='Hello, world.' demo |
||||
|
||||
1. You can also pass in environment variables via a file: |
||||
|
||||
$ cat file.env |
||||
# We define two environment variables (you CAN comment!) |
||||
HELLO_WORLD="Hello, world." |
||||
ENV_VAR2="another environment variable" |
||||
|
||||
$ docker run -it --rm --name fuzzle --env-file=file.env demo |
||||
|
||||
1. If your Docker application does not have many configuration options |
||||
configuring via environment variables is a good method. However, if the |
||||
application has complicated or extnesive configuration this may not be |
||||
feasible. |
||||
|
||||
1. Don't forget to clean up: |
||||
|
||||
$ docker rmi demo:latest (removes the image demo:latest) |
@ -0,0 +1,9 @@ |
||||
#!/bin/bash |
||||
# demo.sh |
||||
|
||||
if [ -z "${HELLO_WORLD}" ]; then |
||||
echo "The environment variable HELLO_WORLD is not defined." |
||||
else |
||||
echo "The environment variable HELLO_WORLD is '${HELLO_WORLD}'." |
||||
fi |
||||
|
@ -0,0 +1,3 @@ |
||||
# We define two environment variables (you CAN comment!) |
||||
HELLO_WORLD="Hello, world." |
||||
ENV_VAR2="another environment variable" |
@ -0,0 +1,6 @@ |
||||
# Useful External Links |
||||
|
||||
* [Container and |
||||
layers](https://docs.docker.com/storage/storagedriver/#container-and-layers): |
||||
a good explanation from Docker on image and container layers |
||||
|
@ -0,0 +1,49 @@ |
||||
[[_TOC_]] |
||||
|
||||
# Good Docker Practices |
||||
|
||||
## Keep the Docker image simple (micro-services) |
||||
|
||||
Although you can run as many processes in a single container as you want, |
||||
it is usually a good idea to design a container to do a single task. If |
||||
your application does several different things you can always add |
||||
"sidecar" containers that do the extra work. |
||||
|
||||
There will be situations where splitting an application into different |
||||
containers is too complicated. Be flexible and use your own judgement. |
||||
|
||||
## Use small base images |
||||
|
||||
A smaller image means faster start-up times and less memory used on the |
||||
container host. One way to acheive is to use a small base image. A popular |
||||
small image is `alpine` based on Alpine Linux. This is a complete Linux |
||||
with image size of 5.5MB with its own packages. By comparison, |
||||
`debian:buster-slim` is about 70MB. |
||||
|
||||
On the other hand, don't let the drive toward small size get in the way |
||||
of needed functionality; remember the IBM Pollyanna Principle: "machines |
||||
should work; people should think". |
||||
|
||||
## When possible use container orchestration |
||||
|
||||
Getting containers to interact and cooperate can be tricky, so use one of |
||||
the orcestration tools like Kubernetes or Docker Compose to do this. |
||||
|
||||
## Use CI/CD (i.e., automation) to keep Docker images up-to-date |
||||
|
||||
Set up automation to rebuild your Docker images periodically making sure |
||||
that you disable caching when building. This way your image will have the |
||||
most up-to-date and secure base images. |
||||
|
||||
## Send diagnostic output to standard output |
||||
|
||||
In the traditional server world we are used to sending logs to files. With |
||||
Docker containers it is usually better to send diagnostic output to |
||||
standard output. Kubernetes and other orchestration tools are designed |
||||
with the expectation that logging is sent to standard output. |
||||
|
||||
## Run containers in "read-only" mode |
||||
|
||||
Running a Docker container in read-only mode helps to reduce the attack |
||||
surface area of your application. Mount external volumes for those parts |
||||
of the file system that need to be writable (`/var/log`, `/tmp`, etc.). |
@ -0,0 +1,15 @@ |
||||
# Prequisites |
||||
|
||||
1. Installation of Docker. |
||||
|
||||
1. Test your Docker install (the hex ID shown below will differ): |
||||
|
||||
$ docker images |
||||
# REPOSITORY TAG IMAGE ID CREATED SIZE |
||||
... more ... |
||||
|
||||
$ docker pull busybox |
||||
Using default tag: latest |
||||
latest: Pulling from library/busybox |
||||
e5d9363303dd: Pull complete |
||||
|
@ -0,0 +1,23 @@ |
||||
# Summary of Docker commands used in this tutorial |
||||
|
||||
# List images |
||||
docker images |
||||
|
||||
# Pull an image from Docker Hub |
||||
docker pull busybox |
||||
|
||||
# Build a Docker image |
||||
docker build -t my-tag . |
||||
|
||||
# Build image ignoring cached layers |
||||
docker build --no-cache -t my-tag . |
||||
|
||||
# Run container in "interactive" mode |
||||
docker run -it <image-name> |
||||
|
||||
# List containers |
||||
docker ps |
||||
docker ps -a (include stopped containers) |
||||
|
||||
# Connect to a running container (i.e., "login") |
||||
docker exec -it <container-name> -- /bin/bash |
Loading…
Reference in new issue