Compare commits

...

14 Commits
main ... main

Author SHA1 Message Date
Aleksey Zubakov ad77de3670 Add docker practice 2 years ago
Aleksey Zubakov 3ff9dee64a Add code from lecture about iterators 2 years ago
Aleksey Zubakov 06193bafba Move mro example to proper place 2 years ago
Aleksey Zubakov c659c423a5 Reorder examples 2 years ago
Aleksey Zubakov 32f4d0c92c Add code for eight seminar 2 years ago
Aleksey Zubakov 910249657a Add problem for seventh seminar 2 years ago
Aleksey Zubakov 6e72f80aa6 Add python decorators seminar 2 years ago
Aleksey Zubakov e8c503d7b6 Rename haskell seminar 2 years ago
Aleksey Zubakov 8382832f42 New code 2 years ago
Aleksey Zubakov 3e886a6c0c Update seventh seminar 2 years ago
Aleksey Zubakov eb10ba2ea8 Seventh seminar for god of haskell 2 years ago
Aleksey Zubakov d673967952 Add drone pipeline 2 years ago
Aleksey Zubakov ac1940db56 Update README.md 2 years ago
Aleksey Zubakov 94312accb5 Sixth seminar 2 years ago
  1. 9
      .drone.yml
  2. 27
      06/README.md
  3. 37
      07_haskell/Main.hs
  4. 35
      07_haskell/Second.hs
  5. 21
      07_python/README.md
  6. 1
      07_python/bench.py
  7. 9
      07_python/closure.py
  8. 14
      07_python/debugger.py
  9. 17
      07_python/deco_troubles.py
  10. 8
      07_python/ex.py
  11. 77
      07_python/name_test.py
  12. 8
      07_python/partial.py
  13. 33
      07_python/retry.py
  14. 11
      08_python/README.md
  15. 15
      08_python/class-deco/getter.py
  16. 28
      08_python/dataclasses/dcls.py
  17. 17
      08_python/definition/cl_defo.py
  18. 7
      08_python/definition/dct.py
  19. 12
      08_python/definition/def_cls.py
  20. 14
      08_python/definition/inheritance.py
  21. 17
      08_python/descriptor/modifiers.py
  22. 8
      08_python/magic/MyHashable.py
  23. 3
      08_python/magic/always_field.py
  24. 23
      08_python/magic/ex_for.py
  25. 11
      08_python/magic/ex_for2.py
  26. 20
      08_python/magic/hasheq.py
  27. 11
      08_python/magic/new_getattr.py
  28. 6
      08_python/magic/nmp.py
  29. 14
      08_python/mixins/cat_girl.py
  30. 34
      08_python/mro/broken_mro.py
  31. 21
      08_python/mro/completely_f_cked.py
  32. 24
      08_python/mro/mro.py
  33. 2
      09_python/dict_gen.py
  34. 9
      09_python/enum.py
  35. 36
      09_python/for_loop.py
  36. 9
      09_python/func_stream.py
  37. 17
      09_python/gen.py
  38. 20
      09_python/lzy_load.py
  39. 13
      10_docker/README.md
  40. 100
      10_docker/lesson01/README.md
  41. 7
      10_docker/lesson02/Dockerfile
  42. 97
      10_docker/lesson02/README.md
  43. 3
      10_docker/lesson02/run.sh
  44. 7
      10_docker/lesson03/Dockerfile
  45. 7
      10_docker/lesson03/Dockerfile-v2
  46. 211
      10_docker/lesson03/README.md
  47. 4
      10_docker/lesson03/run.sh
  48. 7
      10_docker/lesson04/Dockerfile
  49. 100
      10_docker/lesson04/README.md
  50. 3
      10_docker/lesson04/date.sh
  51. 1
      10_docker/lesson04/test.txt
  52. 36
      10_docker/lesson05/README.md
  53. 7
      10_docker/lesson06/Dockerfile
  54. 35
      10_docker/lesson06/README.md
  55. 9
      10_docker/lesson06/demo.sh
  56. 3
      10_docker/lesson06/file.env
  57. 6
      10_docker/links/README.md
  58. 49
      10_docker/practices/README.md
  59. 15
      10_docker/prerequisites/README.md
  60. 23
      10_docker/summary/README.md
  61. 43
      README.md

@ -0,0 +1,9 @@
kind: pipeline
type: docker
name: default
steps:
- name: test
image: nixos/nix:master
commands:
- echo 522

@ -0,0 +1,27 @@
### Что здесь лежит?
Только README.md, больше ничего
### Задания на практику
#### 1. Удваиватель
Написать команду, копирующую содержимое файла `data` в конец этого же файла.
Примечания:
* решение на полный балл не должно использовать временные файлы;
* файл может содержать бинарные данные;
* проверьте, что если изначальный размер файла был n байт, то размер результирующего файла -- 2 * n байт
* проверьте свое решение на больших (> 1 MB) файлах;
#### 2. Числа Фибоначчи
Определим числа Фибоначчи следующим образом:
![fibonacci](https://wikimedia.org/api/rest_v1/media/math/render/svg/caa1e3b2e9c19fe638fc0b3d6548718bfe5119e2)
Вам нужно написать скрипт, который будет считывать из `stdin`
число `n` и выводить в `stdout` `n`-ое число Фибоначчи.
Примечания:
* Время работы функции -- `O(n)`.

@ -0,0 +1,37 @@
data A = True' | False'
deriving Show
foo :: A -> Integer
foo True' = 1
foo False' = 0
data P3 = P Bool A -- P (Bool x A)
data BigP = P1 Bool A | P2 Integer
deriving Show
bar :: BigP -> Integer
bar (P1 _ _) = 1
bar (P2 _) = 2
int_plus_3 :: Integer -> Integer
int_plus_3 n = n + 3
-- examples
comp :: BigP -> Integer
comp = int_plus_3 . bar
-- same as: int_plus_3 $ bar $ P2 10
double_arg :: Integer -> Integer -> Integer
double_arg a b = a + b
partially_applied :: Integer -> Integer
partially_applied = double_arg 7
func_arg :: (Integer -> Integer) -> Integer -> Integer
func_arg f b = f b
main :: IO ()
main = putStrLn "Hello world"

@ -0,0 +1,35 @@
type Name = String
type Table = [ (Name, Name) ]
fathers :: Table
fathers = [
("a", "d"),
("b", "r")
]
head' :: [a] -> a
head' (x:xs) = x
helper :: Integer -> [a] -> Integer
helper acc (x:xs) = helper (acc + 1) xs
helper acc ([]) = acc
len' :: [a] -> Integer
len' xs = helper 0 xs
inv :: [a] -> [a] -> [a]
inv acc (x:xs) = inv (x : acc) xs
inv acc [] = acc
rev :: [a] -> [a]
rev xs = inv [] xs
-- 1 : 2 : 3 : []
-- 3 : 2 : 1 : []
getF :: Name -> Maybe Name
getF n = lookup n fathers

@ -0,0 +1,21 @@
### Задаание 1
Используя библиотеку `time`, написать декоратор `@bench(n)`,
который меняет функцию так, чтобы при каждом её вызове она
вычислялась не один раз, а `n` раз, при этом необходимо выводить:
- имя функции;
- аргументы;
- средняя время работы за `n` запусков;
```python
@bench(50)
def foo(a: int, b: int):
...
>>> foo(5, 5)
<... foo> (5, 5) {}
Mean execution time on <N> calls: ???ns
```

@ -0,0 +1,9 @@
def adder():
n = 0
def add():
nonlocal n
n += 1
return n
return add

@ -0,0 +1,14 @@
def debug_call(f):
def inner(*args, **kwargs):
print(f, args, kwargs)
return f(*args, **kwargs)
return inner
# @debug_call
def foo(a, b):
return a + b
foo = debug_call(foo)

@ -0,0 +1,17 @@
def deco(f):
def inner(*args, **kwargs):
print(f)
return f(*args, **kwargs)
return inner
# @deco
def foo():
"""
Foo is just a function.
"""
return None
foo = deco(foo)

@ -0,0 +1,8 @@
def deco_ch(f):
f.jjjjj = 10
return f
@deco_ch
def foo():
print(foo.jjjjj)

@ -0,0 +1,77 @@
# first attempt
def deco_first(f):
def inner(*args, **kwargs):
return f(*args, **kwargs)
inner.__name__ = f.__name__
inner.__doc__ = f.__doc__
inner.__qualname__ = f.__qualname__
return inner
# second attempt
from functools import update_wrapper
def deco_second(f):
def inner(*args, **kwargs):
return f(*args, **kwargs)
update_wrapper(inner, f)
return inner
# third attempt
def my_wraps(original):
def deco(wrapper):
update_wrapper(wrapper, original)
return wrapper
return deco
from functools import partial
def partial_wraps(original):
return partial(update_wrapper, wrapped=original)
# def deco_third(f):
# def inner(*args, **kwargs):
# return f(*args, **kwargs)
#
# deco = my_wraps(f)
# inner = deco(inner)
# return inner
def deco_third(f):
@partial_wraps(f)
def inner(*args, **kwargs):
return f(*args, **kwargs)
return inner
# third attempt
from functools import wraps
def deco_fourth(f):
@wraps(f)
def inner(*args, **kwargs):
return f(*args, **kwargs)
return inner
@deco_third
def foo(a, b, c):
"""
Foo function docstring
"""
return a + b + c

@ -0,0 +1,8 @@
from functools import partial
def foo(a, b, new_name):
return new_name * (b + a)
bar = partial(foo, c=500)

@ -0,0 +1,33 @@
def adder():
n = 0
def add():
nonlocal n
n += 1
return n
return add
def retry(n: int = 5):
def deco(f):
def inner(*args, **kwargs):
for _ in range(n):
f(*args, **kwargs)
return inner
return deco
# @retry(5)
def foo():
print("Hello world!")
foo = (retry(5))(foo)
@retry(15)
def bar():
print("Bar")

@ -0,0 +1,11 @@
### Практика про классы в питоне
Что здесь есть полезного?
- [definition](./definition) -- определения классов;
- [descriptor](./descriptor) -- про модификаторы и как использовать `@property`;
- [magic](./magic) -- куча примеров магических методов;
- [mro](./mro) -- method resolution order или примеры про плату за сомнительные решения;
- [class-deco](./class-deco) -- как использовать класс в качестве декоратора;
- [dataclasses](./dataclasses) -- датаклассы (классы без методов, но только с
данными);
- [mixins](./mixins) -- пример паттерна mixin;

@ -0,0 +1,15 @@
class ClassDeco:
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
print(self.f, args, kwargs)
return self.f(*args, **kwargs)
@ClassDeco
def foo():
return 45
# foo = ClassDeco(foo)

@ -0,0 +1,28 @@
from dataclasses import dataclass
from typing import Union
# def dataclass(cls=None, /, *, init=True, repr=True, eq=True, order=False,
# unsafe_hash=False, frozen=False):
# """Returns the same class as was passed in, with dunder methods
# added based on the fields defined in the class.
#
# Examines PEP 526 __annotations__ to determine fields.
#
# If init is true, an __init__() method is added to the class. If
# repr is true, a __repr__() method is added. If order is true, rich
# comparison dunder methods are added. If unsafe_hash is true, a
# __hash__() method function is added. If frozen is true, fields may
# not be assigned to after instance creation.
# """
@dataclass(frozen=True)
class A:
a: int
b: bool
c: str = "asd"
f: Union[int, bool, str] = 0
def foo(self):
print(self)

@ -0,0 +1,17 @@
class A:
pass
class C:
a, b = 1, 1
for i in range(15):
a, b = b, a + b
print(a)
f = lambda self, a: a
g = lambda self, a: self.f(a)
def __init__(self, x, y):
print(self)
self.x = x
self.y = y

@ -0,0 +1,7 @@
class A:
n = 5
e = 50
f = lambda self: 42
def __init__(self):
self.a = 1000

@ -0,0 +1,12 @@
class C:
f = 50
a, b = 1, 1
for _ in range(10):
a, b = b, a + b
print(a, b)
def __init__(self, a, b):
print(self)
self.a = a
self.b = b

@ -0,0 +1,14 @@
# class Base(object)
class Base:
def foo(self):
return 42
class NotBase:
def foo(self):
return 43
class A(NotBase, Base):
def bar(self):
return 41

@ -0,0 +1,17 @@
class A:
def __init__(self, a, b):
self._a = a
self.__b = b
@property
def super_duper_var(self):
print(self)
return self.__b
@super_duper_var.setter
def super_duper_var(self, value):
self.__b = value
# @super_duper_var.deleter
# def super_duper_var(self):
# pass

@ -0,0 +1,8 @@
from abc import ABC
class MyHashable(ABC):
@classmethod
def __subclasshook__(cls, sbcls):
hash_func = getattr(sbcls, "__hash__", None)
return hash_func is not None

@ -0,0 +1,3 @@
class AlwaysHaveField:
def __getattr__(self, value):
return value

@ -0,0 +1,23 @@
class Iterator:
def __init__(self, origin_list):
self._ref = origin_list
self.n = -1
def __next__(self):
if self.n < len(self._ref) - 1:
self.n += 1
return self._ref[self.n]
raise StopIteration()
class A:
def __init__(self):
self.a = [1, 2, 3, 4]
def __iter__(self):
return Iterator(self.a)
a = A()
for el in a:
print(el)

@ -0,0 +1,11 @@
class A:
def __init__(self):
self.a = [1, 2, 3, 4]
def __getitem__(self, ind):
return self.a[ind]
a = A()
for el in a:
print(el)

@ -0,0 +1,20 @@
class MADATA:
def __init__(
self,
a: int,
b: int,
c: int,
):
self.a = a
self.b = b
self.c = c
@property
def _tup_view(self):
return self.a, self.b, self.c
def __hash__(self):
return hash(self._tup_view)
def __eq__(self, o):
return isinstance(o, type(self)) and o._tup_view == self._tup_view

@ -0,0 +1,11 @@
class A:
def __init__(self):
self.a = 1
self.b = 2
def foo(self):
print("Hello")
def __getattr__(self, name):
print(f"Called __getattr__ with arg: {name}")
return name

@ -0,0 +1,6 @@
class DF:
def __getitem__(self, key):
return key
def __setitem__(self, key, value):
print(key, value)

@ -0,0 +1,14 @@
class GirlsMixin:
def hooray(self):
print("Hoooray")
class CatMixin:
def feed(self, hp):
print("Meow")
class CatGirls(GirlsMixin, CatMixin):
def main(self):
super().feed(50)
super().hooray()

@ -0,0 +1,34 @@
class Base:
def foo(self):
print("Base")
class A(Base):
def foo(self):
print("A")
super().foo()
class B(Base):
def foo(self):
print("B")
super().foo()
class C(A, B):
def foo(self):
print("C")
super().foo()
class D(B, A):
def foo(self):
print("D")
super().foo()
class E(C, D):
pass
print("HEELLO")

@ -0,0 +1,21 @@
class Base:
def __init__(self, a):
self.x = a
class A(Base):
def __init__(self):
super().__init__(5)
class B(Base):
def __init__(self, x):
super().__init__(x)
class C(A, B):
...
class D(B, A):
...

@ -0,0 +1,24 @@
class Base:
def foo(self):
print("Base")
class A(Base):
def foo(self):
print("A")
super().foo()
class B(Base):
def foo(self):
print("B")
super().foo()
class C(A, B):
def foo(self):
print("C")
super().foo()
print("HEELLO")

@ -0,0 +1,2 @@
a = {k: k * 379 for k in range(10)}
print(a)

@ -0,0 +1,9 @@
els = ["a", "b", "c"]
def _enum(els):
return zip(range(len(els)), els)
for idx, el in _enum(els):
print(idx, el)

@ -0,0 +1,36 @@
class IterableStack:
def __init__(self):
self._lst = [1, 2, 3, 4, 5]
def __iter__(self):
return ReverseIterator(self._lst)
class ReverseIterator:
def __init__(self, _lst):
self._lst = _lst
self.position = len(_lst)
def __next__(self):
self.position -= 1
if self.position < 0:
raise StopIteration()
return self._lst[self.position]
# container = IterableStack()
# for v in container:
# print(v)
container = IterableStack()
it_container = iter(container) # container.__iter__()
while True:
try:
v = next(it_container) # it_container.__next__()
# тело фора
print(v)
# конец тела фора
except StopIteration:
break

@ -0,0 +1,9 @@
def local_gen():
n = 0
def next():
nonlocal n
n += 1
return n
return next

@ -0,0 +1,17 @@
def _range(start, stop):
assert start < stop
def inside_gen():
nonlocal start
while start < stop:
print("Going to return: ", start)
yield start
start += 1
print("here")
return inside_gen()
# for el in _range(0, 10):
# print(el)

@ -0,0 +1,20 @@
from typing import List
def load_files(paths: List[str]):
for path in paths:
with open(path) as f:
yield f.readlines()
paths = [
"dict_gen.py",
"enum.py",
"for_loop.py",
"gen.py",
"lzy_load.py",
]
for cont in load_files(paths):
# code working with content
print(cont)

@ -0,0 +1,13 @@
# Docker Tutorial
* [Prerequisites](prerequisites)
* [Useful external links](links)
* [Summary of commands](summary)
* [Lesson 1: Docker basics and running a container](lesson01)
* [Lesson 2: Introduction to Docker image builds](lesson02)
* [Lesson 3: Image layers](lesson03)
* [Lesson 4: Persisting data](lesson04)
* [Lesson 5: Network access](lesson05)
* [Lesson 6: Environment variables and configuration](lesson06)
* [Good Docker practices](practices)

@ -0,0 +1,100 @@
# Lesson 1: Docker basics and running a container
1. Download the `busybox` Docker image from Docker Hub:
$ docker images
$ docker pull busybox
$ docker images
1. What do the columns mean? The first two are `REPOSITORY` and
`TAG`. Think of these as a way to name-space docker images. The
`REPOSITORY` is the name for a group of related repositories. For the case
of `busybox` the repository name is `busybox`. The second part of the
namespace is `TAG` and is separated from `REPOSITORY` with a `:`
(colon). If not explictly given, the tag defaults to `latest`.
1. We will discuss tagging and the other columns later.
1. Let's run busybox.
$ docker run busybox /bin/sh -c "echo 'Hello' | md5sum"
09f7e02f1290be211da707a266f153b3 -
1. What _is_ a docker container?
> A container is a standard unit of software that packages up code and all
> its dependencies so the application runs quickly and reliably from one
> computing environment to another. (From https://www.docker.com)
1. At heart a Docker container is a set of processes running in a
["namespace"](https://en.wikipedia.org/wiki/Linux_namespaces). These
namespaces isolate the processes from the other processes running on the
server. You can think of all this as a light-weight virtual machine.
1. List the namespace of a running docker container (`lsns` is a Linux
command):
$ docker run busybox /bin/sh -c "sleep 1000" &
root> lsns (must run as root to see the namespaces)
1. Because Docker containers are just processes running on an existing
server inside of a namespace, Docker images use the server's kernel. Thus,
only functionality supported by the underlying kernel will work in a
Docker container.
1. Docker containers also use ["control
groups"](https://en.wikipedia.org/wiki/Cgroups) which allow the host
operating system to put limits on the resources used by the running Docker
container. Limits can be placed on CPU, memory use, and I/O.
# Limit docker to 10MB an use up all the memory
# (idea from https://unix.stackexchange.com/questions/99334/how-to-fill-90-of-the-free-memory)
$ docker run -m=10m busybox /bin/sh -c "cat /dev/zero | head -c 1m | tail"
$ docker run -m=10m busybox /bin/sh -c "cat /dev/zero | head -c 20m | tail"
1. Unless you use an extra option the containers that you run will stick
around. To see this, use the `docker ps` command:
$ docker ps --all
$ docker ps -a # (-a is the same as --all)
1. Note that the names of the containers are random words. To give your
container a name, use the `--name` command:
$ docker run --name=fuzzle busybox /bin/sh -c "echo 'Hello' | md5sum"
$ docker ps -a | grep fuzzle
1. To remove one of these left over containers use `docker rm`:
$ docker ps -a | grep fuzzle
$ docker rm fuzzle
$ docker ps -a | grep fuzzle
1. To remove all stopped containers use `docker container prune`:
$ docker ps -a
$ docker container prune
$ docker ps -a
1. To avoid the whole stopped container messiness, tell Docker to remove
the container once it exits with teh `--rm` option:
$ docker run --rm --name=fuzzle busybox /bin/sh -c "echo 'Hello' | md5sum"
$ docker ps -a | grep fuzzle
1. You can "login" to a running docker container:
$ docker run --rm --name=fuzzle busybox /bin/sh -c "sleep 10000" &
$ docker ps -a | grep fuzzle
$ docker exec -ti fuzzle /bin/sh
/ # # You are "inside" the running container; run some commands
/ # ps -eaf
/ # df -h
1. The `-ti` options tell Docker that you want to allocate a pseudo-TTY
and use "interactive mode". *Warning:* logging into a running container is
not exactly like ssh'ing into a server: some commands that depend on the
terminal type may not work like you expect (e.g., editors, pagers, etc.)
1. Being able to login to a running container is **very** useful when debugging
your Docker builds.

@ -0,0 +1,7 @@
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
RUN chmod a+x /root/run.sh
CMD /root/run.sh

@ -0,0 +1,97 @@
# Lesson 2: Introduction to Docker image builds
1. Clone this git repository.
1. Change directory into `lesson02`:
$ cd lesson02
1. Why are containers useful? What are their advantages over a
traditional server?
- containers are light
- containers are portable
- containers are isolated
- containers can be run "immutably"
- containers are built hierarchically
- developers can create applications without a full server-stack
1. What are some limitations of containers?
- interaction is more difficult for multiple containers than for
multiple server process (although Kubernetes helps)
- some overhead so not quite as fast as "bare-metal" processes
- there are several decades worth of server administration best practices
and tools but only a few years for containers
- not good for large tightly-integrated applications (e.g., Oracle database)
1. Question: what is the difference between an "image" and a "container"?
(See also [this Stackoverflow
question](https://stackoverflow.com/questions/23735149/what-is-the-difference-between-a-docker-image-and-a-container)).
1. Most Docker images are build on top of existing "base" images. These
base containers are usually hosted in Docker Hub. For example, all Debian
releases come as Docker images; see https://hub.docker.com/_/debian for a
list of the base Debian Docker images.
1. Let's build a "Hello, world." Docker image. We will build it on a
Debian buster base. First, pull the Docker image:
$ docker pull debian:buster-slim
# We pull the "slim" image to save disk space
1. Here is an application that echos "Hello, world." and then exits (this
file is also in the current directory).
#!/bin/sh
echo "Hello, world."
exit 0
1. Now we create a "Dockerfile" that tells the build process how to create
the image. We use `debian:buster-slim` as the base and "add" the command
`run.sh`. The first argument of `ADD` is the *local* copy of the file and
the second argument is where we want the file to be in the image.
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
1. We want to make sure that the script will run, so make it executable.
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
RUN chmod a+x /root/run.sh
1. Docker containers must be told which command to run. We do this with
the `CMD` directive
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
RUN chmod a+x /root/run.sh
CMD /root/run.sh
1. We are now ready to build the image:
$ docker build . -t hello-world
$ docker images | grep hello-world
1. Question: what is the purpose of `.` (dot) in the above `docker build`
command?
1. Note that the tag is `latest` (the default).
1. Let's run this image in a container:
$ docker run --rm hello-world
1. Did you see what you expected?

@ -0,0 +1,3 @@
#!/bin/sh
echo "Hello, world."
exit 0

@ -0,0 +1,7 @@
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
RUN chmod a+x /root/run.sh
CMD /root/run.sh

@ -0,0 +1,7 @@
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
RUN chmod a+rx /root/run.sh
CMD /root/run.sh

@ -0,0 +1,211 @@
# Lesson 3: Image layers
1. Change directory into `lesson03`.
1. Build the `hello-world` image with the tag `v1`:
$ docker build . -t hello-world:v1
1. Each Docker image consists of a sequence of file system _layers_ with the
later layers overwriting earlier ones. Each layer corresponds to an
instruction in the `Dockerfile`. Let's look at the layers. (Note: in this and
subsequent displays I leave off the CREATED column to save space).
$ docker history hello-world:v1
IMAGE CREATED BY
f38d648975d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo...
09b895731b10 /bin/sh -c chmod a+x /root/run.sh
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786â¦...
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s...
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"]
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869...
$ docker history debian:buster-slim
IMAGE CREATED BY
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"]
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869...
1. Notice that the `hello-world:v1` image's first two layers are the same
as `debian:buster-slim`'s layers. This reflects the fact that
`hello-world:v1` starts FROM the `debian:buster-slim` image.
1. The subsequent layers of the `hello-world:v1` image each correspond to
the commands in the `Dockerfile`. Here is the Dockerfile again as a
reminder:
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
RUN chmod a+x /root/run.sh
CMD /root/run.sh
1. When you create a running Docker container using `docker run` the
Docker system loads your image with each layer being _read-only_ with a
final, new layer being added. This last layer is writable. Let's try it.
$ docker run --rm --name=fuzzle busybox /bin/sh -c "sleep 10000" &
$ docker exec -ti fuzzle /bin/sh (we "login" to the running container)
/ # cd /root; ls
~ # date > date.txt; ls -l
-rw-r--r-- 1 root root 0 Jan 16 03:56 date.txt
~ # cat /root/date.txt
Sat Jan 16 03:57:15 UTC 2021
~ # exit
1. The container is still running. Let's make sure the file we created is
still there.
$ docker exec -ti fuzzle /bin/sh
~ # cat /root/date.txt
Sat Jan 16 03:57:15 UTC 2021
~ # exit
1. However, once the container exits, that final writable layer is thrown away.
**It does not persist**. Let's see.
$ docker rm fuzzle
# You should get an error here. Why?
1. To stop a running container from the outside use the `docker kill` command.
$ docker kill fuzzle
$ docker rm fuzzle
# This last command returns an error. Why?
1. To see that the file `/root/date.txt` is really gone, let's start the
container again and look.
$ docker run --rm --name=fuzzle busybox /bin/sh -c "sleep 10000" &
$ docker exec fuzzle ls -l /root
1. Notice that we did not login to the container to do an `ls`, rather, we used
the `exec` command.
1. To reiterate, changes made to the containers top writable layer do not
persist. If you want the container to make persistent changes to files
another mechanism is needed such as mounting an external volume or writing
to some other persistent data store; more on that in a later lesson.
1. Back to layers.
1. When we looked at the history of the `hello-world:v1` image we saw this:
$ docker history hello-world:v1
IMAGE CREATED BY
f38d648975d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo...
09b895731b10 /bin/sh -c chmod a+x /root/run.sh
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786â¦...
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s...
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"]
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869...
1. Under the `IMAGE` column are hex strings. Those correspond to the
SHA256 hash of the new layer's content. (More precisely, the SHA256 hash
of the layers configuration object.)
1. Think of these hashes (roughly) corresponding to git commit hashes. The
layers and their hashes are useful for they tell Docker when a layer has
changed. We use a copy of `Dockerfile` with one small change:
# Dockerfile-v2
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD run.sh /root/run.sh
RUN chmod a+rx /root/run.sh # This is the changed line.
CMD /root/run.sh
1. Build the image and look at its history:
$ docker build . -f Dockerfile-v2 -t hello-world:v2
$ docker history hello-world:v2
IMAGE CREATED BY
0dd6cfe9bb51 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo...
39085b76a64f /bin/sh -c chmod a+rx /root/run.sh
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786...
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s...
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"]
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869...
$ docker history hello-world:v1
IMAGE CREATED BY
f38d648975d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo...
09b895731b10 /bin/sh -c chmod a+x /root/run.sh
07d951eff6a0 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786â¦...
34fd6d6fbb15 /bin/sh -c #(nop) LABEL maintainer=adamhl@s...
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"]
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869...
1. Note that the layers have the same image ID except for the layers
starting with the one we changed. The top layers have different id's
since even though they are the same Dockerfile command they derive from
the layer that changed (again, think of a git commit).
1. This has the pleasant result that if the first N Dockerfile commands do
not change but the N+1'st command _does_ change, then `docker build` will
use the cached layers for the first N layers and only rebuild from layer
N+1 onwards. This means rebuilds can be quite fast.
1. **Important.** The image changes whenever one or more of the Dockerfile
`ADD`, `COPY`, or `RUN` command lines change. Even adding whitespace to a
line that has no real effect can trigger a new layer.
1. What does it mean for a Dockerfile line to change? Either the line in
the Dockerfile itself changes (as we saw above) **or**, for commands that
add files from the local environment, the file being added changes. For
example, if we added a comment line to `run.sh` this counts as a change to
the `ADD run.sh /root/run.sh` line and would trigger a layer rebuild from
that layer forward.
1. Edit the file `run.sh` by adding a comment and then rebuild:
#!/bin/sh
# An extra comment
echo "Hello, world."
exit 0
$ docker build . -t hello-world:v3
$ docker history hello-world:v1
IMAGE CREATED BY
2131b85a91d5 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo...
30a489d1e5b0 /bin/sh -c chmod a+x /root/run.sh
94667dcd1388 /bin/sh -c #(nop) ADD file:ae2a94e6e79a95786...
ebca792f7949 /bin/sh -c #(nop) LABEL maintainer=adamhl@s...
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"]
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869...
$ docker history hello-world:v3
IMAGE CREATED BY
acc54a5d1b49 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/roo...
82b9cde2c517 /bin/sh -c chmod a+x /root/run.sh
2a19083a38b0 /bin/sh -c #(nop) ADD file:997845d6d2fdc9492...
ebca792f7949 /bin/sh -c #(nop) LABEL maintainer=adamhl@s...
589ac6f94be4 /bin/sh -c #(nop) CMD ["bash"]
<missing> /bin/sh -c #(nop) ADD file:422aca8901ae3d869...
1. Notice that the `ADD` line has changed its image id.
1. A gotcha: your Dockerfile installs a package downloaded from an
external repository (like Debian). You build the image. A little while
later the package is updated in the Debian repository. You rebuild the
container thinking the rebuild will catch this change. But is does
NOT. The docker build only notices changes to the Dockerfile itself, not
to sources external to the build environment.
1. A way around this gotcha. If you want to be sure that every time you
build your container it rebuilds each layer regardless of any changes to
the Dockerfile use the `--no-cache` option. This has the advantage of
reliability but the disadvantage that it can take significantly more time.
$ docker build . -t hello-world:v3 (this will be fast)
$ docker build . --no-cache -t hello-world:v3 (this will slower)
1. Another advantage of Docker caching layers is that different images
built off the same layer help save space. Recall that images layers are
loaded read-only, so if two different running containers are built off the
same layer, both containers can point to the same physical file. This is
why spawning a second container running on the same image as the first can
be so quick: the base layer is already loaded into memory so all that has
to be done is add the final writable layer that is distinct for each
container.

@ -0,0 +1,4 @@
#!/bin/sh
# Hello
echo "Hello, world."
exit 0

@ -0,0 +1,7 @@
# Dockerfile (version 1)
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD date.sh /root/date.sh
RUN chmod a+x /root/date.sh
CMD /root/date.sh

@ -0,0 +1,100 @@
# Lesson 4: Persisting data
1. Change directory into `lesson04`.
1. We saw in Lesson 3 that data written to the final container layer does
not persist. How do we deal with this? One option is to have the container
write to an external system like a database system or cloud storage. There
is, however, another method: mounting a local file system or directory.
1. Let's create a Docker container that saves the current date to a file. Here is
our first attempt:
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD date.sh /root/date.sh
RUN chmod a+x /root/date.sh
CMD /root/date.sh
# date.sh
#!/bin/sh
echo "echoing date to /tmp/date.output"
date > /tmp/date.output
$ docker build . -t date
$ docker run --rm --name=fuzzle date
echoing date to /tmp/date.output
1. The above will create the file `/tmp/date.output` containing the date
but we know that the file will not persist after the container stops
running. To get around this we "mount" a local directory into the
container's `/tmp` directory. By "local directory" we mean a directory on
the computer where we run the our `docker` commands. We mount the colume
when we run the container.
$ mkdir -p /tmp/docker
$ docker run --rm --name=fuzzle --volume=/tmp/docker:/tmp date
echoing date to /tmp/date.output
$ cat /tmp/docker/date.output
1. You can run a container and override the `CMD` command.
$ docker run --rm --name=fuzzle date ls -ld /etc
drwxr-xr-x 1 root root 4096 Jan 17 17:23 /etc/
(The date.sh script did NOT run)
1. This is especially useful when you want to login to the container and
debug the filesystem or CMD command.
1. You can overwrite a file in the image using the same `--volume`
option. For example, let's overwrite `/etc/debian_version` with a
different file.
$ docker run --rm --name=fuzzle date cat /etc/debian_version
10.7
$ echo "fake-version" > /tmp/deb_ver
$ docker run --rm --name=fuzzle --volume=/tmp/deb_ver:/etc/debian_version date cat /etc/debian_version
fake-version
1. If you mount an external directory onto a container directory
**everything** in the directory in the container is **replaced** with the
external directory.
$ docker run --rm --name=fuzzle date ls -l /usr
total 32
drwxr-xr-x 2 root root 4096 Jan 11 00:00 bin
drwxr-xr-x 2 root root 4096 Nov 22 12:37 games
... more ...
# Create a directory in /tmp with a single file.
$ mkdir -p /tmp/usr; cp test.txt /tmp/usr
# Mount /tmp/usr over /usr in the container
$ docker run --rm --name=fuzzle --volume=/tmp/usr:/usr date ls -l /usr
total 4
-rw-r--r-- 1 52777 root 7 Jan 17 18:04 test.txt
1. You can mount an external directory in read-only mode. This is
particularly useful when injecting secrets or configuration information
into a container. Look for the `ro` in the `--volume` option below.
$ mkdir -p /tmp/secrets
$ echo "my-password" > /tmp/secrets/password
$ docker run --rm --name=fuzzle --volume=/tmp/secrets:/secrets:ro date sleep 100000 &
$ docker exec -ti fuzzle /bin/sh
# cat /secrets/password
my-password
# echo "another secret" >> /secrets/password
/bin/sh: 5: cannot create /secrets/password: Read-only file system
$ docker kill fuzzle
1. You can run the entire container in read-only mode. The `docker run`
option `--read-only` mounts the root filesystem (i.e., everything) in
read-only excepting any externally mounted volumes. This lets you
lock-down the filesystem except for those parts of the filesystem you
know need to be written to (e.g., `/tmp`, `/var/log`, etc.).

@ -0,0 +1,3 @@
#!/bin/sh
echo "echoing date to /tmp/date.output"
date > /tmp/date.output

@ -0,0 +1,36 @@
# Lesson 5: Network access
1. Change directory into `lesson05`.
1. A very popular use for containers is for serving web applications. How
do users get network access to a running container? Put simply, the host
acts as the network proxy for its containers: a network conection is made
to the host which directs the traffic into the container, and vice versa
for outbound traffic.
1. Let's try it:
$ docker pull httpd
$ docker run -dit --name apache --rm -p 8080:80 httpd
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
23c146b78377 httpd "httpd-foreground" 20 seconds ago Up 19 seconds 0.0.0.0:8080->80/tcp apache
1. Note that traffic sent to port 8080 on the *host* gets sent to port 80
in the *container*. The `-d` option tells Docker to run the container in
"detached" mode, i.e., in the background.
1. Look at the network (look for the `Containers` element):
$ docker network inspect bridge
1. Test the application. If running docker on your local machine put this
URL into your browser's address bar: `http://localhost:8080`.
1. If running on a remote host run this command:
$ wget http://localhost:8080 --quiet -O -
1. Don't forget to kill the container:
$ docker kill apache

@ -0,0 +1,7 @@
# Dockerfile
FROM debian:buster-slim
LABEL maintainer="adamhl@stanford.edu"
ADD demo.sh /root/demo.sh
RUN chmod a+x /root/demo.sh
CMD /root/demo.sh

@ -0,0 +1,35 @@
# Lesson 6: Environment variables and configuration
1. Change directory into `lesson06`.
1. A popular way to provide configuration information to a docker container is
via environment variables.
1. Let's try it:
$ docker build . -t demo
$ docker run -it --rm --name fuzzle demo
1. Does the output make sense?
1. Let's pass in a value for the environment variable `HELLO_WORLD`:
$ docker run -it --rm --name fuzzle -e HELLO_WORLD='Hello, world.' demo
1. You can also pass in environment variables via a file:
$ cat file.env
# We define two environment variables (you CAN comment!)
HELLO_WORLD="Hello, world."
ENV_VAR2="another environment variable"
$ docker run -it --rm --name fuzzle --env-file=file.env demo
1. If your Docker application does not have many configuration options
configuring via environment variables is a good method. However, if the
application has complicated or extnesive configuration this may not be
feasible.
1. Don't forget to clean up:
$ docker rmi demo:latest (removes the image demo:latest)

@ -0,0 +1,9 @@
#!/bin/bash
# demo.sh
if [ -z "${HELLO_WORLD}" ]; then
echo "The environment variable HELLO_WORLD is not defined."
else
echo "The environment variable HELLO_WORLD is '${HELLO_WORLD}'."
fi

@ -0,0 +1,3 @@
# We define two environment variables (you CAN comment!)
HELLO_WORLD="Hello, world."
ENV_VAR2="another environment variable"

@ -0,0 +1,6 @@
# Useful External Links
* [Container and
layers](https://docs.docker.com/storage/storagedriver/#container-and-layers):
a good explanation from Docker on image and container layers

@ -0,0 +1,49 @@
[[_TOC_]]
# Good Docker Practices
## Keep the Docker image simple (micro-services)
Although you can run as many processes in a single container as you want,
it is usually a good idea to design a container to do a single task. If
your application does several different things you can always add
"sidecar" containers that do the extra work.
There will be situations where splitting an application into different
containers is too complicated. Be flexible and use your own judgement.
## Use small base images
A smaller image means faster start-up times and less memory used on the
container host. One way to acheive is to use a small base image. A popular
small image is `alpine` based on Alpine Linux. This is a complete Linux
with image size of 5.5MB with its own packages. By comparison,
`debian:buster-slim` is about 70MB.
On the other hand, don't let the drive toward small size get in the way
of needed functionality; remember the IBM Pollyanna Principle: "machines
should work; people should think".
## When possible use container orchestration
Getting containers to interact and cooperate can be tricky, so use one of
the orcestration tools like Kubernetes or Docker Compose to do this.
## Use CI/CD (i.e., automation) to keep Docker images up-to-date
Set up automation to rebuild your Docker images periodically making sure
that you disable caching when building. This way your image will have the
most up-to-date and secure base images.
## Send diagnostic output to standard output
In the traditional server world we are used to sending logs to files. With
Docker containers it is usually better to send diagnostic output to
standard output. Kubernetes and other orchestration tools are designed
with the expectation that logging is sent to standard output.
## Run containers in "read-only" mode
Running a Docker container in read-only mode helps to reduce the attack
surface area of your application. Mount external volumes for those parts
of the file system that need to be writable (`/var/log`, `/tmp`, etc.).

@ -0,0 +1,15 @@
# Prequisites
1. Installation of Docker.
1. Test your Docker install (the hex ID shown below will differ):
$ docker images
# REPOSITORY TAG IMAGE ID CREATED SIZE
... more ...
$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
e5d9363303dd: Pull complete

@ -0,0 +1,23 @@
# Summary of Docker commands used in this tutorial
# List images
docker images
# Pull an image from Docker Hub
docker pull busybox
# Build a Docker image
docker build -t my-tag .
# Build image ignoring cached layers
docker build --no-cache -t my-tag .
# Run container in "interactive" mode
docker run -it <image-name>
# List containers
docker ps
docker ps -a (include stopped containers)
# Connect to a running container (i.e., "login")
docker exec -it <container-name> -- /bin/bash

@ -16,19 +16,46 @@ $ cd 01
$ cat README.md
```
**(!)** Ссылка на wiki-страницу курса: [cs-sh.xyz/d/ompwiki]
**(!)** Не забывайте читать литературу: [список литературы на CSCWiki]
**(!)** И пользоваться ресурсами:
## Как решать задания
- внимательно прочитайте задание;
- если в задании упоминается команда, например, `find`,
попробуйте поискать/попинать `man`, `info`, `help`; но не торопитесь усердствовать
с их чтением: за раз всё в голове, скорее всего, не осядет, поэтому интересные
вещи лучше отложить и читать постепенно и последовательно (а ещё хорошо спать
и регулярно питаться, ага);
- вместо усердного чтения лучше попробуйте прочитать краткое описание и
предположить _что_ эта программа могла бы делать
(обычно секции **NAME**, **SYNOPSIS** в man-страницах или листать оглавление info-страниц
и искать что-нибудь со словами `introduction`, `example`, `real use` и пр.);
unix (и, как следствие, linux) так устроен, что _обычно_ одна программа выполняет
одну функцию, но делает это по возможности хорошо;
- после этого попытайтесь накидать _хоть какое-нибудь рабочее решение_;
- как только у Вас получилось написать первое рабочее решение, вспомните, что Вы
прекрасны;
- попробуйте (возможно, кстати, не сразу, а на следующий день или через
несколько дней) упростить или улучшить Ваше решение; обычно это идёт на пользу
и Вам и Вашим скриптам;
- с каждым таким улучшением, опять же, постарайтесь себя похвалить и
порадоваться, что у Вас получился, _возможно_, более читаемый и красивый
скрипт;
## Материалы
**(!)** [wiki-страница курса]
**(!)** [список литературы на CSCWiki]
**(!)** web-ресурсы:
* посмотреть как исполняется код в ваших любимых языках программирования:
[pythontutor];
* почитать объяснение shell-команды: [explain shell];
[cs-sh.xyz/d/ompwiki]: https://cs-sh.xyz/d/ompwiki
[wiki-страница курса]: https://cs-sh.xyz/d/ompwiki
[pythontutor]: https://pythontutor.com
[explain shell]: https://explainshell.com/explain?cmd=find+.+-name+%27*.py%27+-printf+%22x%22+%7C+grep+-oc+x
[список литературы на CSCWiki]: https://wiki.compscicenter.ru/index.php/%D0%9E%D0%9C%D0%9F_1MIT_%D0%BE%D1%81%D0%B5%D0%BD%D1%8C_1_2022#.D0.9B.D0.B8.D1.82.D0.B5.D1.80.D0.B0.D1.82.D1.83.D1.80.D0.B0
!!!

Loading…
Cancel
Save