Nostr Web Client

So a docker replacement should do:

1. namespace isolation, pivot_root: nspawn and containerd do this part (so does my C program but I wouldn't trust it to be modern enough)

2. cgroup setup - limits on memory, cpu, and disk that a process group can utilize

3. layered filesystem - overlayfs (AUFS and unionfs probably are worse)

4. syncing filesystem layers from a server - like git pull - probably just using git actually.

5. some kind of networking that lets the containered process talk to the internet

I'm probably missing something important.

cloud fodder 2y ago

Still not sure if the filesystem layering was really a necessary part of containers. Having built thousands of docker containers and managing deployment pipelines I noticed how hard it is to actually utilize the layers effectively. Almost any change can easily blow the cache of a docker build. Usually you should be updating the OS frequently anyway for security reasons and caching it can lead to laziness on updates.

Systemd nspawn uses bind mounts and isolated filesystem access (like a chroot) which is nice for performance. It is a slight paradigm shift on what a container means, so far so good though..

One thing I've been really liking so far too is you can Boot your containers for multi process... For my use cases this is way superior to single process. For example, haproxy does hot reloading by managing processes. Additional processes managed by systemd from inside a container are also really handy.

Reply to this note

Please Login to reply.

Discussion

Mike Dilger ☑️ 2y ago

It seems to me that you should update your OS filesystem layer regularly, and many different containers that run on it would then benefit from that one update. That presents a risk that an update might break something, but as long as you don't jump major versions I think it is a reasonable strategy. This is how QubesOS layers things (it usees Xen virtualization) - you do OS updates in TemplateVMs and you build AppVMs to use an underlying TemplateVM.

cloud fodder 2y ago

Yep, that's the pattern I am using now.. you have the base OS filesystem from a debootstrap (Debian), then you can clone it and build the various app images on top. If you wanted to you could build smaller and smaller images, all depends on if you want to treat the container like a single process or have extra tools / shell and init system inside.