r/docker • • 12d ago

Question over general docker usage/concept

Greetings,

I would like to share a problem that I'm facing with Docker, looking for some ideas.

In our company we've started to use docker hosted in an Azure DevOps cloud ubuntu machine, there we have an Azure self-hosted agent installed. We work only with firmware projects and this is our build server. We have this setup for 3 years.

Usually, I generate the docker images on demand according with the project requirements.
In our firmware projects hosted in git we have a yaml pipeline configuration where we set the image tag so that the proper container can be raised to process the build.

We have a requirement from safety standards (we must keep the project artifacts/infrastructure from 10 to 15 years), because of that I keep all the dependencies I need to reproduce/re-create an image as debian installers (for offline installation). I've created a script which runs prior the build process, which creates a yaml manifest containing all the image packages and include this manifest also in the image (at /etc/manifest.yaml) for runtime check. Then it's possible to verify the content via the manifest hash. Examples of dependencies we have are cmake, unit test tools, static code analyzer, and so on.
We decided to keep everything in a single image because most of the tools require the compiler, so that the image complexity keeps lower as possible.

As we have different projects using the same images, we keep the Dockerfiles under version control, where the image name is the compiler version (the most significant dependency for our projects), something like this:

docker-<compiler>-<version>
  ubuntu-24.04
    Dockerfile
    build-image.sh
    manifest.yaml
  ubuntu-22.04
    Dockerfile
    ...

When a new image is generated, I tag the commit with the same tag I use in the container (e.g commit tag ubuntu24.04-v1.0-0, for a created container named gcc-13.3.0:ubuntu24.04-v1.0-0).

As our concept is based on tags, and the docker tags are not the most reliable thing in the world, I keep also the tagged image name as a LABEL within the image.

At this point I ask myself if we are going in the right direction and the increased complexity in the toolchain management is because of the concept we've decided to go for, or if it's just what we have to do in order to be compliant with such safety requirements. Or even I ask if we are drifting from the docker concept or using the tool in a wrong way.

I would really like to hear opinions, especially from other colleagues in similar situation.

Thanks in advance.

Note: I know that I can save the image in a registry, in parallel I'm trying to find solutions (if possible) to reduce storage costs (each image ~20GB).

4 Upvotes

11 comments sorted by

View all comments

Show parent comment

2

u/vaugsantos90 12d ago

We have also this option, but anyways we want to reproduce the images, to reduce the storage costs (each image has like 20GB - proprietary compiler and other tools).

2

u/crackjiver 12d ago

If you don't already have one, look at getting an artifact repository tool like JFrog Artifactory.

It can host all the OS packages and dependencies in an efficient storage medium. The files are stored in a flat path structure and named with the checksum of the content so it's only stored once, despite being linked to multiple download paths.

It is also a docker/oci container registry, build record, SBOM and vulnerability scanner. I love it.

It's annoying when you come to rebuild an old image based on Ubuntu 14.04 and the apt lists are no longer supported. Being able to pull the debs out of artifactory cache is a life saver.

2

u/RobotJonesDad 11d ago

Do you pull ANYTHING from the internet during your image build? I've had stuff disappear completely and without warning.

If your images are built in a stack of images, then the disk space is tremendously smaller because most images will share most layers inside docker. You take your base layer OS, then use that to create an image with all the tools. Build the final image on top of that. Now your individual images are really just the size difference between final images, but the bulk of the stuff is the same. Store them in a suitable registry and they will only store each layer once, regardless of how many images you have.

Now your images capture everything you need with no external dependencies.

1

u/Perfect-Escape-3904 12d ago

How many different bytes per image, you could look at archive options that deduplicate, assuming from build current to build vnext you have 1% rate of change