Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [unreleased] – …

### Added
- Podman is now officially supported and a related regression introduced in v0.27.1 was fixed
- Is is now possible to provide the path to the Podman or Docker binary on the command-line

## [0.27.1] - 2026-09-01

### Changed
Expand Down
6 changes: 6 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ FROM --platform=$TARGETPLATFORM alpine:3.23.4
# Install the Docker CLI.
RUN apk add --no-cache docker-cli

# Install Podman-Remote without pulling in the misspecified dependencies for local execution.
RUN apk fetch podman-remote && tar -xvf podman-remote-*.apk -C / usr/bin/podman-remote && rm podman-remote-*.apk

# Make Podman-Remote available under its usual name.
RUN ln /usr/bin/podman-remote /usr/bin/podman

# Install Docuum.
COPY --from=build /usr/local/bin/docuum /usr/local/bin/docuum

Expand Down
29 changes: 23 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

[![Build status](https://github.com/stepchowfun/docuum/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/stepchowfun/docuum/actions?query=branch%3Amain)

*Docuum* performs least recently used (LRU) eviction of Docker images to keep the disk usage below a given threshold.
*Docuum* performs least recently used (LRU) eviction of Docker images in Docker or Podman to keep the disk usage below a given threshold.

Docker's built-in `docker image prune --all --filter until=…` command serves a similar purpose. However, the built-in solution isn't ideal since it uses the image creation time, rather than the last usage time, to determine which images to remove. That means it can delete frequently used images, which may be expensive to rebuild or time-consuming to pull.
Docker’s built-in `docker image prune --all --filter until=…` command serves a similar purpose. However, the built-in solution isn't ideal since it uses the image creation time, rather than the last usage time, to determine which images to remove. That means it can delete frequently used images, which may be expensive to rebuild or time-consuming to pull.

Docuum is ideal for use cases such as continuous integration (CI) workers, developer workstations, or any other environment in which Docker images accumulate on disk over time. Docuum works well with tools like [Toast](https://github.com/stepchowfun/toast) and [Docker Compose](https://docs.docker.com/compose/).

Expand All @@ -17,7 +17,7 @@ Docuum is used by Netflix (on its production Kubernetes nodes) and Airbnb (on it
When Docuum first starts and subsequently whenever a new Docker event comes in, LRU eviction is performed until the total disk usage due to Docker images is below the given threshold. This design has a few advantages over evicting images based on a fixed [time to live](https://en.wikipedia.org/wiki/Time_to_live) (TTL), which is what various other tools in the Docker ecosystem do:

1. There is no need to configure and tune an interval to run on. Docuum evicts images immediately whenever the disk usage exceeds the threshold without waiting for any timers.
2. Docuum uses no CPU resources when there is no Docker activity. You can run it on your laptop without worrying about draining your battery.
2. Docuum uses no CPU resources when there is no Docker/Podman activity. You can run it on your laptop without worrying about draining your battery.
3. In order to prevent your disk from filling up, it's more straightforward to set a threshold based on disk usage rather than guessing an appropriate maximum image age.

Docuum also respects the parent-child relationships between images. In particular, it will delete children of a parent before deleting the parent (even if the children were used more recently than the parent), because Docker doesn't allow images with children to be deleted.
Expand All @@ -30,7 +30,7 @@ Once Docuum is [installed](#installation-instructions), you can run it manually
docuum --threshold '10 GB'
```

Docuum will then start listening for Docker events. You can use `Ctrl`+`C` to stop it.
Docuum will then start listening for Docker or Podman events. You can use `Ctrl`+`C` to stop it.

You probably want to run Docuum as a [daemon](https://en.wikipedia.org/wiki/Daemon_\(computing\)), e.g., with [launchd](https://www.launchd.info/), [systemd](https://www.freedesktop.org/wiki/Software/systemd/), etc. See the [Configuring your operating system to run the binary as a daemon](#configuring-your-operating-system-to-run-the-binary-as-a-daemon) section below for instructions.

Expand All @@ -42,6 +42,10 @@ Usage: docuum [OPTIONS]
Options:
-v, --version
Print version
-e, --engine [docker|podman]
Container engine to target [default: docker]
-c, --command <COMMAND>
Path to engine binary to execute [default: "docker" or "podman" depending on `--engine`]
-t, --threshold <THRESHOLD>
Set the maximum amount of space to use for Docker images [default: "10 GB"]
-k, --keep <REGEX>
Expand Down Expand Up @@ -139,6 +143,19 @@ docker container run \
stephanmisc/docuum --threshold '10 GB'
```

For Podman, you must have Podman listening in daemon mode running on the host (`podman system service`), then you can Docuum using Podman:
Comment thread
ntninja marked this conversation as resolved.

```sh
docker container run \
Comment thread
ntninja marked this conversation as resolved.
--init \
--rm \
--tty \
--name docuum \
--mount type=bind,src=/run/podman/podman.sock,dst=/run/podman/podman.sock \
--mount type=volume,source=docuum,target=/root \
stephanmisc/docuum --threshold '10 GB' --engine podman
```

If you're on a Windows system configured to run Linux containers, use this command:

```powershell
Expand All @@ -154,10 +171,10 @@ docker container run `

We don't currently publish a Windows-based image, because some Windows machines (namely, those which run containers with process isolation rather than Hyper-V) can only run Windows containers that were built for the exact build of Windows (e.g., 1809) which is running on the host. This makes Windows-based images less portable, and as a result we'd need to publish a separate Windows-based image for each build of Windows we want to support. At this time, we don't have the infrastructure to do that.

The instructions below for configuring your operating system to run Docuum as a daemon assume it's installed as an executable binary. If you prefer to run it as a Docker container, change the relevant service definition to run a Docker command like the relevant one above, with the following adjustments:
The instructions below for configuring your operating system to run Docuum as a daemon assume it's installed as an executable binary. If you prefer to run it as a Docker container, change the relevant service definition to run a Docker or Podman command like the relevant one above, with the following adjustments:

- Omit the `--tty` flag. This prevents Docuum from printing colored logs, which you probably don't want for a daemon.
- Configure Docker as a hard dependency. Ordinarily, Docuum and Docker can be started in any order, and Docuum will patiently wait for Docker to start if needed. However, when running Docuum as a Docker container, then of course Docker must be started first.
- If using Docker, configure Docker as a hard dependency. Ordinarily, Docuum and Docker can be started in any order, and Docuum will patiently wait for Docker to start if needed. However, when running Docuum as a Docker container, then of course Docker must be started first.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is inside a section called "Running Docuum in a Docker container on a host capable of running Linux containers", I think maybe "If using Docker" reads a bit odd. Maybe we can adapt this whole section to apply to both Docker and Podman? (I see some change in that direction above.)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left the wording of “Docker container” as-is as I see it as a synonym for “OCI container”. I can change the words “Docker container” to something else but otherwise I think the section is already made generic over Docker vs Podman.

(Minus the mistakes in that section you also noticed, of course.)


### Configuring your operating system to run the binary as a daemon

Expand Down
38 changes: 25 additions & 13 deletions integration-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,22 @@
set -euxo pipefail

# See [ref:integration_test_step] for how to run this integration test.
ENGINE=docker
if [ $# -gt 0 ];
then
if [ "$1" = "podman" ];
then
ENGINE=podman
elif [ "$1" != "docker" ];
then
echo "Unexpected argument: Must be “docker” or “podman”" >&2
exit 1
fi
fi

# Wait for the Docker daemon to start up.
echo 'Waiting for Docker to start…'
while ! docker container ls > /dev/null 2>&1; do
echo 'Waiting for $ENGINE to start…'
while ! "$ENGINE" container ls > /dev/null 2>&1; do
sleep 1
done

Expand Down Expand Up @@ -54,13 +66,13 @@ build_test_image() {
# Regenerate the payload so Docker cannot deduplicate layers across test images.
echo "Building $IMAGE…"
head -c "$IMAGE_SIZE_BYTES" /dev/urandom > "$BUILD_CONTEXT_DIR/payload"
docker image build --no-cache --tag "$IMAGE" "$BUILD_CONTEXT_DIR"
"$ENGINE" image build --no-cache --tag "$IMAGE" "$BUILD_CONTEXT_DIR"
}

# Print Docker's current image space usage in bytes.
image_space_bytes() {
# Parse the human-readable image size reported by `docker system df`.
docker system df --format '{{.Type}}\t{{.Size}}' | awk '
"$ENGINE" system df --format '{{.Type}}\t{{.Size}}' | awk '
$1 == "Images" {
SIZE = $2
if (SIZE ~ /kB$/) {
Expand Down Expand Up @@ -99,7 +111,7 @@ echo "Using a threshold of $THRESHOLD_BYTES bytes…"
# Start Docuum in the background, capturing its output in a log file. Trace-level logging is
# needed because the synchronization logic below relies on Docuum logging each incoming event.
echo 'Starting Docuum…'
LOG_LEVEL=trace /docuum-x86_64-unknown-linux-musl --threshold "$THRESHOLD_BYTES B" \
LOG_LEVEL=trace /docuum-x86_64-unknown-linux-musl --engine "$ENGINE" --threshold "$THRESHOLD_BYTES B" \
--keep "$PROTECTED_IMAGE:latest" \
> "$LOG_FILE" 2>&1 &
DOCUUM_PID="$!"
Expand All @@ -116,8 +128,8 @@ wait_for_docuum() {
# Emit a sentinel network event.
echo 'Waiting for Docuum to process events…'
SENTINEL_INDEX="$((SENTINEL_INDEX + 1))"
docker network create "docuum-sentinel-$SENTINEL_INDEX" > /dev/null
docker network rm "docuum-sentinel-$SENTINEL_INDEX" > /dev/null
"$ENGINE" network create "docuum-sentinel-$SENTINEL_INDEX" > /dev/null
"$ENGINE" network rm "docuum-sentinel-$SENTINEL_INDEX" > /dev/null

# Wait until Docuum has received the sentinel event.
while ! grep -q "docuum-sentinel-$SENTINEL_INDEX" "$LOG_FILE"; do
Expand All @@ -142,8 +154,8 @@ wait_for_docuum_to_start() {
local PROBE_INDEX=0
while ! grep -q 'docuum-probe-' "$LOG_FILE"; do
PROBE_INDEX="$((PROBE_INDEX + 1))"
docker network create "docuum-probe-$PROBE_INDEX" > /dev/null
docker network rm "docuum-probe-$PROBE_INDEX" > /dev/null
"$ENGINE" network create "docuum-probe-$PROBE_INDEX" > /dev/null
"$ENGINE" network rm "docuum-probe-$PROBE_INDEX" > /dev/null
sleep 1
done
}
Expand All @@ -153,7 +165,7 @@ wait_for_docuum_to_start

# Assert that the first non-protected image is still present before the threshold has been exceeded.
echo 'Checking that the first non-protected image is still present…'
docker image inspect "$EVICTED_IMAGE" > /dev/null 2>&1
"$ENGINE" image inspect "$EVICTED_IMAGE" > /dev/null 2>&1

# Build the latest image after Docuum is listening so the build emits events Docuum must process.
build_test_image "$LATEST_IMAGE"
Expand All @@ -163,15 +175,15 @@ wait_for_docuum

# Assert that the image protected by the `--keep` flag is still present.
echo 'Checking that the protected image is still present…'
docker image inspect "$PROTECTED_IMAGE" > /dev/null 2>&1
"$ENGINE" image inspect "$PROTECTED_IMAGE" > /dev/null 2>&1

# Assert that the last image is still present.
echo 'Checking that the last image is still present…'
docker image inspect "$LATEST_IMAGE" > /dev/null 2>&1
"$ENGINE" image inspect "$LATEST_IMAGE" > /dev/null 2>&1

# Assert that the first non-protected image was deleted.
echo 'Checking that the first non-protected image was deleted…'
if docker image inspect "$EVICTED_IMAGE" > /dev/null 2>&1
if "$ENGINE" image inspect "$EVICTED_IMAGE" > /dev/null 2>&1
then
echo "The image wasn't deleted."
exit 1
Expand Down
48 changes: 46 additions & 2 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@ mod state;
use crate::{format::CodeStr, run::run};
use byte_unit::Byte;
use chrono::Local;
use clap::{ArgAction, Parser};
use clap::{ArgAction, Parser, ValueEnum};
use env_logger::{Builder, fmt::style::Effects};
use humantime::parse_duration;
use log::LevelFilter;
use regex::RegexSet;
use std::{
env,
fmt::{Display, Formatter},
io::{self, IsTerminal, Write},
process::exit,
str::FromStr,
Expand Down Expand Up @@ -111,6 +112,22 @@ struct Cli {
#[arg(short, long, help = "Print version", action = ArgAction::Version)]
_version: Option<bool>,

#[arg(
value_enum,
short,
long,
help = "Container engine to target",
default_value_t = Engine::Docker
)]
engine: Engine,

#[arg(
short,
long,
help = "Path to engine binary to execute\n\n[default: \"docker\" or \"podman\" depending on `--engine`]",
)]
command: Option<String>,

#[arg(
short,
long,
Expand Down Expand Up @@ -145,14 +162,34 @@ struct Cli {
min_age: Option<String>,
}

// This struct represents the parsed command-line arguments.
/// Represents the parsed command-line arguments.
pub struct Settings {
engine: Engine,
command: String,
deletion_chunk_size: usize,
keep: Option<RegexSet>,
min_age: Option<Duration>,
threshold: Threshold,
}

/// Container engine to target.
#[derive(Clone, Copy, Debug, PartialEq, ValueEnum)]
enum Engine {
/// Docker container engine
Docker,
/// Podman container engine
Podman,
}

impl Display for Engine {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
f.write_str(match self {
Self::Docker => "Docker",
Self::Podman => "Podman",
})
}
}

// Set up the logger.
fn set_up_logging() {
Builder::new()
Expand Down Expand Up @@ -212,6 +249,13 @@ fn settings() -> io::Result<Settings> {
let threshold = Threshold::from_str(&cli.threshold)?;

Ok(Settings {
engine: cli.engine,
command: cli.command.unwrap_or_else(||
String::from(match cli.engine {
Engine::Docker => "docker",
Engine::Podman => "podman",
})
),
deletion_chunk_size,
keep,
min_age,
Expand Down
Loading
Loading