{ "componentChunkName": "component---src-templates-issues-tsx", "path": "/issues/569", "result": {"data":{"issuesJson":{"id":"d17c7839-ca84-511f-81c0-f167d5ec8281","title":"stepchowfun/toast: 一个基于名为 toastfile 的 YAML 文件中定义任务，用于容器化工作流程的工具。","number":569,"bodyHTML":"
\n



\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Reposstepchowfun/toast
Views\"views\"
Stars\"stars\"
Forks\"forks\"
License\"license\"
UpdatedAt\"last-commit\"
CreatedAt\"create-at\"
\n\n

\"\"

\n\n
\n

Toast 🥂

\n

\"Build

\n

Toast is a tool for containerizing your workflows such as building and testing a project. You define tasks in a YAML file called a toastfile, and Toast runs them in a container based on a Docker image of your choosing. What constitutes a \"task\" is up to you: tasks can install system packages, compile an application, run a test suite, or even serve web pages. Tasks can depend on other tasks, so Toast can be understood as a high-level containerized build system.

\n

\"Welcome

\n

Here's the toastfile for the example shown above:

\n
image: ubuntu\ntasks:\n  install_gcc:\n    command: |\n      apt-get update\n      apt-get install --yes gcc\n\n  build:\n    dependencies:\n      - install_gcc\n    input_paths:\n      - main.c\n    command: gcc main.c\n\n  run:\n    dependencies:\n      - build\n    command: ./a.out
\n

Toast caches each task by committing the container to an image. The image is tagged with a cryptographic hash of the shell command for the task, the contents of the files copied into the container, and all the other task inputs. This hash allows Toast to skip tasks that haven't changed since the last run.

\n

In addition to local caching, Toast can use a Docker registry as a remote cache. You, your teammates, and your continuous integration (CI) system can all share the same remote cache. Used in this way, your CI system can do all the heavy lifting like building and installing dependencies so you and your team can focus on development.

\n

Related tools:

\n\n

To prevent Docker images from accumulating on your machine when using Docker-related tools such as Toast or Docker Compose, we recommend using Docuum to perform least recently used (LRU) image eviction.

\n

Tutorial

\n

Defining a simple task

\n

Let's create a toastfile. Create a file named toast.yml with the following contents:

\n
image: ubuntu\ntasks:\n  greet:\n    command: echo 'Hello, World!' # Toast will run this in a container.
\n

Now run toast. You should see the following:

\n

\"Defining

\n

If you run it again, Toast will find that nothing has changed and skip the task:

\n

\"Caching.\"

\n

Toast caches tasks to save you time. For example, you don't want to reinstall your dependencies every time you run your tests. However, caching may not be appropriate for some tasks, like running a development server. You can disable caching for a specific task and all tasks that depend on it with the cache option:

\n
image: ubuntu\ntasks:\n  greet:\n    cache: false # Don't cache this task.\n    command: echo 'Hello, World!'
\n

Adding a dependency

\n

Let's make the greeting more fun with a program called figlet. We'll add a task to install figlet, and we'll change the greet task to depend on it:

\n
image: ubuntu\ntasks:\n  install_figlet:\n    command: |\n      apt-get update\n      apt-get install --yes figlet\n\n  greet:\n    dependencies:\n      - install_figlet # Toast will run this task first.\n    command: figlet 'Hello, World!'
\n

Run toast to see a marvelous greeting:

\n

\"Adding

\n

Importing files from the host

\n

Here's a more realistic example. Suppose you want to compile and run a simple C program. Create a file called main.c:

\n
#include <stdio.h>\n\nint main(void) {\n  printf(\"Hello, World!\\n\");\n  return 0;\n}
\n

Update toast.yml to compile and run the program:

\n
image: ubuntu\ntasks:\n  install_gcc:\n    command: |\n      apt-get update\n      apt-get install --yes gcc\n\n  build:\n    dependencies:\n      - install_gcc\n    input_paths:\n      - main.c # Toast will copy this file into the container before running the command.\n    command: gcc main.c\n\n  run:\n    dependencies:\n      - build\n    command: ./a.out
\n

Notice the input_paths array in the build task. Here we're copying a single file into the container, but we could instead import the entire directory containing the toastfile with .. By default, the files will be copied into a directory called /scratch in the container. The commands will be run in that directory as well.

\n

Now if you run toast, you'll see this:

\n

\"Importing

\n

For subsequent runs, Toast will skip the task if nothing has changed. But if you update the greeting in main.c, Toast will detect the change and rerun the build and run tasks on the next invocation.

\n

Exporting files from the container

\n

A common use case for Toast is to build a project. Naturally, you might wonder how to access the build artifacts produced inside the container from the host machine. It's easy to do with output_paths:

\n
image: ubuntu\ntasks:\n  install_gcc:\n    command: |\n      apt-get update\n      apt-get install --yes gcc\n\n  build:\n    dependencies:\n      - install_gcc\n    input_paths:\n      - main.c\n    output_paths:\n      - a.out # Toast will copy this file onto the host after running the command.\n    command: gcc main.c
\n

When Toast runs the build task, it will copy the a.out file to the host.

\n

\"Exporting

\n

Passing arguments to a task

\n

Sometimes it's useful for tasks to take arguments. For example, a deploy task might want to know whether you want to deploy to the staging or production cluster. To do this, add an environment section to your task:

\n
image: ubuntu\ntasks:\n  deploy:\n    cache: false\n    environment:\n      CLUSTER: staging # Deploy to staging by default.\n    command: echo \"Deploying to $CLUSTER...\"
\n

When you run this task, Toast will read the value from the environment:

\n

\"Passing

\n

If the variable doesn't exist in the environment, Toast will use the default value:

\n

\"Using

\n

If you don't want to have a default, set it to null:

\n
image: ubuntu\ntasks:\n  deploy:\n    cache: false\n    environment:\n      CLUSTER: null # No default; this variable must be provided at runtime.\n    command: echo \"Deploying to $CLUSTER...\"
\n

Now if you run toast deploy without specifying a CLUSTER, Toast will complain about the missing variable and refuse to run the task.

\n

Environment variables listed in a task are also set for any tasks that run after it.

\n

Running a server and mounting paths into the container

\n

Toast can be used for more than just building a project. Suppose you're developing a website. You can define a Toast task to run your web server! Create a file called index.html with the following contents:

\n
<!DOCTYPE html>\n<html>\n  <head>\n    <title>Welcome to Toast!</title>\n  </head>\n  <body>\n    <p>Hello, World!</p>\n  </body>\n</html>
\n

We can use a web server like nginx. The official nginx Docker image will do, but you could also use a more general image and define a Toast task to install nginx.

\n

In our toast.yml file, we'll use the ports field to make the website accessible outside the container. We'll also use mount_paths rather than input_paths so we can edit the web page without having to restart the server.

\n
image: nginx\ntasks:\n  serve:\n    cache: false # It doesn't make sense to cache this task.\n    mount_paths:\n      - index.html # Updates to this file will be visible inside the container.\n    ports:\n      - 3000:80 # Expose port 80 in the container as port 3000 on the host.\n    location: /usr/share/nginx/html/ # Nginx will serve the files in here.\n    command: nginx -g 'daemon off;' # Run in foreground mode.
\n

Now you can use Toast to run the server:

\n

\"Running

\n

Configuring the shell

\n

It's often desirable to configure the shell in some way before running any commands. Shells are typically configured with so-called \"startup files\" (e.g., ~/.bashrc). However, many shells skip loading such configuration files when running in non-interactive, non-login mode, which is how the shell is invoked by Toast. Toast provides an alternative mechanism to configure the shell that doesn't require creating any special files or invoking the shell in a particular way.

\n

Consider the following toastfile which uses Bash as the shell, since that's the default preferred login shell in Ubuntu:

\n
image: ubuntu\ntasks:\n  install_figlet:\n    command: |\n      apt-get update\n      apt-get install --yes figlet
\n

What happens if apt-get update fails? Due to the way Bash works, the failure would be ignored and execution would continue to the subsequent line. You can fix this with set -e as follows:

\n
image: ubuntu\ntasks:\n  install_figlet:\n    command: |\n      set -e # Make Bash fail fast.\n      apt-get update\n      apt-get install --yes figlet
\n

However, it's tedious and error-prone to add that to each task separately. Instead, you can add it to every task at once by setting command_prefix as follows:

\n
image: ubuntu\ncommand_prefix: set -e # Make Bash fail fast.\ntasks:\n  install_figlet:\n    command: |\n      apt-get update\n      apt-get install --yes figlet
\n

For Bash in particular, we recommend going even further and setting set -euxo pipefail instead of just set -e.

\n

Dropping into an interactive shell

\n

If you run Toast with --shell, Toast will drop you into an interactive shell inside the container when the requested tasks are finished, or if any of them fails. This feature is useful for debugging tasks or exploring what's in the container. Suppose you have the following toastfile:

\n
image: ubuntu\ntasks:\n  install_figlet:\n    command: |\n      apt-get update\n      apt-get install --yes figlet
\n

You can run toast --shell to play with the figlet program:

\n

\"Dropping

\n

When you're done, the container is deleted automatically.

\n

How Toast works

\n

Given a set of tasks to run, Toast computes a topological sort of the dependency DAG to determine in what order to run the tasks. Toast then builds a Docker image for each task based on the image from the previous task in the topological sort, or the base image in the case of the first task.

\n

The topological sort of an arbitrary DAG isn't necessarily unique. Toast uses an algorithm based on depth-first search, traversing children in lexicographical order. The algorithm is deterministic and invariant to the order in which tasks and dependencies are listed, so reordering tasks in a toastfile won't invalidate the cache. Furthermore, toast foo bar and toast bar foo are guaranteed to produce identical schedules to maximize cache utilization.

\n

For each task in the schedule, Toast first computes a cache key based on a hash of the shell command, the contents of the input_paths, the cache key of the previous task in the schedule, etc. Toast will then look for a Docker image tagged with that cache key. If the image is found, Toast will skip the task. Otherwise, Toast will create a container, copy any input_paths into it, run the shell command, copy any output_paths from the container to the host, commit the container to an image, and delete the container. The image is tagged with the cache key so the task can be skipped for subsequent runs.

\n

Toast aims to make as few assumptions about the container environment as possible. Toast only assumes there is a program at /bin/su which can be invoked as su -c COMMAND USER. This program is used to run commands for tasks in the container as the appropriate user with their preferred shell. Every popular Linux distribution has a su utility that supports this usage. Toast has integration tests to ensure it works with popular base images such as debian, alpine, busybox, etc.

\n

Toastfile reference

\n

A toastfile is a YAML file (typically named toast.yml) that defines tasks and their dependencies. The schema contains the following top-level keys and defaults:

\n
image: <required>   # Docker image name with optional tag or digest\ndefault: null       # Name of default task to run or `null` to run all tasks by default\nlocation: /scratch  # Path in the container for running tasks\nuser: root          # Name of the user in the container for running tasks\ncommand_prefix: ''  # A string to be prepended to all commands by default\ntasks: {}           # Map from task name to task
\n

Tasks have the following schema and defaults:

\n
description: null           # A description of the task for the `--list` option\ndependencies: []            # Names of dependencies\ncache: true                 # Whether a task can be cached\nenvironment: {}             # Map from environment variable to optional default\ninput_paths: []             # Paths to copy into the container\nexcluded_input_paths: []    # A denylist for `input_paths`\noutput_paths: []            # Paths to copy out of the container if the task succeeds\noutput_paths_on_failure: [] # Paths to copy out of the container if the task fails\nmount_paths: []             # Paths to mount into the container\nmount_readonly: false       # Whether to mount the `mount_paths` as readonly\nports: []                   # Port mappings to publish\nlocation: null              # Overrides the corresponding top-level value\nuser: null                  # Overrides the corresponding top-level value\ncommand: ''                 # Shell command to run in the container\ncommand_prefix: null        # Overrides the corresponding top-level value\nextra_docker_arguments: []  # Additional arguments for `docker container create`
\n

The toastfile for Toast itself is a comprehensive real-world example.

\n

Configuration

\n

Toast can be customized with a YAML configuration file. The default location of the configuration file depends on the operating system:

\n\n

The schema of the configuration file is described in the subsections below.

\n

Cache configuration

\n

Toast supports local and remote caching. By default, only local caching is enabled. Remote caching requires that the Docker Engine is logged into a Docker registry (e.g., via docker login).

\n

The cache-related fields and their default values are as follows:

\n
docker_repo: toast        # Docker repository\nread_local_cache: true    # Whether Toast should read from local cache\nwrite_local_cache: true   # Whether Toast should write to local cache\nread_remote_cache: false  # Whether Toast should read from remote cache\nwrite_remote_cache: false # Whether Toast should write to remote cache
\n

Each of these options can be overridden via command-line options (see below).

\n

A typical configuration for a CI environment will enable all forms of caching, whereas for local development you may want to set write_remote_cache: false to avoid waiting for remote cache writes.

\n

Docker CLI

\n

You can configure the Docker CLI binary used by Toast. Toast uses the PATH environment variable to search for the specified binary. You can use this mechanism to switch to a drop-in replacement for the Docker CLI, such as Podman.

\n

The relevant field and its default value are as follows:

\n
docker_cli: docker
\n

Command-line options

\n

By default, Toast looks for a toastfile called toast.yml in the working directory, then in the parent directory, and so on. Any paths in the toastfile are relative to where the toastfile lives, not the working directory. This means you can run Toast from anywhere in your project and get the same results.

\n

Run toast with no arguments to execute the default task, or all the tasks if the toastfile doesn't define a default. You can also execute specific tasks and their dependencies:

\n
toast task1 task2 task3…
\n

Here are all the supported command-line options:

\n
USAGE:\n    toast [OPTIONS] [--] [TASKS]...\n\nOPTIONS:\n    -c, --config-file <PATH>\n            Sets the path of the config file\n\n        --docker-cli <CLI>\n            Sets the Docker CLI binary\n\n    -r, --docker-repo <REPO>\n            Sets the Docker repository for remote caching\n\n    -f, --file <PATH>\n            Sets the path to the toastfile\n\n        --force <TASK>...\n            Runs a task unconditionally, even if it’s cached\n\n        --force-all\n            Pulls the base image and runs all tasks unconditionally\n\n    -h, --help\n            Prints help information\n\n    -l, --list\n            Lists the tasks that have a description\n\n    -o, --output-dir <PATH>\n            Sets the output directory\n\n        --read-local-cache <BOOL>\n            Sets whether local cache reading is enabled\n\n        --read-remote-cache <BOOL>\n            Sets whether remote cache reading is enabled\n\n    -s, --shell\n            Drops you into a containerized shell after the tasks are finished\n\n    -v, --version\n            Prints version information\n\n        --write-local-cache <BOOL>\n            Sets whether local cache writing is enabled\n\n        --write-remote-cache <BOOL>\n            Sets whether remote cache writing is enabled\n\n\nARGS:\n    <TASKS>...\n            Sets the tasks to run\n
\n

Installation instructions

\n

Installation on macOS or Linux (AArch64 or x86-64)

\n

If you're running macOS or Linux (AArch64 or x86-64), you can install Toast with this command:

\n
curl https://raw.githubusercontent.com/stepchowfun/toast/main/install.sh -LSfs | sh
\n

The same command can be used again to update to the latest version.

\n

The installation script supports the following optional environment variables:

\n\n

For example, the following will install Toast into the working directory:

\n
curl https://raw.githubusercontent.com/stepchowfun/toast/main/install.sh -LSfs | PREFIX=. sh
\n

If you prefer not to use this installation method, you can download the binary from the releases page, make it executable (e.g., with chmod), and place it in some directory in your PATH (e.g., /usr/local/bin).

\n

Installation on Windows (AArch64 or x86-64)

\n

If you're running Windows (AArch64 or x86-64), download the latest binary from the releases page and rename it to toast (or toast.exe if you have file extensions visible). Create a directory called Toast in your %PROGRAMFILES% directory (e.g., C:\\Program Files\\Toast), and place the renamed binary in there. Then, in the \"Advanced\" tab of the \"System Properties\" section of Control Panel, click on \"Environment Variables...\" and add the full path to the new Toast directory to the PATH variable under \"System variables\". Note that the Program Files directory might have a different name if Windows is configured for a language other than English.

\n

To update an existing installation, simply replace the existing binary.

\n

Installation with Homebrew

\n

If you have Homebrew, you can install Toast as follows:

\n
brew install toast
\n

You can update an existing installation with brew upgrade toast.

\n

Installation with MacPorts

\n

On macOS, you can also install Toast via MacPorts as follows:

\n
sudo port install toast
\n

You can update an existing installation via:

\n
sudo port selfupdate\nsudo port upgrade toast
\n

Installation with Cargo

\n

If you have Cargo, you can install Toast as follows:

\n
cargo install toast
\n

You can run that command with --force to update an existing installation.

\n

Running Toast in CI

\n

The easiest way to run Toast in CI is to use GitHub Actions. Toast provides a convenient GitHub action that you can use in your workflows. Here's a simple workflow that runs Toast with no arguments:

\n
# .github/workflows/ci.yml\nname: Continuous integration\non:\n  pull_request:\n  push:\n    branches:\n    - main\njobs:\n  ci:\n    runs-on: ubuntu-latest\n    steps:\n    - uses: actions/checkout@v4\n    - uses: stepchowfun/toast/.github/actions/toast@main
\n

Here's a more customized workflow that showcases all the options:

\n
# .github/workflows/ci.yml\nname: Continuous integration\non:\n  pull_request:\n  push:\n    branches:\n    - main\njobs:\n  ci:\n    runs-on: ubuntu-latest\n    steps:\n    - uses: actions/checkout@v4\n    - if: github.event_name == 'push'\n      uses: docker/login-action@v3\n      with:\n        username: DOCKER_USERNAME\n        password: ${{ secrets.DOCKER_PASSWORD }}\n    - uses: stepchowfun/toast/.github/actions/toast@main\n      with:\n        file: toastfiles/toast.yml\n        tasks: build lint test\n        docker_repo: DOCKER_USERNAME/DOCKER_REPO\n        read_remote_cache: true\n        write_remote_cache: ${{ github.event_name == 'push' }}
\n

Requirements

\n\n

Acknowledgements

\n

Toast was inspired by an in-house tool used at Airbnb for CI jobs. The design was heavily influenced by the lessons I learned working on that tool and building out Airbnb's CI system with the fabulous CI Infrastructure Team.

\n

Special thanks to Julia Wang (@juliahw) for valuable early feedback. Thanks to her and Mark Tai (@marktai) for coming up with the name Toast.

\n

The terminal animations were produced with asciinema and svg-term-cli.

","updatedAt":"2025-05-22T01:51:14Z","upvoteCount":null,"author":{"login":"eryajf","avatarUrl":"https://avatars.githubusercontent.com/u/33259379?u=e4a4090a38ac2473aaed4ef9945233636776c6c3&v=4","url":"https://github.com/eryajf"},"category":null,"labels":{"edges":[{"node":{"name":"更多","color":"25B472"}},{"node":{"name":"命令行工具","color":"e05879"}},{"node":{"name":"stepchowfun","color":"a3bd31"}}]},"comments":{"edges":[]}}},"pageContext":{"number":569,"previous":{"title":"tl-open-source/tl-rtc-file: 基于 WebRTC 的媒体流传输工具","number":568},"next":{"title":"silenceshell/topic: 基于 Golang 实现的容器内运行 top 命令的工具","number":570}}}, "staticQueryHashes": ["151096407","2861350382"]}