{ "componentChunkName": "component---src-templates-issues-tsx", "path": "/issues/58", "result": {"data":{"issuesJson":{"id":"737b1a70-4e64-5fc9-9218-ecfd0e12d3ff","title":"moby/buildkit: 增强 Docker build 的功能，支持并行构建、缓存、镜像导出等功能","number":58,"bodyHTML":"
\n



\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Reposmoby/buildkit
Views\"views\"
Stars\"stars\"
Forks\"forks\"
License\"license\"
UpdatedAt\"last-commit\"
CreatedAt\"create-at\"
\n\n

\"\"

\n\n
\n

\"asciicinema

\n

BuildKit

\n

\"GitHub
\n\"PkgGoDev\"
\n\"CI
\n\"CI
\n\"Go
\n\"Codecov\"

\n

BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner.

\n

Key features:

\n\n

Read the proposal from moby/moby#32925

\n

Introductory blog post https://blog.mobyproject.org/introducing-buildkit-17e056cc5317

\n

Join #buildkit channel on Docker Community Slack

\n

Note

If you are visiting this repo for the usage of BuildKit-only Dockerfile features
\nlike RUN --mount=type=(bind|cache|tmpfs|secret|ssh), please refer to the
\nDockerfile reference.

\n
\n

Note

docker build uses Buildx and BuildKit by default since Docker Engine 23.0.
\nYou don't need to read this document unless you want to use the full-featured
\nstandalone version of BuildKit.

\n
\n\n\n\n\n

Used by

\n

BuildKit is used by the following projects:

\n\n

Quick start

\n

ℹ️ For Kubernetes deployments, see examples/kubernetes.

\n

BuildKit is composed of the buildkitd daemon and the buildctl client.
\nWhile the buildctl client is available for Linux, macOS, and Windows, the buildkitd daemon is only available for Linux and *Windows currently.

\n

The latest binaries of BuildKit are available here for Linux, macOS, and Windows.

\n

Linux Setup

\n

The buildkitd daemon requires the following components to be installed:

\n\n

Starting the buildkitd daemon:
\nYou need to run buildkitd as the root user on the host.

\n
$ sudo buildkitd
\n

To run buildkitd as a non-root user, see docs/rootless.md.

\n

The buildkitd daemon supports two worker backends: OCI (runc) and containerd.

\n

By default, the OCI (runc) worker is used. You can set --oci-worker=false --containerd-worker=true to use the containerd worker.

\n

We are open to adding more backends.

\n

To start the buildkitd daemon using systemd socket activation, you can install the buildkit systemd unit files.
\nSee Systemd socket activation

\n

The buildkitd daemon listens gRPC API on /run/buildkit/buildkitd.sock by default, but you can also use TCP sockets.
\nSee Expose BuildKit as a TCP service.

\n

Windows Setup

\n

See instructions and notes at docs/windows.md.

\n

macOS Setup

\n

Homebrew formula (unofficial) is available for macOS.

\n
$ brew install buildkit
\n

The Homebrew formula does not contain the daemon (buildkitd).

\n

For example, Lima can be used for launching the daemon inside a Linux VM.

\n
brew install lima\nlimactl start template://buildkit\nexport BUILDKIT_HOST=\"unix://$HOME/.lima/buildkit/sock/buildkitd.sock\"
\n

Build from source

\n

To build BuildKit from source, see .github/CONTRIBUTING.md.

\n

For a buildctl reference, see this document.

\n

Exploring LLB

\n

BuildKit builds are based on a binary intermediate format called LLB that is used for defining the dependency graph for processes running part of your build. tl;dr: LLB is to Dockerfile what LLVM IR is to C.

\n\n

See solver/pb/ops.proto for the format definition, and see ./examples/README.md for example LLB applications.

\n

Currently, the following high-level languages have been implemented for LLB:

\n\n

Exploring Dockerfiles

\n

Frontends are components that run inside BuildKit and convert any build definition to LLB. There is a special frontend called gateway (gateway.v0) that allows using any image as a frontend.

\n

During development, Dockerfile frontend (dockerfile.v0) is also part of the BuildKit repo. In the future, this will be moved out, and Dockerfiles can be built using an external image.

\n

Building a Dockerfile with buildctl

\n
buildctl build \\\n    --frontend=dockerfile.v0 \\\n    --local context=. \\\n    --local dockerfile=.\n# or\nbuildctl build \\\n    --frontend=dockerfile.v0 \\\n    --local context=. \\\n    --local dockerfile=. \\\n    --opt target=foo \\\n    --opt build-arg:foo=bar
\n

--local exposes local source files from client to the builder. context and dockerfile are the names Dockerfile frontend looks for build context and Dockerfile location.

\n

If the Dockerfile has a different filename it can be specified with --opt filename=./Dockerfile-alternative.

\n

Building a Dockerfile using external frontend

\n

External versions of the Dockerfile frontend are pushed to https://hub.docker.com/r/docker/dockerfile-upstream and https://hub.docker.com/r/docker/dockerfile and can be used with the gateway frontend. The source for the external frontend is currently located in ./frontend/dockerfile/cmd/dockerfile-frontend but will move out of this repository in the future (#163). For automatic build from master branch of this repository docker/dockerfile-upstream:master or docker/dockerfile-upstream:master-labs image can be used.

\n
buildctl build \\\n    --frontend gateway.v0 \\\n    --opt source=docker/dockerfile \\\n    --local context=. \\\n    --local dockerfile=.\nbuildctl build \\\n    --frontend gateway.v0 \\\n    --opt source=docker/dockerfile \\\n    --opt context=https://github.com/moby/moby.git \\\n    --opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org
\n

Output

\n

By default, the build result and intermediate cache will only remain internally in BuildKit. An output needs to be specified to retrieve the result.

\n

Image/Registry

\n
buildctl build ... --output type=image,name=docker.io/username/image,push=true
\n

To export the image to multiple registries:

\n
buildctl build ... --output type=image,\\\"name=docker.io/username/image,docker.io/username2/image2\\\",push=true
\n

To export the cache embed with the image and pushing them to registry together, type registry is required to import the cache, you should specify --export-cache type=inline and --import-cache type=registry,ref=.... To export the cache to a local directly, you should specify --export-cache type=local.
\nDetails in Export cache.

\n
buildctl build ...\\\n  --output type=image,name=docker.io/username/image,push=true \\\n  --export-cache type=inline \\\n  --import-cache type=registry,ref=docker.io/username/image
\n

Keys supported by image output:

\n\n

If credentials are required, buildctl will attempt to read Docker configuration file $DOCKER_CONFIG/config.json.
\n$DOCKER_CONFIG defaults to ~/.docker.

\n

Local directory

\n

The local client will copy the files directly to the client. This is useful if BuildKit is being used for building something else than container images.

\n
buildctl build ... --output type=local,dest=path/to/output-dir
\n

To export specific files use multi-stage builds with a scratch stage and copy the needed files into that stage with COPY --from.

\n
...\nFROM scratch as testresult\n\nCOPY --from=builder /usr/src/app/testresult.xml .\n...
\n
buildctl build ... --opt target=testresult --output type=local,dest=path/to/output-dir
\n

With a multi-platform build, a subfolder matching
\neach target platform will be created in the destination directory:

\n
FROM busybox AS build\nARG TARGETOS\nARG TARGETARCH\nRUN mkdir /out && echo foo > /out/hello-$TARGETOS-$TARGETARCH\n\nFROM scratch\nCOPY --from=build /out /
\n
$ buildctl build \\\n  --frontend dockerfile.v0 \\\n  --opt platform=linux/amd64,linux/arm64 \\\n  --output type=local,dest=./bin/release\n\n$ tree ./bin\n./bin/\n└── release\n    ├── linux_amd64\n    │   └── hello-linux-amd64\n    └── linux_arm64\n        └── hello-linux-arm64
\n

You can set platform-split=false to merge files from all platforms together
\ninto same directory:

\n
$ buildctl build \\\n  --frontend dockerfile.v0 \\\n  --opt platform=linux/amd64,linux/arm64 \\\n  --output type=local,dest=./bin/release,platform-split=false\n\n$ tree ./bin\n./bin/\n└── release\n    ├── hello-linux-amd64\n    └── hello-linux-arm64
\n

Tar exporter is similar to local exporter but transfers the files through a tarball.

\n
buildctl build ... --output type=tar,dest=out.tar\nbuildctl build ... --output type=tar > out.tar
\n

Docker tarball

\n
# exported tarball is also compatible with OCI spec\nbuildctl build ... --output type=docker,name=myimage | docker load
\n

OCI tarball

\n
buildctl build ... --output type=oci,dest=path/to/output.tar\nbuildctl build ... --output type=oci > output.tar
\n

containerd image store

\n

The containerd worker needs to be used

\n
buildctl build ... --output type=image,name=docker.io/username/image\nctr --namespace=buildkit images ls
\n

To change the containerd namespace, you need to change worker.containerd.namespace in /etc/buildkit/buildkitd.toml.

\n

Cache

\n

To show local build cache (/var/lib/buildkit):

\n
buildctl du -v
\n

To prune local build cache:

\n
buildctl prune
\n

Garbage collection

\n

See ./docs/buildkitd.toml.md.

\n

Export cache

\n

BuildKit supports the following cache exporters:

\n\n

In most case you want to use the inline cache exporter.
\nHowever, note that the inline cache exporter only supports min cache mode.
\nTo enable max cache mode, push the image and the cache separately by using registry cache exporter.

\n

inline and registry exporters both store the cache in the registry. For importing the cache, type=registry is sufficient for both, as specifying the cache format is not necessary.

\n

Inline (push image and cache together)

\n
buildctl build ... \\\n  --output type=image,name=docker.io/username/image,push=true \\\n  --export-cache type=inline \\\n  --import-cache type=registry,ref=docker.io/username/image
\n

Note that the inline cache is not imported unless --import-cache type=registry,ref=... is provided.

\n

Inline cache embeds cache metadata into the image config. The layers in the image will be left untouched compared to the image with no cache information.

\n

ℹ️ Docker-integrated BuildKit (DOCKER_BUILDKIT=1 docker build) and docker buildxrequires
\n--build-arg BUILDKIT_INLINE_CACHE=1 to be specified to enable the inline cache exporter.
\nHowever, the standalone buildctl does NOT require --opt build-arg:BUILDKIT_INLINE_CACHE=1 and the build-arg is simply ignored.

\n

Registry (push image and cache separately)

\n
buildctl build ... \\\n  --output type=image,name=localhost:5000/myrepo:image,push=true \\\n  --export-cache type=registry,ref=localhost:5000/myrepo:buildcache \\\n  --import-cache type=registry,ref=localhost:5000/myrepo:buildcache
\n

--export-cache options:

\n\n

--import-cache options:

\n\n

Local directory

\n
buildctl build ... --export-cache type=local,dest=path/to/output-dir\nbuildctl build ... --import-cache type=local,src=path/to/input-dir
\n

The directory layout conforms to OCI Image Spec v1.0.

\n

--export-cache options:

\n\n

--import-cache options:

\n\n

GitHub Actions cache (experimental)

\n
buildctl build ... \\\n  --output type=image,name=docker.io/username/image,push=true \\\n  --export-cache type=gha \\\n  --import-cache type=gha
\n

GitHub Actions cache saves both cache metadata and layers to GitHub's Cache service. This cache currently has a size limit of 10GB that is shared across different caches in the repo. If you exceed this limit, GitHub will save your cache but will begin evicting caches until the total size is less than 10 GB. Recycling caches too often can result in slower runtimes overall.

\n

Similarly to using actions/cache, caches are scoped by branch, with the default and target branches being available to every branch.

\n

Following attributes are required to authenticate against the GitHub Actions Cache service API:

\n\n

ℹ️ This type of cache can be used with Docker Build Push Action
\nwhere url and token will be automatically set. To use this backend in an inline run step, you have to include crazy-max/ghaction-github-runtime
\nin your workflow to expose the runtime.

\n

--export-cache options:

\n\n

--import-cache options:

\n\n

S3 cache (experimental)

\n
buildctl build ... \\\n  --output type=image,name=docker.io/username/image,push=true \\\n  --export-cache type=s3,region=eu-west-1,bucket=my_bucket,name=my_image \\\n  --import-cache type=s3,region=eu-west-1,bucket=my_bucket,name=my_image
\n

The following attributes are required:

\n\n

Storage locations:

\n\n

S3 configuration:

\n\n

AWS Authentication:

\n

BuildKit relies on the [AWS Go SDK](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/config#EnvConfig). This means that all standard authentication methods through environment variables or config files are supported. This is especially true for AWS EC2 IAM Profile and AWS Web Identity Token (IAM roles in Kubernetes).

\n

Beware, these configurations must be available at buildkit daemon level, not at client level.

\n\n

--export-cache options:

\n\n

--import-cache options:

\n\n

Azure Blob Storage cache (experimental)

\n
buildctl build ... \\\n  --output type=image,name=docker.io/username/image,push=true \\\n  --export-cache type=azblob,account_url=https://myaccount.blob.core.windows.net,name=my_image \\\n  --import-cache type=azblob,account_url=https://myaccount.blob.core.windows.net,name=my_image
\n

The following attributes are required:

\n\n

Storage locations:

\n\n

Azure Blob Storage configuration:

\n\n

Azure Blob Storage authentication:

\n

There are 2 options supported for Azure Blob Storage authentication:

\n\n

Note

Account name can also be specified with account_name attribute (or $BUILDKIT_AZURE_STORAGE_ACCOUNT_NAME)
\nif it is not part of the account URL host.

\n
\n

--export-cache options:

\n\n

--import-cache options:

\n\n

Consistent hashing

\n

If you have multiple BuildKit daemon instances, but you don't want to use registry for sharing cache across the cluster,
\nconsider client-side load balancing using consistent hashing.

\n

See ./examples/kubernetes/consistenthash.

\n

Metadata

\n

To output build metadata such as the image digest, pass the --metadata-file flag.
\nThe metadata will be written as a JSON object to the specified file.
\nThe directory of the specified file must already exist and be writable.

\n
buildctl build ... --metadata-file metadata.json
\n
jq '.' metadata.json
\n
{\n  \"containerimage.config.digest\": \"sha256:2937f66a9722f7f4a2df583de2f8cb97fc9196059a410e7f00072fc918930e66\",\n  \"containerimage.descriptor\": {\n    \"annotations\": {\n      \"config.digest\": \"sha256:2937f66a9722f7f4a2df583de2f8cb97fc9196059a410e7f00072fc918930e66\",\n      \"org.opencontainers.image.created\": \"2022-02-08T21:28:03Z\"\n    },\n    \"digest\": \"sha256:19ffeab6f8bc9293ac2c3fdf94ebe28396254c993aea0b5a542cfb02e0883fa3\",\n    \"mediaType\": \"application/vnd.oci.image.manifest.v1+json\",\n    \"size\": 506\n  },\n  \"containerimage.digest\": \"sha256:19ffeab6f8bc9293ac2c3fdf94ebe28396254c993aea0b5a542cfb02e0883fa3\"\n}
\n

Systemd socket activation

\n

On Systemd based systems, you can communicate with the daemon via Systemd socket activation, use buildkitd --addr fd://.
\nYou can find examples of using Systemd socket activation with BuildKit and Systemd in ./examples/systemd.

\n

Expose BuildKit as a TCP service

\n

The buildkitd daemon can listen the gRPC API on a TCP socket.

\n

It is highly recommended to create TLS certificates for both the daemon and the client (mTLS).
\nEnabling TCP without mTLS is dangerous because the executor containers (aka Dockerfile RUN containers) can call BuildKit API as well.

\n
buildkitd \\\n  --addr tcp://0.0.0.0:1234 \\\n  --tlscacert /path/to/ca.pem \\\n  --tlscert /path/to/cert.pem \\\n  --tlskey /path/to/key.pem
\n
buildctl \\\n  --addr tcp://example.com:1234 \\\n  --tlscacert /path/to/ca.pem \\\n  --tlscert /path/to/clientcert.pem \\\n  --tlskey /path/to/clientkey.pem \\\n  build ...
\n

Load balancing

\n

buildctl build can be called against randomly load balanced buildkitd daemons.

\n

See also Consistent hashing for client-side load balancing.

\n

Containerizing BuildKit

\n

BuildKit can also be used by running the buildkitd daemon inside a Docker container and accessing it remotely.

\n

We provide the container images as moby/buildkit:

\n\n

To run daemon in a container:

\n
docker run -d --name buildkitd --privileged moby/buildkit:latest\nexport BUILDKIT_HOST=docker-container://buildkitd\nbuildctl build --help
\n

Podman

\n

To connect to a BuildKit daemon running in a Podman container, use podman-container:// instead of docker-container:// .

\n
podman run -d --name buildkitd --privileged moby/buildkit:latest\nbuildctl --addr=podman-container://buildkitd build --frontend dockerfile.v0 --local context=. --local dockerfile=. --output type=oci | podman load foo
\n

sudo is not required.

\n

Nerdctl

\n

To connect to a BuildKit daemon running in a Nerdctl container, use nerdctl-container:// instead of docker-container://.

\n
nerdctl run -d --name buildkitd --privileged moby/buildkit:latest\nbuildctl --addr=nerdctl-container://buildkitd build --frontend dockerfile.v0 --local context=. --local dockerfile=. --output type=oci | nerdctl load
\n

sudo is not required.

\n

Kubernetes

\n

For Kubernetes deployments, see examples/kubernetes.

\n

Daemonless

\n

To run the client and an ephemeral daemon in a single container (\"daemonless mode\"):

\n
docker run \\\n    -it \\\n    --rm \\\n    --privileged \\\n    -v /path/to/dir:/tmp/work \\\n    --entrypoint buildctl-daemonless.sh \\\n    moby/buildkit:master \\\n        build \\\n        --frontend dockerfile.v0 \\\n        --local context=/tmp/work \\\n        --local dockerfile=/tmp/work
\n

or

\n
docker run \\\n    -it \\\n    --rm \\\n    --security-opt seccomp=unconfined \\\n    --security-opt apparmor=unconfined \\\n    -e BUILDKITD_FLAGS=--oci-worker-no-process-sandbox \\\n    -v /path/to/dir:/tmp/work \\\n    --entrypoint buildctl-daemonless.sh \\\n    moby/buildkit:master-rootless \\\n        build \\\n        --frontend \\\n        dockerfile.v0 \\\n        --local context=/tmp/work \\\n        --local dockerfile=/tmp/work
\n

OpenTelemetry support

\n

BuildKit supports OpenTelemetry for buildkitd gRPC
\nAPI and buildctl commands. To capture the trace to
\nJaeger, set JAEGER_TRACE
\nenvironment variable to the collection address.

\n
docker run -d -p6831:6831/udp -p16686:16686 jaegertracing/all-in-one:latest\nexport JAEGER_TRACE=0.0.0.0:6831\n# restart buildkitd and buildctl so they know JAEGER_TRACE\n# any buildctl command should be traced to http://127.0.0.1:16686/
\n
\n

On Windows, if you are running Jaeger outside of a container, jaeger-all-in-one.exe,
\nset the environment variable setx -m JAEGER_TRACE \"0.0.0.0:6831\",
\nrestart buildkitd in a new terminal and the traces will be collected automatically.

\n
\n

Running BuildKit without root privileges

\n

Please refer to docs/rootless.md.

\n

Building multi-platform images

\n

Please refer to docs/multi-platform.md.

\n

Configuring buildctl

\n

Color Output Controls

\n

buildctl has support for modifying the colors that are used to output information to the terminal. You can set the environment variable BUILDKIT_COLORS to something like run=green:warning=yellow:error=red:cancel=255,165,0 to set the colors that you would like to use. Setting NO_COLOR to anything will disable any colorized output as recommended by no-color.org.

\n

Parsing errors will be reported but ignored. This will result in default color values being used where needed.

\n\n

Number of log lines (for active steps in tty mode)

\n

You can change how many log lines are visible for active steps in tty mode by setting BUILDKIT_TTY_LOG_LINES to a number (default: 6).

\n

Contributing

\n

Want to contribute to BuildKit? Awesome! You can find information about contributing to this project in the CONTRIBUTING.md

","updatedAt":"2025-05-21T15:50:44Z","upvoteCount":null,"author":{"login":"eryajf","avatarUrl":"https://avatars.githubusercontent.com/u/33259379?u=e4a4090a38ac2473aaed4ef9945233636776c6c3&v=4","url":"https://github.com/eryajf"},"category":null,"labels":{"edges":[{"node":{"name":"更多","color":"25B472"}},{"node":{"name":"Docker-Tools","color":"abbcfb"}},{"node":{"name":"moby","color":"0791ef"}}]},"comments":{"edges":[]}}},"pageContext":{"number":58,"previous":{"title":"aristocratos/btop: 系统资源监视器显示处理器、内存、磁盘、网络和进程的使用情况和统计​​信息","number":57},"next":{"title":"TeaWeb/build: TeaWeb 是一款集静态资源、缓存、代理、统计、日志、安全、监控于一体的可视化智能 WebServer。","number":59}}}, "staticQueryHashes": ["151096407","2861350382"]}