Provide alternative ways to building Docker image with and without the daemon

[kneal]

Description

Users of Vela are looking for more ways to build images. Today, Vela provides a Kaniko plugin as the main way to build a Docker image. This at times builds images that are not expected results the customer is getting with the Docker CLI.

Value

Having more options can allow developers to pick from an Array of options that best fits their needs for building a Docker image.

Definition of Done

The team narrowed down a list of options to build images with and without the daemon:

• img - Standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder.
• Makisu - build tool designed for unprivileged containerized environments such as Mesos or Kubernetes.
• Docker with Buildkit - Docker native CLI with buildki enabled

Impacted Personas (Optional)

Vela users using the Docker plugin

[kneal]

First change was to rename vela-docker to vela-kaniko:
https://github.com/go-vela/vela-kaniko

All images in Docker Hub were moved to the new location:
https://hub.docker.com/r/target/vela-kaniko

[kneal]

Repos have been created for the following tools:

• img - https://github.com/go-vela/vela-img
• makisu - https://github.com/go-vela/vela-makisu
• docker - https://github.com/go-vela/vela-docker

[kneal]

Repositories have been updated to include a plugin skeleton:

go-vela/vela-img#1
go-vela/vela-makisu#1
go-vela/vela-docker#1

[kneal]

Draft PR available for using img to build Docker images:
go-vela/vela-img#2

[kneal]

Having some issues with img flags that have been reported to the upstream repo:
genuinetools/img#301

[kneal]

Overall, this summary is in the spirit of Vela will have two core ways to build Docker.

1. Allowing Socket access to use the native Docker CLI
2. A secure method of not mounting a socket or using privilege. (Required using a plugin designed for this method)

After working with img I've concluded the following:

• I foresee issues running it within Vela since it requires custom security opts on the container. The workaround is just to run as privileged if you're in a platform that does not have the option to add those flags to the container. This presents a problem in Vela because we do not have a way to provide custom security controls to containers at this time.

• There appear to be inconsistencies with using flags with CLI commands. The behavior that occurs is flags can be used with aliases but not with the explicit flag name. There is a chance the os/exec Go library is affecting how the commands are being executed but at this time research has not been done to identify potential bugs in that library that are affecting flags being used with img commands. The behavior does appear consistent across commands (build, login, etc)

• The Docker config was a sore point on the CLI. The documentation recommends mounting the config into /root/.docker/config.json I tried multiple methods of writing the config at build time in Docker and using our afero library standard for writing configs and was unable to find a consistent method to make a config available without mounting. Reading up on the img login command I was able to identify a way to login which adds credentials to a config file that then can be used with img build commands. However, there appear to be registries that are not compatible with the login method the CLI is using.

My conclusion at this time, is I think we should merge go-vela/vela-img#2 then put a notice on the repository that it's not a valid plugin at this time until the img is a bit more mature or we decide to investigate the following bugs further.

[kneal]

The makisu plugin is ready for review and merge:

feat: add image build action
go-vela/vela-makisu#2

[kneal]

As a part of this effort, I also added a new --snapshot-mode flag which allows uses to tune their performance with Kaniko for increased build time. Details can be found in PR @ go-vela/vela-kaniko#56

[kneal]

Adding/Updating docs with information on how to build Vela without elevated daemon access:
go-vela/docs#174

[kneal]

Implementation for the plugin using Docker CLI:
go-vela/vela-docker#2

[kneal]

RFR: go-vela/vela-docker#2

[kneal]

Add open container and vela standard labels to kaniko:
go-vela/vela-kaniko#60

Makisu will need to wait on labels until a feature has been added to allow labels to be added during makisu build
uber-archive/makisu#348

[kneal]

This PR on the runtime:
go-vela/pkg-runtime#62

adds the ability for workers to run a docker image in privileged mode. That flag will be required to be set on a worker in order for the vela-docker to work since it uses a dind (Docker in Docker) pattern for building image.

[kneal]

Add plugin documentation for Docker plugin:
go-vela/vela-docker#4
go-vela/docs#177