Merge branch 'master' into v2

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,30 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: "[BUG]"
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Call '....' (make it reproducable by eventually providing the chart the problem occured with)
+3. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**khelm version:**
+The khelm version you used.
+Please copy the first line of the khelm log when templating or the output of `khelm version` (make sure you use the binary the problem occured with) here.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[FEAT]"
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,10 @@
+---
+name: Question
+about: Raise a question
+title: "[QUESTION]"
+labels: ''
+assignees: ''
+
+---
+
+Please provide details and relevant context to your question here.

.gitignore

@@ -10,3 +10,4 @@
 /example/kpt/**/requirements.lock
 /example/**/generated-manifest-without-secrets.yaml
 /example/kpt/linkerd/dep
+/example/kpt/cache-dependencies/chart/charts

CODE_OF_CONDUCT.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at [email protected]. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

CONTRIBUTING.md

@@ -0,0 +1,47 @@
+# How to contribute
+
+Contributions, questions and feedback are welcome.
+There are just a few small guidelines you need to follow.
+
+## Community guidelines
+
+This project follows the [Contributor Covenant Code of Conduct](./CODE_OF_CONDUCT.md).
+
+## Creating issues
+
+Feedback, questions, bug reports and feature requests are welcome in form of issues.
+Though before creating an issue please have a look [here](https://github.com/mgoltzsche/khelm/issues) if the issue doesn't exist already in order to avoid duplicates.
+When you cannot find a corresponding issue please create a [new one](https://github.com/mgoltzsche/khelm/issues/new/choose).
+
+### Bug reports
+
+When reporting a bug please answer the following questions:
+* Which environment and version do you use? Does it happen with the latest version as well?
+* What did you do and what happened? (in a reproducible way ideally)
+* What do you expect instead?
+
+### Feature requests
+
+When creating a feature request please answer the following questions:
+* What is the problem?
+* What do you want to change?
+* How do you want it to behave?
+* Why do you need it?
+* Which alternatives did you consider?
+
+## Creating pull requests (PRs)
+
+While there is no need to create an issue before creating a PR please consider discussing large changes within an issue first.
+
+When creating a PR please follow these rules:
+* Create one PR per feature.
+* Provide one commit per PR.
+* Provide a description with each PR and commit that links an issue (if any), describes the problem, how it was solved and how the solution changes the user-facing behaviour.
+* All submissions require a review.
+* All commits should be signed using PGP and a [DCO](https://en.wikipedia.org/wiki/Developer_Certificate_of_Origin). This can be done using `git commit -s -S`.
+* When submitting a bug fix or feature PR please also provide tests.
+* When submitting a new feature please document it.
+
+## Building and testing the source
+
+See [here](./README.md#build-and-test).

Makefile

@@ -62,6 +62,7 @@ e2e-test: image khelm-static kpt kustomize
 	./e2e/kustomize-plugin-test.sh; \
 	IMAGE=$(IMAGE) ./e2e/image-cli-test.sh; \
 	./e2e/kpt-function-test.sh; \
+	./e2e/kpt-cache-test.sh; \
 	./e2e/kpt-cert-manager-test.sh; \
 	./e2e/kpt-linkerd-test.sh; \
 	}

README.md

@@ -87,6 +87,10 @@ If necessary the chart output can be transformed using kustomize.
 This can be done by declaring the khelm and a kustomize function orderly within a file and specifying the chart output kustomization as input for the kustomize function as shown in the [cert-manager example](example/kpt/cert-manager).
 A more complex example that also manages a Helm chart from another git repository locally as kpt dependency can be found [here](example/kpt/linkerd).
 
+#### Caching Helm Charts and repository index files
+
+When external Helm Charts are used the download of their repositories' index files and of the charts itself can take a significant amount of time that adds up when running multiple functions or calling a function frequently during development.  
+To speed this up caching can be enabled by mounting a host directory to `/helm` within the function container as shown [here](example/kpt/cache-dependencies).
 
 ### kustomize exec plugin
 
@@ -200,7 +204,9 @@ It exposes a `Helm` struct that provides a `Render()` function that returns the
 
 ### Repository configuration
 
-Repository credentials can be configured using helm's `repositories.yaml` which can be passed through as `Secret` to generic build jobs. khelm downloads repo index files when needed.  
+Repository credentials can be configured using Helm's `repositories.yaml` which can be passed through as `Secret` to generic build jobs. khelm downloads the corresponding repo index files when needed.  
+
+When running khelm as kpt function or within a container the `repositories.yaml` should be mounted to `/helm/repository/repositories.yaml`.  
 
 Unlike Helm khelm allows usage of any repository when `repositories.yaml` is not present or `--trust-any-repo` is enabled.
 

cmd/khelm/fn_test.go

@@ -246,6 +246,7 @@ func TestKptFnCommand(t *testing.T) {
 				"\n    config.kubernetes.io/path: my/output/path/kustomization.yaml\n",
 				"\n    config.kubernetes.io/path: my/output/path/serviceaccount_sa1.yaml\n",
 				"\n    config.kubernetes.io/path: my/output/path/serviceaccount_sa2.yaml\n",
+				" myannotation: should-be-preserved\n",
 			},
 		},
 		{

e2e/kpt-cache-test.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+
+cd "$(dirname "$0")/../example"
+
+echo
+echo "  TEST $0: Run kpt functions of example/kpt/cache-dependencies"
+echo
+
+set -e
+
+cd kpt/cache-dependencies
+make clean
+
+(
+	set -ex
+	make manifest
+	[ -f generated-manifests/manifest1.yaml ]
+	[ -f generated-manifests/manifest1-from-cache.yaml ]
+	[ -f generated-manifests/manifest2.yaml ]
+	[ -f generated-manifests/manifest2-from-cache.yaml ]
+)
+
+echo SUCCESS

example/empty-annotations/templates/template.yaml

@@ -2,6 +2,8 @@ apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: sa1
+  annotations:
+    myannotation: should-be-preserved
 ---
 apiVersion: v1
 kind: ServiceAccount

example/kpt/cache-dependencies/Kptfile

@@ -0,0 +1,6 @@
+apiVersion: kpt.dev/v1alpha1
+kind: Kptfile
+metadata:
+  name: cache-dependencies
+packageMetadata:
+  shortDescription: An caching example

example/kpt/cache-dependencies/Makefile

@@ -0,0 +1,11 @@
+all: manifest
+
+manifest:
+	mkdir -p $$HOME/.helm
+	kpt fn run --as-current-user --network \
+		--mount "type=bind,src=$$HOME/.helm,dst=/helm,rw=true" \
+		--mount "type=bind,src=`pwd`,dst=/source,rw=true" \
+		.
+
+clean:
+	rm -rf chart/charts chart/requirements.lock generated-manifests

example/kpt/cache-dependencies/README.md

@@ -0,0 +1,5 @@
+# kpt function example with caching
+
+This example is used as e2e test and shows that Helm dependencies can be cached.  
+
+Caching can be enabled by mounting a host directory to `/helm` within the container (see `Makefile`).

example/kpt/cache-dependencies/chart/Chart.yaml

@@ -0,0 +1,4 @@
+apiVersion: v1
+description: example chart
+name: example
+version: 0.0.1

example/kpt/cache-dependencies/chart/requirements.yaml

@@ -0,0 +1,4 @@
+dependencies:
+- name: cert-manager
+  version: 1.0.2
+  repository: https://charts.jetstack.io

example/kpt/cache-dependencies/functions-local-chart-dependencies.yaml

@@ -0,0 +1,31 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: local-chart-deps-uncached
+  annotations:
+    config.kubernetes.io/function: |
+      container:
+        image: mgoltzsche/khelm:latest
+        network: true
+    config.kubernetes.io/local-config: "true"
+data:
+  chart: ./source/chart
+  name: cert-manager
+  namespace: cert-manager
+  outputPath: generated-manifests/manifest2.yaml
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: local-chart-deps-cached
+  annotations:
+    config.kubernetes.io/function: |
+      container:
+        image: mgoltzsche/khelm:latest
+        network: false
+    config.kubernetes.io/local-config: "true"
+data:
+  chart: ./source/chart
+  name: cert-manager
+  namespace: cert-manager
+  outputPath: generated-manifests/manifest2-from-cache.yaml

example/kpt/cache-dependencies/functions-remote-chart.yaml

@@ -0,0 +1,35 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: remote-chart-uncached
+  annotations:
+    config.kubernetes.io/function: |
+      container:
+        image: mgoltzsche/khelm:latest
+        network: true
+    config.kubernetes.io/local-config: "true"
+data:
+  repository: https://charts.jetstack.io
+  chart: cert-manager
+  version: 1.1.0
+  name: cert-manager
+  namespace: cert-manager
+  outputPath: generated-manifests/manifest1.yaml
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: remote-chart-cached
+  annotations:
+    config.kubernetes.io/function: |
+      container:
+        image: mgoltzsche/khelm:latest
+        network: false
+    config.kubernetes.io/local-config: "true"
+data:
+  repository: https://charts.jetstack.io
+  chart: cert-manager
+  version: 1.1.0
+  name: cert-manager
+  namespace: cert-manager
+  outputPath: generated-manifests/manifest1-from-cache.yaml

pkg/helm/locate.go

@@ -54,6 +54,10 @@ func locateChart(ctx context.Context, cfg *config.LoaderConfig, repos repository
 	}
 
 	if _, err = os.Stat(cacheFile); err == nil {
+		cacheFile, err = filepath.EvalSymlinks(cacheFile)
+		if err != nil {
+			return "", errors.Wrap(err, "normalize cached file path")
+		}
 		if cfg.Verify {
 			if _, err := downloader.VerifyChart(cacheFile, cfg.Keyring); err != nil {
 				return "", err

pkg/helm/repositories.go

@@ -90,7 +90,7 @@ type repositories struct {
 }
 
 func (f *repositories) RequireTempHelmHome(createTemp bool) {
-	f.entriesAdded = createTemp
+	f.entriesAdded = f.entriesAdded || createTemp
 }
 
 func newRepositories(settings *cli.EnvSettings, getters getter.Providers) (r *repositories, err error) {