Consider adding build artifact support for CI or provide easy process for QA to build XPIs locally

[biancadanforth]

If we can keep QA testing inside of GitHub and provide a single location where QA can find the latest build, that'd save us a lot of back and forth in Bugzilla/e-mail.

Option 1: Develop an easy process for QA to build the XPI from source

From Paul Oiegas of SoftVision:

Manually building XPI's for studies and experiments is not a problem for us if we have the right steps. We are already doing daily builds on Linux or Mac OS for all the Test Pilot experiments, which are also in GitHub. I was doing daily local builds for this study too from master by following the steps from README.md file.

Option 1 is perfect for us if the flow is working without errors. Just let us know in what PR you did the change, we will build from it and report back after testing.

Option 2: Add build artifact support for CI

From @pdehaan

https://github.com/bwinton/SnoozeTabs/blob/master/circle.yml shows how we could copy any generated XPI files files into the $CIRCLE_ARTIFACTS directory during CI.
Then, you can download those XPIs (addon-dev.xpi and addon.xpi) from Circle's artifacts tab on https://circleci.com/gh/bwinton/SnoozeTabs/669#artifacts/containers/0 (not sure how permissions work and if everybody can see the Artifacts tab, or if you have to be a collaborator on the repo). If I try viewing that page on a non logged in profile, I don't see that tab -- but I did after logging in to Circle-CI w/ GitHub.

[pdehaan]

Also, this was probably the more relevant piece:
https://github.com/bwinton/SnoozeTabs/blob/870727f3e0279b14cb496b0546a595e0db81ea31/circle.yml#L5-L8

Note how we're copying over any generated *.xpi assets into the $CIRCLE_ARTIFACTS directory on Circle-CI.

More details on the details at https://circleci.com/blog/build-artifacts/ and https://circleci.com/docs/2.0/artifacts/ (the latter uses the newer, preferred version 2 of the Circle API).

[biancadanforth]

Update: @jrbenny35 has kindly offered to help us with this. The current goal would be to automatically generate a build artifact (XPI) per commit via Circle CI that would be available in each open PR for QA to download and test.

[b4handjr]

@pdehaan is this already happening here: https://github.com/mozilla/shield-studies-addon-template/blob/master/.circleci/config.yml#L51 ?

[pdehaan]

Yeah, it looks like we might be...

shield-studies-addon-template/.circleci/config.yml

Lines 51 to 52 in af8ed48

	- store_artifacts:
	path: "dist"

Where we're calling npm run build on L43-45 above, and the package.json "build" script is:

shield-studies-addon-template/package.json

Line 67 in af8ed48

"build": "bash ./bin/xpi.sh",

... and ./bin/xpi.sh appears to copy everything into aforementioned "./dist/" directory (whew!):

shield-studies-addon-template/bin/xpi.sh

Lines 31 to 36 in af8ed48

	# xpi all of 'addon' to 'dist'
	mkdir -p dist
	pushd addon > /dev/null
	zip -r "../dist/${XPI_NAME}" .
	popd > /dev/null

[pdehaan]

All that said, I have to find where to get the artifacts on Circle-CI. If I click the Circle-CI badge in the README, it seems to open a new tab with the badge image (not a link to the Circle-CI builds). And I don't seem to see the Circle-CI results on any PRs. Not sure if that is because I don't have write access to this repo, or something else...

Although, if I manually navigate to Circle, I can see some artifacts in the latest build!

https://circleci.com/gh/mozilla/shield-studies-addon-template/44#artifacts/containers/0

[b4handjr]

@biancadanforth Does peter's answer help a bit? I am still willing to help in anyway I can.

[biancadanforth]

@jrbenny35 Thanks for checking in.

I know that we currently have a bash script to build the XPI in our local working directory, but I don't quite understand how we go from there to having it (or a URL like the page pdehaan linked above) made available in the GitHub PR itself? It sounds like from pdehaan that his method of finding that URL was pretty convoluted.

Is there a way to include a link to the right URL for downloading the correct XPI in the PR itself automatically?

[b4handjr]

Is there a way to include a link to the right URL for downloading the correct XPI in the PR itself automatically?

Hmm, I know testpilot has a storybook bot that comments on the PR with a link to the PR deployed. Maybe we could find something like that.

[motin]

Closing since this was added with #73