chore: added hack/verify-reports.sh script #3167

mlavacca · 2024-06-19T10:55:55Z

What type of PR is this?

/area conformance

What this PR does / why we need it:

Recently, many submitted reports were not compliant with the rules stated in this document. This PR aims to create a script to automatically check the correctness of the uploaded reports and the structure. The following are the checks performed:

a README file exists in the project conformance page
such a README does not contain broken links
the reports are named according to the pattern <experimental | standard>-<semver>-<mode>-report.yaml
~~the project version is a valid semver~~
the GatewayAPIVersion field matches the folder in which the report has been created
the GatewayAPIChannel is either "standard" or "experimental"

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

k8s-ci-robot · 2024-06-19T10:55:57Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

mlavacca · 2024-06-19T10:57:01Z

/test pull-gateway-api-verify

mlavacca · 2024-06-19T11:21:33Z

/test pull-gateway-api-verify

mlavacca · 2024-06-19T13:19:21Z

The CI job is intended to fail until #3168 does not get merged

robscott · 2024-06-20T15:39:55Z

The CI job is intended to fail until #3168 does not get merged

That PR merged, this one may also need a rebase, we'll see.

/retest

robscott

Thanks @mlavacca!

hack/verify-reports.sh

robscott · 2024-06-20T19:00:32Z

hack/verify-reports.sh

+                # Check if the README.md has broken links
+                    docker run -v $(readlink -f "$implementation_dir"):/${implementation}:ro --rm -i ghcr.io/tcort/markdown-link-check:stable /${implementation}/README.md


How does this utility work? Can/should we use this for all of our docs? I think some of our links are only valid with mkdocs-material context, would that be able to understand this?

It checks that all the links (internal and external) in a .md file are not broken. for what concerns mkdocs-material, all the internal links should already be checked when building the documentation. For what concerns the internal links of the documentation generated by mkdocs, I checked and this tool cannot be used, because as you pointed out, it is not able to properly resolve them. We could use such a tool for all the other .md files.

I've tried to run find . -path ./site-src -prune -o -name '*.md' -print0 | xargs -0 -n1 markdown-link-check -q (checks all the .md files but the ones under ./site-src) and many links are broken. I think this is out of scope for this PR, would you agree with me opening an issue to improve the verify.sh script to fix them?

all the internal links should already be checked when building the documentation

Not that I know of?

I think this is out of scope for this PR, would you agree with me opening an issue to improve the verify.sh script to fix them?

Agree it's out of scope for this PR, but a follow up would be great.

👍 I've created #3258 for this. For what concerns internal links with mkdocs, I asked confirmation directly to the community: https://matrix.to/#/!agmMfyRRWLhOnjmeFl:gitter.im/$sc_nZQIzGCVAu3hLKj0hzLLCeroYTEbn2Wfc12kUfGU?via=gitter.im&via=matrix.org

robscott

Thanks @mlavacca!

hack/verify-reports.sh

robscott · 2024-07-29T18:50:54Z

hack/verify-reports.sh

+                # Check if the README.md has broken links
+                    docker run -v $(readlink -f "$implementation_dir"):/${implementation}:ro --rm -i ghcr.io/tcort/markdown-link-check:stable /${implementation}/README.md


all the internal links should already be checked when building the documentation

Not that I know of?

I think this is out of scope for this PR, would you agree with me opening an issue to improve the verify.sh script to fix them?

Agree it's out of scope for this PR, but a follow up would be great.

robscott · 2024-07-29T18:51:18Z

hack/verify-reports.sh

@@ -0,0 +1,109 @@
+#!/bin/bash


Is this intended to be run automatically as a presubmit?

It's intended to be run as part of the make verify target. Some magic hack in verify-all.sh runs all the scripts matching the pattern verify-*.sh

Signed-off-by: Mattia Lavacca <[email protected]>

k8s-ci-robot · 2024-08-06T10:24:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mlavacca

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mlavacca]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

youngnick · 2024-08-09T04:12:35Z

@robscott, have you got any more thoughts here? What's our path forward for this one?

robscott · 2024-08-09T16:57:58Z

Thanks for the ping on this @youngnick and for all the work @mlavacca! I think this is good to go.

/lgtm

jgao1025 · 2024-08-12T04:58:47Z

hack/verify-reports.sh

+
+                if [[ -f "${implementation_dir}/README.md" ]]; then
+                # Check if the README.md has broken links
+                    docker run -v $(readlink -f "$implementation_dir"):/${implementation}:ro --rm -i ghcr.io/tcort/markdown-link-check:stable /${implementation}/README.md


So installing docker has become a dev prerequisite of this project? Otherwise we can't manfully run make verify on my local machine before submitting a PR?

It was already a prerequisite, as other verify-*.sh scripts make use of it (https://github.com/kubernetes-sigs/gateway-api/tree/main/hack/verify-docker-build.sh#L25-L26, https://github.com/kubernetes-sigs/gateway-api/blob/main/hack/verify-golint.sh#L33-L41). That said, I 100% agree we should clearly state it in the verify section of our docs. Given the PR you already raised on that side, would you be interested in updating such a section as well with all the deps? That would be great and much appreciated.

It was already a prerequisite, as other verify-*.sh scripts make use of it (https://github.com/kubernetes-sigs/gateway-api/tree/main/hack/verify-docker-build.sh#L25-L26, https://github.com/kubernetes-sigs/gateway-api/blob/main/hack/verify-golint.sh#L33-L41). That said, I 100% agree we should clearly state it in the verify section of our docs. Given the PR you already raised on that side, would you be interested in updating such a section as well with all the deps? That would be great and much appreciated.

Unfortunately my PR has just been merged, so I can't just change it in my old PR. But I have created a new one (#3267 ). I added background and steps on how to update the doc. It's should be pretty easy and straightforward to fix.

Thanks for creating such an issue, @jgao1025!

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. area/conformance labels Jun 19, 2024

k8s-ci-robot requested review from michaelbeaumont and robscott June 19, 2024 10:56

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 19, 2024

mlavacca force-pushed the verify_reports branch from 779b372 to 514f748 Compare June 19, 2024 11:21

mlavacca force-pushed the verify_reports branch 3 times, most recently from 982dcaa to 5bbaf47 Compare June 19, 2024 12:16

mlavacca marked this pull request as ready for review June 19, 2024 12:31

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 19, 2024

k8s-ci-robot requested a review from youngnick June 19, 2024 12:31

mlavacca force-pushed the verify_reports branch 3 times, most recently from 2a10efc to b64f6cc Compare June 19, 2024 13:01

robscott reviewed Jun 20, 2024

View reviewed changes

robscott reviewed Jul 29, 2024

View reviewed changes

mlavacca added 2 commits August 6, 2024 12:22

chore: hack/verify-reports added

99340eb

Signed-off-by: Mattia Lavacca <[email protected]>

remove version check

cfd2bd0

Signed-off-by: Mattia Lavacca <[email protected]>

mlavacca force-pushed the verify_reports branch from b64f6cc to cfd2bd0 Compare August 6, 2024 10:24

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 6, 2024

k8s-ci-robot assigned robscott Aug 9, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 9, 2024

k8s-ci-robot merged commit 1661a89 into kubernetes-sigs:main Aug 9, 2024
7 of 8 checks passed

jgao1025 reviewed Aug 13, 2024

View reviewed changes

jgao1025 mentioned this pull request Aug 13, 2024

Developer Guide needs to mention that Docker is one of the prerequisite. #3267

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: added hack/verify-reports.sh script #3167

chore: added hack/verify-reports.sh script #3167

mlavacca commented Jun 19, 2024 •

edited

Loading

k8s-ci-robot commented Jun 19, 2024

mlavacca commented Jun 19, 2024

mlavacca commented Jun 19, 2024

mlavacca commented Jun 19, 2024

robscott commented Jun 20, 2024

robscott left a comment

robscott Jun 20, 2024

mlavacca Jul 4, 2024

robscott Jul 29, 2024

mlavacca Aug 9, 2024

robscott left a comment

robscott Jul 29, 2024

robscott Jul 29, 2024

mlavacca Aug 6, 2024

k8s-ci-robot commented Aug 6, 2024

youngnick commented Aug 9, 2024

robscott commented Aug 9, 2024

jgao1025 Aug 12, 2024

mlavacca Aug 13, 2024 •

edited

Loading

jgao1025 Aug 13, 2024 •

edited

Loading

mlavacca Aug 13, 2024

		# Check if the README.md has broken links
		docker run -v $(readlink -f "$implementation_dir"):/${implementation}:ro --rm -i ghcr.io/tcort/markdown-link-check:stable /${implementation}/README.md

chore: added hack/verify-reports.sh script #3167

chore: added hack/verify-reports.sh script #3167

Conversation

mlavacca commented Jun 19, 2024 • edited Loading

k8s-ci-robot commented Jun 19, 2024

mlavacca commented Jun 19, 2024

mlavacca commented Jun 19, 2024

mlavacca commented Jun 19, 2024

robscott commented Jun 20, 2024

robscott left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robscott left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Aug 6, 2024

youngnick commented Aug 9, 2024

robscott commented Aug 9, 2024

Choose a reason for hiding this comment

mlavacca Aug 13, 2024 • edited Loading

Choose a reason for hiding this comment

jgao1025 Aug 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mlavacca commented Jun 19, 2024 •

edited

Loading

mlavacca Aug 13, 2024 •

edited

Loading

jgao1025 Aug 13, 2024 •

edited

Loading