Problem with tar command when using aldoc on self-hosted Windows runner

[rbability]

I wonder if someone figured out a good solution for that.

When activating aldoc in a repo, we run into an issue in the "Deploy Reference Documentation" step "Upload pages artifact" when the tar command is invoked. The tar command invoked uses the --hard-dereference parameter and throws an error on our self-hosted runners because that option is not supported in tar for windows (at least the one that ships with Server 2022).

Windows ships with bsdtar by default and it just does not seem to have this option. GNU tar however has. Now the fun part is to find a recent GNU tar version for windows. When installing Cygwin, you can get your hands on the current GNU tar version for Windows. But since the bsdtar version that ships with windows sits in the system32 folder, it gets used first by whomever is just running "tar".

Replacing the tar.exe in Windows with the gnutar version might be a fix, but I am sure after the next Windows update, it will get replaced. And tinkering with Windows default executables is not a nice practice anyway.

I am pretty sure the Action works in the GitHub hosted runners and they include GNU tar somehow, but I wonder if anyone else is struggling with this on the self-hosted runners and found a good solution around it.

Here´s the relevant output from the Action:

Run tar
tar
--dereference --hard-dereference
--directory "$INPUT_PATH"
-cvf "$RUNNER_TEMP\artifact.tar"
--exclude=.git
--exclude=.github
--force-local
"."
[...]
tar.exe: Option --hard-dereference is not supported
Usage:
List: tar.exe -tf
Extract: tar.exe -xf
Create: tar.exe -cf [filenames...]
Help: tar.exe --help
Error: Process completed with exit code 1.

[freddydk]

Software installed on GitHub runners is this: https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md - cannot find gnutar there - but I will setup some tests with self-hosted runners and figure out what to do.

[rbability]

Thank you very much @freddydk. I went through the Readme too, even looked at the JSON in their Repo, just in case and I can not find an explicit GNU tar installation either. But there is hardly any to find in the Internet anyway, mostly bundled with other stuff.
In their MacOS runners they specifically include GNU + BSD tar, but they are entirely different anyway.

If I call up "tar --version" on a fresh Windows Server 2022 it shows that it includes bsdtar and it does not know --hard-dereference.

tar --version
bsdtar 3.5.2 - libarchive 3.5.2 zlib/1.2.5.f-ipp

and

tar --hard-dereference
tar.exe: Option --hard-dereference is not supported
Usage:
List: tar.exe -tf
Extract: tar.exe -xf
Create: tar.exe -cf [filenames...]
Help: tar.exe --help

So it is at least the same message I have during a run.

I was just thinking about creating a simple action that runs a tar --version on a GitHub runner... will try to do this and see what the output is.

[freddydk]

The upload action is here: https://github.com/actions/upload-pages-artifact
In the code it even has a special check for os' - and for MacOS, they use gtar:
https://github.com/actions/upload-pages-artifact/blob/main/action.yml

You could create an issue there and ask how people are supposed to use that action with self-hosted windows runners - other people might be in the same situation.

[rbability]

Oh, I see! You use an existing Action, it´s not all custom code. I was about to track down what your Action exactly does, because I just did a test with a GitHub runner and they do include by default the bsdtar:

Thanks for pointing me to the Action you are using. I will check that one and do as you suggest. Maybe they use Bash instead of Powershell, which might change things too. If you don't mind I will keep this open for a bit. If I find a solution, I will post it here, so others with Self-Hosted runners might find this one here. I´ll try to look into this deeper tomorrow if time allows.

[freddydk]

actions/upload-pages-artifact#17

[freddydk]

They use bash

[freddydk]

AL-Go uses our own actions + github+microsoft actions under actions only.

[rbability]

Yeah, I´ve just seen that Bash is included in the GitHub runners. I will update one of our Runners and see how it goes. There´s nothing you can do of course, so as said before: If this can stay open for a bit I will try to report back when I found a solution, but of course I´m not mad if you close this one to keep things tidy around here.
Thanks again for your help :-)

[freddydk]

I think this is the perfect way of discussing and resolving issues, so thanks a million.

[freddydk]

If bash solves the issue - I will add that to the documentation for self-hosted runners.

[rbability]

This took me far too long and going through a lot of stuff with cygwin, WSL and what not. And the fix is so easy, I just did not see it🙈. It was all there already, no additional Software is needed, just an extension to the Systems PATH variable.

How to fix the error:

1. Make sure you have Git (Git-SCM) installed, which you of course have on every Self-Hosted runner anyway.
2. Extend the System PATH variable with "C:\Program Files\Git\bin".
3. Make sure it sits above the "C:\Program Files\Git\usr\bin" path
4. If you have WSL installed, make sure it also sits above the "C:\Windows\System32" path.
5. Restart the System / your Runners

I described more details in the Issue over here if this is of interest.

To sum it up: The change above makes sure that the "bash" command starts the one in the Path you add to the PATH variable, because this is the one that uses GNU tar when running "tar". The other one that comes with Git by default for some reason starts bsdtar, but is also "fixed" when the PATH is set as mentioned above. And if you happen to have WSL installed, you will run into a myriad of other issues because it does not work with LocalSystem Accounts (GH Runners default) and even if you change that, you get other issues that are not worth it to solve with the easy fix mentioned above.

We are not using all of the AL-Go Actions on our Self-Hosted runners, but with the Software setup we have, we use the following Actions without any issues regularly:

• CI/CD
• Create release
• Deploy Reference Documentation
• Increment Version Number
• Publish To AppSource
• Publish To Environment
• Test Current
• Test Next Major
• Test Next Minor
• Update AL-Go System Files

Our Runners have the following Software installed at the moment:

• Windows Server 2022
• Docker CE Runtime 25.0.2
• 7-Zip
• Git
• GitHub CLI
• .NET Runtime 7 x64 and x86**
• .NET SDK 6 x64**
• .NET SDK 7 x64**
• Microsoft Azure CLI
• Visual C++ 2013 Redistributable 2013 x64
• Visual C++ 2015-2022 Redistributable x64 and x86
• PowerShell 7.x x64

** With these I am not sure if all of them are still needed. When we started to CodeSign our packages and when there were changes in AL-Go later on how the signing worked, I think we had to add one version or another to keep it working. It might be possible that you do not need the SDK6 version anymore, so if you start fresh, maybe skip the older version and try with 7 first.

Edit: Fixed ** shown as ** instead of a bulletpoint

[freddydk]

@rbability thanks a million for your investigations and this thorough description.
Will check and ensure that the https://aka.ms/getbuildagent ARM template has the needed stuff.