Update build system for Linux, macOS, overhaul GitHub Actions, remove serial driver

[garrettsummerfi3ld]

This PR introduces a number of changes to the build system and GitHub Actions workflows, as well as removing the unused serial driver.

• Overhauled GitHub Actions workflows to include Linux, macOS platforms, alongside concurrency for builds and split larger steps into smaller steps for legibility and debugging of failing actions.
  • New runners include X86, ARM32, and ARM64 Linux runners, macOS runners compiling universal binaries alongside the existing Windows runner. ARM runners are using a WPI provided Docker container to build against.
  • Added special rulesets for workflow concurrency for jobs running in sync and queueing builds for branches.
  • Added special rulesets for allowing all branches to run actions instead of only main and PRs to main for testing builds on multiple platforms before a PR.
  • Split apart larger tasks into smaller tasks for better verbosity during workflow runs.
  • Updated action dependencies to latest as older actions are deprecated due to old Node.js runtimes.
  • Restructure how artifacts are created, where instead of a single large artifact including CANBridge binaries and WPILib headers, it is now separated into its own artifact as the headers are universal across all platforms.
  • Cleaned up release workflow for pushing tags to create a draft release where instead of using the GitHub CLI, a dedicated action is used instead for simplicity.
• Updated GoogleTest to fix an issue with macOS utilizing incorrect platform from the WPILib Artifactory. This would fail macOS builds completely.
• Added IOKit linker args for macOS builds to use with the serial driver.
• Removed Windows-only preprocessor macros on the serial driver for all platforms.
• Removed serial driver completely. This driver was not used, nor supported or maintained.
• Add GitHub Dependabot to automate updating GitHub Actions.

There are some changes made in this PR that breaks builds for node-can-bridge, plans to update build and CI for node-can-bridge are underway to incorporate Linux and macOS builds for future releases.

PR that addresses these concerns is created at REVrobotics/node-can-bridge#30 for discussion and review.

[qwertychouskie]

Is this necessary with the removal of the serial driver?

[garrettsummerfi3ld]

This is not necessary as the serial driver is removed. However, building back with SocketCAN or some other framework with CAN would be beneficial to leave in. There is no hurting with keeping it here.

[NoahAndrews]

Is that true if we end up using DriverKit instead of IOKit? According to this page, DriverKit is an IOKit replacement: https://developer.apple.com/system-extensions/

DriverKit provides a fully modernized replacement for IOKit to create device drivers.

[NoahAndrews]

USBDriverKit certainly sounds like what we want: https://developer.apple.com/documentation/usbdriverkit

[NoahAndrews]

Looking at the transcript of the video in that first link, there's a lot of emphasis on DriverKit being better than kernel extensions, which I'd think never would have been necessary in the first place for what we're doing.

It seems like DriverKit would require a separate System Extension that would handle all direct communication with the USB device, and then CANBridge would communicate with our DriverKit driver. That would be fine, but I think we'd have to design a whole API between our driver and CANBridge, which does sound like more work.

IOKit (and specifically IOUSBLib.h) on the other hand looks like a model more similar to what we're doing on Windows (raw USB transactions through WinUSB). The "Important" box on this page makes me a bit nervous, but it's really not clear what it's trying to say:

Devices supported on macOS 11 and later require DriverKit. Use IOKit in your apps and services to discover and use devices.

[NoahAndrews]

Just noticed this in the DriverKit presentation transcript:

The DriverKit APIs are an extension of the IOKit APIs to userspace. And we have collected them into a new DriverKit SDK that is separate to the macOS SDK. This SDK has a limited API surface for reliability and security and there is no direct access to the file system, networking, or mock messaging. This allows Apple to tailor the userspace process to running drivers and can give it elevated priority and increased capabilities. So, let's talk about some of the classes in the DriverKit SDK. First, the IOService class exists in DriverKit and is very similar to the IOKit class. There also IOMemoryDescriptor and IOBufferMemoryDescriptor classes available that are, again, very similar to IOKit. We also have replacements for the IOWorkLoop and EventSource classes in IOKit. And finally, there's a new class called OSAction that is required to represent a C Function Pointer.

So yeah, I think IOKit is probably the way to go if we don't need any kind of extra OS-level permissions like a kext would have.

[garrettsummerfi3ld]

So from reading about macOS drivers as a whole when I made the PR, its a mess at the moment on which framework to use.

IOKit is only available on macOS but is being given the "deprecated but not deprecated" treatment. Those drivers run as kernel extensions and because in the name of security and stability those are getting moved over to DriverKit which solves the security and stability issues. Lots of documentation for how to do things since its been around for so much longer, but Apple has other ideas on what to do.

If we want maintained support for the future, IOKit will be discouraged from developing for in favor of using DriverKit.

Could be totally misinterpreting this as there was a whole macOS update that happened that added more things to DriverKit that I haven't dived into the docs for but the writing on the wall looks to be there.

[NoahAndrews]

Thanks for making this (at first glance) so much easier to review! Going to try to get to this this week.

[garrettsummerfi3ld]

This PR had downstream issues with building node-can-bridge, the PR that addresses this concern has been created here: REVrobotics/node-can-bridge#30

[NoahAndrews]

Finally got started looking at this in depth! So far, I've got a request to reduce code duplication, and a PR that will add static WPILib dependencies to the artifacts instead of just dynamic ones (unofficial-rev-port#8). I'd strongly recommend reviewing and merging that PR before creating assemblePlatformSpecificArtifacts.sh.

[NoahAndrews]

I'm ok with the checkout, setup, and artifact upload steps being duplicated between the docker builds and the native builds, but all of the steps in between that are just shell-in-yaml should be moved to an assemblePlatformSpecificArtifacts.sh file so that we're not duplicating so much code.

Instead of directly referencing GHA variables like we're able to now, we can assign the GHA variables to environment variables, and reference those from the shell script. For example:

- name: Build
  run: ./assemblePlatformSpecificArtifacts.sh
  env:
    BUILD_OPTIONS: ${{ matrix.build-options }}
    PLATFORM_TYPE: ${{ matrix.platform-type }}
    ARCH: ${{ matrix.arch }}

[qwertychouskie]

This should be resolved with 2fa667d as the workflows are no longer duplicated between the container and non-container versions. Let me know if this works for you or if you still want the bash code to also be split out into a separate file.

[qwertychouskie]

In addition to fixing the code duplication issue, I also just rebased the PR. Unless any new issues are found, this should be ready to merge.