How to get a more detailed clang-format report in GitHub

[rturrado]

I'm adding cpp-linter to my project. And I'm getting this kind of reports from the C++ linter:

They seem not to go further than:

1. Advising you to run clang-format on a given file (they don't even say what clang-format version).
2. Pointing to the lines that are raising some clang-format errors.

Is it possible to get a more detailed explanation of the formatting errors? This screenshot from cpp-linter's GitHub README seems to suggest so:

This is the recipe I'm using:

jobs:
  clang-format:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: C++ linter
        uses: cpp-linter/cpp-linter-action@v2
        id: linter
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          extensions: 'hpp,cpp'
          files-changed-only: 'false'
          format-review: 'true'
          step-summary: 'true'
          style: 'file'  # use .clang-format config file
          tidy-checks: '-*'  # disable clang-tidy checks
          version: 18
      - name: Fail fast
        if: steps.linter.outputs.clang-format-checks-failed > 0
        run: exit 1

How do you get that line by line error explanation?

Many thanks!

[2bndy5]

they don't even say what clang-format version

This is a good idea. We should be telling contributors what version was used to generate the concerns. It also wouldn't be a bad idea to include the commit SHA for the event that triggered the analysis.

Clang-format doesn't output very much

Unlike clang-tidy, clang-format does not tell you what rule was violated. The only thing we can get from clang-format (using it's XML output) is

• the range spanning the replacement(s). We translate byte offsets into line and column numbers ourselves.
• the actual text of the replacement that would be applied.

File annotations cannot include code snippets

I feel like I'm repeating this too often, so maybe it should go into the docs somewhere. This is a GitHub limitation.

Given that each clang-format file annotation would look the same (and GitHub limits each job to only 50 maximum annotations), I decided to group all the line numbers that clang-format would change into a single annotation at the beginning of the file.

---

How do you get that line by line error explanation?

The closest solution I can think of is our format-review feature. But there are documented caveats about our PR review feature.

There is currently no way to get more info about clang-format's output on push events. If you want detailed diagnostics, then I would suggest configuring clang-tidy to your liking by either

• setting tidy-checks to a comma-separated list of checks
• setting tidy-checks to a blank string and check in the .clang-tidy config file(s)

Note, clang-tidy works a lot better when it has a compilation database (typically generated by your build system generator -- like CMAKE, Meson, Bazel, etc) to detail what compiler flags are used for each file.

[2bndy5]

Clang-format doesn't output very much

Locally, you can see what we're dealing with by running clang-format --dry-run on a particular file(s).

clang-format --dry-run cpp-linter/tests/demo/demo.cpp
cpp-linter/tests/demo/demo.cpp:3:19: warning: code should be clang-formatted [-Wclang-format-violations]
#include <stdio.h>
                  ^
cpp-linter/tests/demo/demo.cpp:8:11: warning: code should be clang-formatted [-Wclang-format-violations]
int main(){
          ^
cpp-linter/tests/demo/demo.cpp:10:13: warning: code should be clang-formatted [-Wclang-format-violations]
    for (;;) break;
            ^
cpp-linter/tests/demo/demo.cpp:10:20: warning: code should be clang-formatted [-Wclang-format-violations]
    for (;;) break;
                   ^
cpp-linter/tests/demo/demo.cpp:13:30: warning: code should be clang-formatted [-Wclang-format-violations]
    printf("Hello world!\n");
                             ^
cpp-linter/tests/demo/demo.cpp:18:14: warning: code should be clang-formatted [-Wclang-format-violations]
    return 0;}
             ^

[rturrado]

Many thanks for your quick reply @2bndy5!

This is a good idea. We should be telling contributors what version was used to generate the concerns. It also wouldn't be a bad idea to include the commit SHA for the event that triggered the analysis.

I was thinking more on the lines of just the clang-format version. For example, if you don't specify a version, and you get some errors, you won't know what version of clang-format was used (in order to rerun it locally, for example).

Clang-format doesn't output very much

Unlike clang-tidy, clang-format does not tell you what rule was violated. The only thing we can get from clang-format (using it's XML output) is

• the range spanning the replacement(s). We translate byte offsets into line and column numbers ourselves.
• the actual text of the replacement that would be applied.

Fair enough, thanks for explaining.

File annotations cannot include code snippets

I feel like I'm repeating this too often, so maybe it should go into the docs somewhere. This is a GitHub limitation.

Given that each clang-format file annotation would look the same (and GitHub limits each job to only 50 maximum annotations), I decided to group all the line numbers that clang-format would change into a single annotation at the beginning of the file.

Understood, thanks!

How do you get that line by line error explanation?

The closest solution I can think of is our format-review feature. But there are documented caveats about our PR review feature.

There is currently no way to get more info about clang-format's output on push events.

OK! I haven't got that far (PR) yet, just using push events at the moment.

If you want detailed diagnostics, then I would suggest configuring clang-tidy to your liking by either

• setting tidy-checks to a comma-separated list of checks
• setting tidy-checks to a blank string and check in the .clang-tidy config file(s)

One question here (because I haven't started yet with clang-tidy). Can clang-tidy then also report about formatting errors? I thought it was more about code guidelines (this variable name looks wrong, you could use [[nodiscard]] here, and so on)

Note, clang-tidy works a lot better when it has a compilation database (typically generated by your build system generator -- like CMAKE, Meson, Bazel, etc) to detail what compiler flags are used for each file.

I'll keep this in mind. I'm using Conan together with CMake. I've seen somewhere that, for the database, it was assumed that the CMake project had CMAKE_EXPORT_COMPILE_COMMANDS set.

[2bndy5]

Can clang-tidy then also report about formatting errors?

Yes and no. It does focus on code style rather than code format. There a number of rules that might "clash" with clang-format style, but I will defer to the clang-tidy docs for details on that.

Although, in cpp-linter, if you set the style option for clang-format as not a blank string (and tidy-checks is not -*), then cpp-linter will also pass the style value into clang-tidy. This should produce suggestions more consistent with clang-format's replacements, but that's not always guaranteed.

I've seen somewhere that, for the database, it was assumed that the CMake project had CMAKE_EXPORT_COMPILE_COMMANDS set.

Nope. You have to enable it. This can be done from the CLI:

cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -B build -S .

Or in one of your CMakeLists.txt files (typically at root-level):

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

I think there might be other ways to enable it (using CMake's user-defined presets), but the above offerings have suited me well in the past.

Just remember to tell cpp-linter where the database exists. This is practically required when using ninja to build your project.

- uses: cpp-linter/cpp-linter-action@v2
  with: 
    database: build
    ignore: build # ignore any generated build artifacts

[rturrado]

@2bndy5 Many thanks for the detailed answer!

[2bndy5]

@rturrado Thanks to your feedback, we just released a patch that will now show the version number of the clang tool(s) used.

[rturrado]

@2bndy5 Many thanks, very kind!