HTTP 524 Error in v1.1.1

[mhellmeier]

Situation

I tried to update the DEPENDENCIES file of eclipse-tractusx/eclipse-tractusx.github.io.

Problem

When using the latest Dash tool in version 1.1.1, the update fails with the HTTP 524 error, even after several tries:
First try:

$ java -jar ./org.eclipse.dash.licenses-1.1.1.jar package-lock.json -project automotive.tractusx -summary ./DEPENDENCIES
[main] INFO Querying Eclipse Foundation for license data for 500 items.
[main] INFO Found 148 items.
[main] INFO Querying Eclipse Foundation for license data for 500 items.
[main] INFO Found 77 items.
[main] INFO Querying Eclipse Foundation for license data for 447 items.
[main] INFO Found 73 items.
[main] INFO Querying ClearlyDefined for license data for 500 items.
[main] ERROR Error response from ClearlyDefined HTTP 524

Second try:

$ java -jar ./org.eclipse.dash.licenses-1.1.1.jar package-lock.json -project automotive.tractusx -summary ./DEPENDENCIES
[main] INFO Querying Eclipse Foundation for license data for 500 items.
[main] INFO Found 148 items.
[main] INFO Querying Eclipse Foundation for license data for 500 items.
[main] INFO Found 77 items.
[main] INFO Querying Eclipse Foundation for license data for 447 items.
[main] INFO Found 73 items.
[main] INFO Querying ClearlyDefined for license data for 500 items.
[main] ERROR Error response from ClearlyDefined HTTP 524

When using the older version 1.1.0, it works:

$ java -jar ./org.eclipse.dash.licenses-1.1.0.jar package-lock.json -project automotive.tractusx -summary ./DEPENDENCIES
[main] INFO Querying Eclipse Foundation for license data for 500 items.
[main] INFO Found 148 items.
[main] INFO Querying Eclipse Foundation for license data for 500 items.
[main] INFO Found 77 items.
[main] INFO Querying Eclipse Foundation for license data for 447 items.
[main] INFO Found 73 items.
[main] INFO Querying ClearlyDefined for license data for 500 items.
[main] INFO Found 473 items.
[main] INFO Querying ClearlyDefined for license data for 500 items.
[main] INFO Found 459 items.
[main] INFO Querying ClearlyDefined for license data for 92 items.
[main] INFO Found 92 items.
[main] INFO Vetted license information was found for all content. No further investigation is required.

Used Versions

• v1.1.0 (works)
• v1.1.1-20250207.065028-427 (doesn't work)

[waynebeaton]

There are no meaningful changes in how we access the ClearlyDefined API between those versions. We're using the HTTPClient from the JRE, so -- unless you're using a different JRE to invoke the tool -- there are no differences there. To the best of my knowledge, the calls to the API are identical in these (and every other) version.

500 level responses describe server errors. My experience has been that the ClearlyDefined server sometimes does this and you just have to wait and try again. We don't currently handle rate limits and what-not, but a 524 does not describe a rate limit related problem.

My assumption at this point is that difference must be an unfortunate consequence of timing.

Does the problem still exist?

HTTP 524 is a timeout error, so you might try playing with the timeout parameter. Although, I'm not sure that setting timeout in the client actually has an effect in this case (the error is coming from the server). Also, the default value is 120 seconds which already seems like a long time.

Another option might be to lower the batch size with the batch parameter (e.g., set it to 100). This will result in more calls, but the server may have a better time with the smaller request.

I'll add some debug code to the HTTP client so that we can at least capture the response headers to see if there's anything helpful in there.

[waynebeaton]

I've pushed an update that will appear in the next build. If you're still getting the error, please add -Dorg.slf4j.simpleLogger.defaultLogLevel=debug when you invoke the tool and share the response headers.