Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[teraslice] Asset load dependency check #3822

Merged
merged 5 commits into from
Nov 14, 2024
Merged

Conversation

busma13
Copy link
Contributor

@busma13 busma13 commented Nov 12, 2024

This PR makes the following changes:

  • adds a check on asset upload to see if the asset is compatible with the teraslice version
  • creates a test asset with a high minimum teraslice version
  • test the test asset to ensure an error is thrown
  • fix a bug that leaves behind an s3 object in some situations if an asset fails to upload

ref: #3685

@busma13 busma13 added this to the Teraslice 2.8.0 milestone Nov 12, 2024
Comment on lines 65 to 67
minimum_versions?: {
teraslice: string;
};
Copy link
Contributor Author

@busma13 busma13 Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably want to think hard about what we name this and if it should be an object that can hold several versions or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would we prevent the opposite scenario? Where a new teraslice version breaks all the assets? For example a major change to job-components? This may be out of scope for this PR, or this just a tolerable defect?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say it's unavoidable since you can't know what breaking changes can be implemented in the future.

As long as this value won't need to be validated by convict that nested value looks OK. But every time I've tried to nest something like that I've gotten hung up when it goes through convict for validation (I think that was the issue). If we wanted to play it dumb and safe we could just do this: "minimum-teraslice-version".

@busma13 busma13 marked this pull request as ready for review November 12, 2024 22:00
@busma13 busma13 requested review from godber and sotojn November 12, 2024 22:00
@busma13 busma13 changed the title Asset load dependency check [teraslice] Asset load dependency check Nov 12, 2024
@godber
Copy link
Member

godber commented Nov 13, 2024

We really shouldn't be adding large binary files to the repo, especially one this large. At this point, since you've pushed the branch it's already there. So leave it.

Copy link
Member

@godber godber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets discuss this in the morning.

}
if (metadata.minimum_versions) {
const terasliceVersion = getPackageJSON().version;
if (semver.gt(metadata.minimum_versions.teraslice, terasliceVersion)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the asset contains:

    minimum_versions: {
    };

So metadata.minimum_versions.teraslice is undefined?

Maybe we should just go back to a single property, no nested thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this would have thrown an error because semver.gt() won't take undefined as a parameter.

@busma13
Copy link
Contributor Author

busma13 commented Nov 14, 2024

I think the single property makes more sense. Any other compatibility check can have it's own top level property. This also gives me a chance to use a smaller asset.

@godber godber merged commit c99f3e7 into master Nov 14, 2024
49 checks passed
@godber godber deleted the asset-load-dependency-check branch November 14, 2024 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants