Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ENG-6313] allow raw data to expire #826

Merged
merged 4 commits into from
Oct 17, 2024
Merged

[ENG-6313] allow raw data to expire #826

merged 4 commits into from
Oct 17, 2024

Conversation

aaxelb
Copy link
Contributor

@aaxelb aaxelb commented Oct 15, 2024

  • add RawDatum.expiration_date model field (with index)
  • add "expiration_date" query param to /trove/ingest
  • update digestive tract to appropriately handle expired data
  • add periodic task to expel expired data daily
    • if a supplementary record's latest RawDatum expires, delete the SupplementaryIndexcardRdf and send affected indexcards back thru digestive_tract.derive
    • if an indexcard's latest non-supplementary RawDatum expires, mark that indexcard deleted (using existing mechanisms)

aaxelb added 2 commits October 14, 2024 18:21
@aaxelb
fix: more consistent expulsion by suid
d696fdf
@aaxelb
allow raw data to expire
9679110 
- add `RawDatum.expiration_date` model field (with index)
- add "expiration_date" query param to `/trove/ingest`
- update digestive tract to appropriately handle expired data
- add periodic task to expel expired data daily
@coveralls
Copy link

coveralls commented Oct 15, 2024 •
edited
Loading

Coverage Status

coverage: 90.426% (+0.1%) from 90.279%
when pulling 36ea734 on aaxelb:feature/perishable-metadata
into 86b1a9b on CenterForOpenScience:develop.

@aaxelb aaxelb marked this pull request as ready for review October 15, 2024 16:42
@aaxelb aaxelb requested a review from mfraezz October 15, 2024 17:16
mfraezz
mfraezz approved these changes Oct 16, 2024
View reviewed changes
Copy link
Member

@mfraezz mfraezz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ⭐.
Legible, well-tested, no obvious bugs.

@aaxelb aaxelb merged commit 4fb6ed8 into CenterForOpenScience:develop Oct 17, 2024
3 checks passed
@aaxelb aaxelb deleted the feature/perishable-metadata branch October 18, 2024 13:25
@aaxelb aaxelb mentioned this pull request Oct 18, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mfraezz mfraezz mfraezz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@aaxelb @coveralls @mfraezz