Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a Node Operator, I want to confirm which dictionaries were used #151

Open
plawton-umd opened this issue Jan 24, 2024 · 0 comments
Open

Comments

@plawton-umd
Copy link

Checked for duplicates

Yes - I've already checked

πŸ§‘β€πŸ”¬ User Persona(s)

Node Operator, api developer

πŸ’ͺ Motivation

...so that I can confirm that schema that were not available during first upload to the registry were "seen" during an overwrite and/or reload (after deletion). I want to know that the terms from the schema are available as part of the schema and not a "keyword".

πŸ“– Additional Details

During the first upload, there were multiple messages about updating LDDs

2023-12-19 16:46:05,575 [INFO] Updating LDDs.
2023-12-19 16:46:05,575 [INFO] Updating 'pds' LDD. Schema location: https://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_1J00.xsd
2023-12-19 16:46:05,845 [INFO] Downloading https://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_1J00.JSON to /tmp/LDD-65544278006652262.JSON
2023-12-19 16:46:06,261 [INFO] Creating temporary ES data file /tmp/es-2015592314936138723.json
2023-12-19 16:46:06,330 [WARN] Could not parse LDD date 2022-09-19T07:35:36
2023-12-19 16:46:06,418 [ERROR] Could not parse date from 2022-09-19T07:35:36 using patterns defined in LddUtils.Accepted_LDD_DateFormat
s
2023-12-19 16:46:06,418 [WARN] Will use field definitions from [PDS4_PDS_1500.JSON, PDS4_PDS_1600.JSON, PDS4_PDS_1800.JSON, PDS4_PDS_190
0.JSON, PDS4_PDS_1A10.JSON, PDS4_PDS_1B00.JSON, PDS4_PDS_1D00.JSON, PDS4_PDS_1E00.JSON, PDS4_PDS_1F00.JSON, PDS4_PDS_1G00.JSON, PDS4_PDS
_1H00.JSON]
2023-12-19 16:46:06,419 [INFO] Updating Elasticsearch schema.
2023-12-19 16:46:06,975 [INFO] Updated 5 fields

and the messages about the ones not found

2023-12-19 16:46:19,601 [INFO] Updating 'sb' LDD. Schema location: https://pds.nasa.gov/pds4/sb/v1/PDS4_SB_1J00_1000.xsd
2023-12-19 16:46:19,853 [INFO] Downloading https://pds.nasa.gov/pds4/sb/v1/PDS4_SB_1J00_1000.JSON to /tmp/LDD-9953225849575519692.JSON
2023-12-19 16:46:20,060 [INFO] 404 - Not Found
2023-12-19 16:46:20,061 [INFO] Will retry in 5 seconds
2023-12-19 16:46:25,061 [INFO] Downloading https://pds.nasa.gov/pds4/sb/v1/PDS4_SB_1J00_1000.JSON to /tmp/LDD-9953225849575519692.JSON
2023-12-19 16:46:25,107 [INFO] 404 - Not Found
2023-12-19 16:46:25,107 [INFO] Will retry in 5 seconds
2023-12-19 16:46:30,107 [INFO] Downloading https://pds.nasa.gov/pds4/sb/v1/PDS4_SB_1J00_1000.JSON to /tmp/LDD-9953225849575519692.JSON
2023-12-19 16:46:30,211 [INFO] 404 - Not Found
2023-12-19 16:46:30,212 [ERROR] Could not download https://pds.nasa.gov/pds4/sb/v1/PDS4_SB_1J00_1000.JSON
2023-12-19 16:46:30,212 [WARN] Will use 'keyword' data type.

2023-12-19 16:46:30,212 [INFO] Updating 'nh' LDD. Schema location: https://pds.nasa.gov/pds4/mission/nh/v1/PDS4_NH_1J00_1000.xsd
2023-12-19 16:46:30,471 [INFO] Downloading https://pds.nasa.gov/pds4/mission/nh/v1/PDS4_NH_1J00_1000.JSON to /tmp/LDD-892648505940796031
9.JSON
2023-12-19 16:46:30,573 [INFO] 404 - Not Found
2023-12-19 16:46:30,573 [INFO] Will retry in 5 seconds
2023-12-19 16:46:35,574 [INFO] Downloading https://pds.nasa.gov/pds4/mission/nh/v1/PDS4_NH_1J00_1000.JSON to /tmp/LDD-892648505940796031
9.JSON
2023-12-19 16:46:35,607 [INFO] 404 - Not Found
2023-12-19 16:46:35,607 [INFO] Will retry in 5 seconds
2023-12-19 16:46:40,607 [INFO] Downloading https://pds.nasa.gov/pds4/mission/nh/v1/PDS4_NH_1J00_1000.JSON to /tmp/LDD-892648505940796031
9.JSON
2023-12-19 16:46:40,707 [INFO] 404 - Not Found
2023-12-19 16:46:40,707 [ERROR] Could not download https://pds.nasa.gov/pds4/mission/nh/v1/PDS4_NH_1J00_1000.JSON
2023-12-19 16:46:40,707 [WARN] Will use 'keyword' data type.

After the 2 missing dictionaries were confirmed to be in the correct locations, an overwrite of the data and then a deletion and load of the data were done. Neither of the types of messages above displayed during the overwrite or reload activities.

Acceptance Criteria

Given
When I perform
Then I expect

βš™οΈ Engineering Details

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: ToDo
Development

No branches or pull requests

2 participants