Complete refactoring per API WG discussion on end-point redesign

[tloubrieu-jpl]

Follow on to NASA-PDS/registry-api#131

See slides on https://docs.google.com/presentation/d/1CyrstNQ04Vm3eQvUNSSezxYzdyoyv3pSQXNCahMM2cg/edit#slide=id.gf8baa2253f_0_1

[al-niessner]

I see the proposal, but what was the conclusion?

Do you really want /classes/bundle/{id}/members instead of /products/{id}/members? What is the error if /classes/bundles/{id} is not a bundle?

[tloubrieu-jpl]

Table this ticket for this sprint, so to focus on bug fixes

[jordanpadams]

@al-niessner we are going forward with Proposal 2 from the slides. For the error message, something like Class does not exist works for now.

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

To be clear, these are all of the endpoints:

/products/{id}/members
/products/{id}/members/members
/products/{id}/member-of

/classes/bundles/{id}/members 
/classes/collections/{id}/members
/classes/documents/{id}/member-of
/classes/observational/{id}/member-of
/classes/observational/{id}/member-of/member-of

This seems like a short list but it is all of the endpoints listed for proposal 2. Seems like you may also want endpoints in proposal 1:

/products/{id}
/products/{id}/all
/products?q=... searches products (any class) for
/classes
/classes/bundles?q=... searches bundles for

Note: change searches for x to searches x for

[jordanpadams]

@al-niessner sorry for the confusion.

we must support ALL existing endpoints AND the new endpoints from proposal 2

[al-niessner]

@jordanpadams

Existing as in endpoints in version 1.0.2 or as in endpoints in new approach? Proposal 1 seems to be the base ones where /gid changed to /classes and /uid changed to /products in the newer style and proposal 2 a rework of referencing. So, when you say ALL do you mean 1.0.2 endpoints plus proposal 1 and proposal 2?

[jordanpadams]

@al-niessner sorry for the lack of clarity here.

By existing endpoints, I am referring to the currently released version 1.0.2, e.g. what is here: https://app.swaggerhub.com/apis-docs/PDS_APIs/registry/1.0.0

beyond that, we want the endpoints proposed with Proposal 2:

/products/{id}/members returns products contained directly in the current product
/products/{id}/members/members returns products indirectly contained in the current product
/products/{id}/member-of returns products directly containing the current product (e.g. collections, bundles)

/classes/bundles/{id}/members get collections of a bundle
/classes/collections/{id}/members get observational products or documents of a collection
/classes/documents/{id}/member-of
/classes/observational/{id}/member-of
/classes/observational/{id}/member-of/member-of

[jordanpadams]

actually, you are right. implement everything:

• current v1.0.2 API endpoints
• Proposal 1 (search/queries)
• Proposal 2 (hierarchy)

[al-niessner]

@jordanpadams

If I can, do you want me to mark the older not represented by newer (no overlap) as deprecated? Obviously, /products/{identifier} is the same in both. Several overlap.

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

In the case of these:

/classes/bundles/{id}/members get collections of a bundle
/classes/collections/{id}/members get observational products or documents of a collection
/classes/documents/{id}/member-of
/classes/observational/{id}/member-of
/classes/observational/{id}/member-of/member-of

Do you want just those or do you want:

/classes/{class id}/{identifier}/members
/classes/{class id}/{identifier}/members/members
/classes/{class id}/{identifier}/members-of
/classes/{class id}/{identifier}/members-of/members-of

Obviously bundles and collections would be strange with bundles and members-of like an error message but keeps from having to maintain a bigger list that can expand wildly given the large number of PDS product classes.

---

Actually, never mind. Have the same error message with /products/{identifier}/member-of when the {identifier} is a bundle. So, just do it with latter and worry about messages later.

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

Have not added the old 1.0 version stuff, but have the proposal stuff in place at swagger pds-api-202.5

for unknown reasons swagger made me update versions again

[jordanpadams]

@al-niessner just added a couple comments. but looks good.

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

These will be put into a deprecated group unless you want named something else.

  /bundles/{identifier}
  /bundles/{identifier}/all
  /bundles/{identifier}/latest
  /bundles/{identifier}/collections
  /bundles/{identifier}/collections/all
  /bundles/{identifier}/collections/latest
  /bundles/{identifier}/products
  /collections/{identifier}
  /collections/{identifier}/all
  /collections/{identifier}/latest
  /collections/{identifier}/bundles
  /collections/{identifier}/products
  /collections/{identifier}/products/all
  /collections/{identifier}/products/latest
  /products/{identifier}/bundles
  /products/{identifier}/bundles/all
  /products/{identifier}/bundles/latest
  /products/{identifier}/collections
  /products/{identifier}/collections/all
  /products/{identifier}/collections/latest

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

All have been added. Check it out on swagger using link 3 comments above.

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

Snag - as in stuck until clarified. The old API and the interim API that I ad-hoc did as a demonstration contained /latest and /all for references. The new API on the slides does not. Was this an omission on the slide but they should exist in the new API or did we get rid of them for good and only the latest are ever returned?

If we got rid of them, then what should be done with the /all in the old API that has been deprecated? Should those endpoints return an error or just return the latest ignoring the all request?

If they require /all and /latest then that is fine but it needs to be made clear.

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

New API is still broken. Trying to implement and not working out well as in not possible.

/products/{identifier}/member-of is supposed to mean what? If it depends on the type of the identifier then that really is going to slow down the API. It means have to look up the identifier to determine is product type then decide what member_of means. Can it mean multiple things like collection or mission or something else? If you change it to /products/{identifier}/member-of/{class} then that appropriately constrains the request and not defining class can mean any/all so that you get all members not just bundle->collection->product tree. Same goes for member and member/member and member-of/member-of.

---

Oh, I think I am beginning to understand the disconnect. The documentation is written as if /products/{identifier}/member-of means the identifier is neither collection nor bundle. However, there is nothing to prevent the user from using a bundle or collection identifier which then changes the meaning of member in all forms. It also limits the membership to just hierarchy (bundle->collection->product) removing references in general.

Am I reading the proposal correctly?

[jordanpadams]

/products/{identifier}/member-of
this specifically applies to traversing the bundle->collection->product hierarchy. so for example:

/products/urn:nasa:pds:some_collection_identifier/member-of/

should return the latest version of the parent bundle(s)

/products/urn:nasa:pds:some_product_identifier/member-of/

should return the latest version of the parent collection(s)

/products/urn:nasa:pds:some_product_identifier/member-of/member-of

should return the latest version of the grandparent bundle(s)

[jordanpadams]

looks like i should have finished reading your comment

It also limits the membership to just hierarchy (bundle->collection->product) removing references in general.

Correct. "Membership" is specifically related to organizational tree structure of the data (bundle->collection->product == trunk->branch->leaf).

I would considered "references" a much broader relationship type than "membership", including trunk<->trunk, branch<->branch, and leaf<->leaf relationships, which would not necessarily be considered "membership".

[jordanpadams]

@jordanpadams @jimmie @tloubrieu-jpl

Snag - as in stuck until clarified. The old API and the interim API that I ad-hoc did as a demonstration contained /latest and /all for references. The new API on the slides does not. Was this an omission on the slide but they should exist in the new API or did we get rid of them for good and only the latest are ever returned?

If we got rid of them, then what should be done with the /all in the old API that has been deprecated? Should those endpoints return an error or just return the latest ignoring the all request?

If they require /all and /latest then that is fine but it needs to be made clear.

@al-niessner if we can, I would like to keep all and latest endpoints, with the default always going to latest when it is not specified. The rationale being /all provides the ability to display the provenance of a product.

[al-niessner]

@jordanpadams @jimmie @tloubrieu-jpl

I will put the all and latest choices back in then.

As a user reading a document for a camera, I cannot use member-of to find all the instruments that use that camera -- as in the document is a member of the instrument. Are going to add something other than membership for this example?

[jordanpadams]

@al-niessner possibly. TBD on whether we handle that explicitly via the API endpoints or just provide an example query. That specific example can be a bit tricky with PDS metadata.

[jordanpadams]

But that is a valid example we will want to be able to support somehow

[jimmie]

Mostly done, needs a looking at by @jordanpadams, @jimmie.