Query string implementation of profile selection #3
Comments
|
The relevance and requirement to deal with QSA implementations has been discussed at length and rules in scope for the conneg doc with both a motivating Use Case (https://github.com/w3c/dxwg/issues/239) and derivative Requirements. What we are doing in the doc is explaining the conceptual logic of what "content negotiation by profile" is and then showing how it may be done. HTTP conneg is the first/primary way but there are other ways, such as RESTful APIs that are already in use (Epimorphic's Linked Data Registry, CSIRO's SISSVoc, pyLDAPI etc...) and we want to ensure functional equivalence between methods.
Well, we should and will refer to the conneg doc from the Guidance doc, but we just have to be sure not to duplicate work. We have a duty to at least describe the IETF doc's information in the conneg doc since the IETF doc isn't a recognised W3C doc and that's why that info's there.
Well, as a tool maker (APIs) I need, or would like, to be told how to implement. The test API I've implemented (https://github.com/CSIRO-enviro-informatics/profile-conneg-qsa-realisation) actually demos QSA & conneg methods for dummy resources. The real production toolkit I use (https://github.com/RDFLib/pyLDAPI) will, when the spec is settled, implement both. As per the Use Case at the top, we benefit from multiple ways of getting at things.
This redundancy has been removed in the FPWD candidate now at https://w3c.github.io/dxwg/conneg-by-ap/. I'm pretty sure I've not touched on each of your concerns in the text above. Could you perhaps itemise the issues again so we can deal with them one-by-one? I just find the discursive paragraph a little hard to follow. |
|
@nicholascar scripsit
It's better but not really good yet. I think we should call §6.1 "HTTP Headers" and §6,2 "URI Query String Arguments" (or something like that). Would that solve your issue, @agreiner? |
|
@larsgsvensson Yes, that would solve the naming issue for me. |
|
Re the existing use case, it is pretty clear about what it asks for. That is a use case that I have enthusiastically supported. "In addition to any HTTP Content Negotitation-based approaches to navigate available profiles of information about a resource, we wish to provide a web-browser-friendly equivalent methods for humans." This is about navigation, not recreating negotiation. It is for humans. |
|
@agreiner scripsit:
OK, I've created w3c/dxwg#553 to fix that one leaving this issue to cope with the more difficult parts. |
Changed issue #594 to #544 Added [Annette's proposed text](https://lists.w3.org/Archives/Public/public-dxwg-wg/2018Nov/0544.html) as text for the note
|
I think I need to be more clear about my concerns with the query string negotiation. They are twofold, first that the query string treatment should not be normative, and secondly that if we give an informational example of handling the choice of a profile via a browser, we should offer something that is a better example for human use. Our charter calls for development of content negotiation by profile. Nothing in that document suggests a requirement to offer a query-string-based specification for handling profiles. I see no reason to think that such a thing would even be implied by the charter, as there exists no prior standard for handling content negotiation with query strings. If such a thing is needed at all, it would make the most sense to develop it in the context of existing use cases of content negotiation, such as language and media type. Offering normative specifications for a query-string-based negotiation method is overreaching our charter. The use case on which this work is based is about making the choice of a profile easy for humans. We agreed to that use case because the header-based negotiation creates something of a black box for users, and it is not made available to users by browsers. Indeed, the whole point of content negotiation is to enable a behind-the-scenes selection that the user need not think about. The problem is that, in the case of a human consumer of datasets, it makes sense for them to have agency in the choice of profile. Forcing them to suffer through the experience of negotiation by faithfully replicating that black box does not help users. Requiring non-developer users to be aware of a fine detail in a standards document, know whether to apply it to a given site, and type in a correct URL is anything but user friendly. If we offer a method of enabling humans to select a dataset by profile, we should be modeling good usability, where the user is shown a list of options and simply selects the one they prefer. Our use case comes down to allowing users to select, not to negotiate. Aside from the danger of specifying a usability nightmare, prescribing a specific way of coding a web application goes against the freedom web developers expect in building applications. Development practices change rapidly, and at any given time there is little agreement on the "right way" to do something. In particular, there has been much debate about the use of query strings in REST-based web applications. Calling a specific implementation normative is tantamount to taking sides in that debate. We risk alienating developers and creating a standard that ages quickly. In addition to creating issues for human consumption of datasets, the use of query strings for negotiation may also have an unintended side-effect for programmatic access. It introduces the possibility that developers will use query strings instead of http headers, creating a competing approach for programmatic access, making the web of data more complex and difficult to use. We may caution people to use both techniques, but the likelihood is that people will use what they like and avoid the extra work. The query string approach is also the main driver for the use of tokens to identify profiles. Tokens without some sort of registry or other means of enabling discovery and minimizing conflict are problematic. Without a requirement for query strings, a URI could serve as a single universal identifier for a profile. To be clear, there is nothing technically wrong with creating a means of using query strings to pass user choices to an API, especially if it enables a usable graphical solution for user profile selection, but it is that graphical selection that we need to exemplify, not the API itself, and not negotiation. One could just as well use an API that accepts URLs without query strings. Not every web API deserves to become a standard. Making query-string profile negotiation normative overreaches our charter, models poor usability for humans, risks alienating developers, competes with header-based authentication, and blocks the assignment of unique profile IDs. Unless this is addressed, I don't see myself voting to publish the profile negotiation document. |
What is normative is that one adheres to the Abstract Model for any Realizations. It's not normative to, for instance, have to implement a QSA API just because you implemented an HTTP API according to the Abstract Model. If you do implement a QSA approach then you should do so according to the QSA Realization.
Please make a suggestion! We have lots of APIs that use a primitive QSA version of the Realization outlined here and they work quite well. We hope that future APIs adhering to this QSA will work better, for humans, and will accord with the HTTP Realization. "better example for human use" doesn't really give us anywhere to go unless you can either point out an un-met Use Case or perhaps a demo solution you must have in mind.
Motivation indeed to make one if it were true! There are plenty of standards though: OAI-PMH for one. Fixed to XML for Content-Type sure but it's all about negotiating by profile, e.g.: Sample AU1 from the Geoscience Australia in "Dublin Core" profile: http://ldapi.ga.gov.au/sss/oai?verb=GetRecord&identifier=AU1&metadataPrefix=oai_dc Sample AU1 in "CSIROv3" profile: http://ldapi.ga.gov.au/sss/oai?verb=GetRecord&identifier=AU1&metadataPrefix=csirov3
Do you mean like this one: https://github.com/w3c/dxwg/issues/239 to which I see you've commented "I think this use case is extremely important and fully in scope"? Do you mean we should define, in a QSA Realization, how conneg by profile works within a context of conneg by Content Type & conneg by Language? That's what we are doing so I don't follow.
Can you make a proposal?
The goal here is to present an Abstract Model with 2+ Realizations. HTTP as the Internet's core and QSA for a more human-usable form. We need 2+ to prove the point. Feel free to contribute a GraphQL API. I might consider that as a WG note anyway.
The API tooling I use that implements a precursor of this handles both HTTP & QSA negotiation. A precedence order is given (anything human-set wins over machine-set so QSA tends to win over HTTP) but either approach many be used.
So what? You just mentioned "Development practices change rapidly, and at any given time there is little agreement on the "right way" to do something" so why should developers not use QSA (or GraphQL if implemented?)
True, QSA is the driver for tokens and we have been over the mechanics of this a few times: tokens can be used as long as they deterministically map to a URI within a client/server session. Seems sound...
Again, we are not saying that any implementation of conneg by profile is required to implement all Realizations.
It's designed to be easier to use, by humans, than HTTP conneg so you'll have to unpack this one if you're to convince me!
How so? A profile definition, by PROF and as recommended by Guidance, will be identified by URI. |
|
Issue addressed with PR w3c/dxwg#898 |
|
Reading through the doc now, I see that that section is once again marked as normative. |
|
@agreiner Trying to understand your comment - was it marked as non-normative in earlier drafts? Was there an agreement that this section would be non-normative? Thanks. |
The conneg deliverable is defined as "An explanation of how to implement the expected RFC and suitable fallback mechanisms as discussed at the SDSVoc workshop." QSA is proposed as a fallback mechanism, so I'd say it's definitely covered by the charter. That the implementation description is normative doesn't say that you have to implement it, only that if you implement it, that is the interoperable way of doing it. |
|
A further comment from @agreiner that came over the mailing list as a reply to Karen's comment:
|
|
Since Annette re-opened this issue, I remove the due-for-closing tag. |
|
Adding comments from the mailing list for completeness: Annette wrote
Lars wrote:
Annette wrote as a follow-up on https://github.com/w3c/dxwg/issues/544#issuecomment-534850916:
Lars wrote:
Annette wrote:
Lars, Annette and Lars wrote (re-ordered for easier reading):
|
|
And a further comment from Nick:
|
|
On the mailing list, Annette wrote:
|
|
On the mailing list, Nick wrote:
|
I see no reason why there can't be an abstract model and then a few examples of implementations of that model; examples generally are employed to help the reader understand the normative parts of the specification. I believe this is what @agreiner is asking for. Those examples would be considered "good" examples but they wouldn't be normative parts of the spec, as in general examples are not. The tricky part about that is that it makes it hard to show sample code throughout the document, and indeed some specs that define abstract models are very hard to read for that reason. How about stating at the beginning that the document defines an abstract model and uses examples to show how the model could be implemented. Then you could say that throughout the document you include some sample code using the http example implementation to provide clarity for the reader, but that those examples within the code as well as those in the example section are not normative. I will say that I consider the DCAT -> DCAT-APs to be a good practice. Even though it wasn't originally designed to be a general vocabulary plus profiles, that's actually not a bad approach: creating something quite general and then letting communities develop their specific implementations that conform to the general approach. In DCAT's case, DCAT is normative, but the APs are not, at least not as W3C specifications. It would make sense for Conneg to follow this same approach, with the functional APs being non-normative. |
|
The document has now been updated with the changes agreed on in https://github.com/w3c/dxwg/issues/544#issuecomment-535388527. Calling on @agreiner and @kcoyle to review. |
|
Is the conclusion that to conform a service must implement one of the three offered profiles? Or can they develop their own based on the abstract profile, INSTEAD OF one of the three? |
They can develop their own based on the abstract profile instead of one of the three. |
|
OK, so I would clarify the wording from: If a system wishes to show conformance to this specification, conformance to: To demonstrate conformance a system MUST implement at least one functional profile that |
|
The problem there was the term "additional" which could be interpreted to mean "additional to one of the provided ones" rather than "instead of". This is why I suggest doing some copy editing, because the spec needs to be absolutely clear. |
|
I still don't like all this stuff about conforming to functional profiles. If I want to create an app that conforms to this spec, do I need to write a profile and have it registered somewhere? What exactly constitutes a functional profile for purposes of verifying conformity? I would much rather say "To demonstrate conformance a system MUST implement all elements of the abstract model." |
|
Its true that "To demonstrate conformance a system MUST implement all elements of the abstract model." however that is true only of the abstract profile. This doesnt detract from the utility (i.e. interoperability goals of this specification) of identifying specific implementations using the canonical IETF RFC or the provided QSA option. an app that "conforms to the spec" will conform to one of the two concrete profiles of the abstract spec - or it will conform to some other functional profile of the abstract specification (modulo an interpretation of conforms that implies that instances of the app conform, or that the app is not reusable software but an instance in its own right). This profile exists in practice even if you dont name it or document it... you just cant make statements about conformance to it (e.g. in DCAT) without some form of "registering" - but that could be as simple as minting a URI. This specification says nothing about the "registration" of other profiles, it only sets out the conditions for it to be said to conform to this specification. |
|
@rob-metalinkage Note, I have suggested other wording for that conformance paragraph at w3c/dxwg#544. |
OK, I get your point. I find your proposed text clearer. |
That's true. However, @agreiner has a point when she asks if she needs to write and register a profile, since the §2.1 of Profile Guidance -- which we've agreed is part of our family of documents -- says:
So that means that a profile needs to have a name (i. e. a URI). I don't think we've intended to have anonymous profiles as an equivalent to anonymous inner classes in e. g. Java. |
|
I've adopted @kcoyle's suggested wording essentially unchanged (just removed duplicate Fig 2 and added 'also' in a branch, see https://raw.githack.com/w3c/dxwg/conneg-Issue-544/conneg-by-ap/index.html#conformance-profiles. Will add other updates to this branch as we establish what they should be. |
|
I still don't see anything reassuring to developers that explains whether they can just build something that conforms to the abstract model without also formalizing and posting a "profile" for it. I don't think that should be required at all. I do think it would be reasonable to consider what would be required to make a web API usable programmatically. This is typically accomplished by documenting the API itself. I wonder if there is some more specific approach that would be more helpful for systems attempting to harvest data, though. It might be something closer to a list of URIs that meet the requirements of the abstract model. |
|
@agreiner - a client will need to find a statement somewhere that says your API implements the abstract model - so whatever the subject of that statement is a "profile" - no extra work is needed really. "what would be required to make a web API usable programatically" sounds like a difficult scope to encompass - can you be specific about what that means from a Conneg-by-ap perspective? |
What environment are your developers likely to build for? If it's the "HTTP environment", then the HTTP Functional Profile may be all they need. If it's GraphQL, and that's seen by you/them as a separate environment, then they'll have to implement a new Functional Profile that defines how the Abstract Specification is implemented in that environment. I expect that new environment implementers may often implement both HTTP and some other new Functional Profile. This is what I've done with pyLDAPI and the demo implementation at http://conneg.info/mediatypes but for CKAN work, I can't emit all the required HTTP headers since the system's behind layers of proxies that knock them out so I can't make it conformant with the HTTP spec and thus it's only going to be conformant with QSA Alternate Keywords FP. I do expect to make a GraphQL Environment FP as I'm interested in GraphQL unles someone makes one first, and again, implementations conforming to that spec are likely to conform to HTTP if the implementing system can also emit HTTP headers (I guess depends the particular implementation). So, what's the environment you're most likely to implement in? If HTTP, QSA or QSA Alt Keywords, there's already an FP for you (which you might disagree with and make another) but if another environment, what is it? |
|
From what @nicholascar says, it sounds like there is a fair amount of extra work needed, that one would need to write up a detailed specification of the general approach, as we are supplying for QSA. That would be work in addition to documenting the API calls themselves. Where I work, we have developers using React and GraphQL, others using REST (often with clean URLs, or avoiding query strings except for media types), others using RPC, maybe even some still using SOAP. And there will be others I haven't thought of, or that haven't yet been invented. My point is that there are going to be many ways to do conneg by profile. Some people may want to implement it only slightly differently, and the approach we have right now permits that. What happens, then, when you want to write a program to harvest data according to multiple profiles automatically? Asking the programmer to test for and handle all the possibilities doesn't strike me as feasible. But if there were some standard way of indicating what options are available, something closer to documentation for the API than a generalized spec, then we could have something workable. Maybe it would be an extension to OAI-PMH. |
|
I would love to see an extension to OAI-PMH, which I have used with clients and and created servers for, to allow it to be conformant with ConnegP. I do hope to have a project (paid) requirement to do this, perhaps around physical samples metadata which use the protocol. |
|
@agreiner with two URL QSA implementations now listed in the Implementation Report can this Issue be closed or is there a specific action you propose? |
|
I think this needs further discussion in the working group. What I'm suggesting is that we develop a standard way for data publishers/developers to express how they are implementing conneg by profile, so that it becomes reasonable to have multiple implementation approaches co-existing. |
|
Could this be addressed by standardising, with use of a required template, systems reports on how they implement ConnegP, such as the implementation notes I prepared for the Media Types Service: https://mediatypes.conneg.info/connegp? There I've itemised all the normative requirements for the Functional Profiles I've implemented in that API and then shown examples of the endpoints fulfilling the requirements. |
|
Yes, something like that, only machine readable. |
|
Need to resolve if this is a "developers" scope problem or something the service needs to express to conform to this specification. If machine readable then a link header may be appropriate - however we need to be careful to discrimiate between metadata about the service and the returned resource. |
I understand the value of offering an example of how to implement selection of a dataset by its profile in ways other than using content negotiation. My expectation has been that we would describe one example of how a system could be designed to enable requests for datasets that implement specific profiles. I would expect that to appear in the profiles guidance document. The conneg document, however, currently specifies a method of implementing negotiation (not just selection), which I see little reason to demonstrate. We have a mandate to explain how to use content negotiation, not to explain how to implement the equivalent in some other way. In fact, I think that if people want to enable negotiation by profile, they should simply be using conneg. I see little reason to create an alternative that implements the same features. Also, this exercise should not be normative. I don't believe in specifying architecture choices that are better made in the context of a particular system.
A related nit is that the document draws a distinction between 'HTTP' and 'QSA', which erroneously suggests that using query strings is not using HTTP.
The text was updated successfully, but these errors were encountered: