Collect & add more Playground examples

[fsteeg]

Small examples with different source and target formats, e.g. MARC-to-JSON, JSON-to-JSON, HTML-to-JSON.

• Update existing sample
• MARC-to-JSON
• JSON-to-JSON
• HTML-to-JSON
• OAI-PMH-MODS-to-JSON
• Dropdown in UI, see comment

We can discuss these with Playground links here in the ticket.

Functional reviewer: @TobiasNx

Code reviewer: @fsteeg

[TobiasNx]

MarcXML as file in web -> JSON (does not need any as-lines or as-record)

http://test.lobid.org/playground/process?flux=%22https%3A//d-nb.info/1106253078/about/marcxml%22%0A%7C+open-http%28accept%3D%22application/xml%22%29%0A%7C+decode-xml%0A%7C+handle-marcxml%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%0A%3B

[TobiasNx]

JSON -> JSON with small FIX (but needs newer version of FIX, since it cannot process wildcards in path's) Should come back to this example.

https://test.metafacture.org/playground/?flux=%22https%3A//lobid.org/gnd/11942150X%22%0A%7C+open-http%28accept%3D%22application/json%22%29%0A%7C+as-lines%0A%7C+decode-json%0A%7C+fix%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%3B&fix=copy_field%28%22preferredName%22%2C+%22name%22%29%0Acopy_field%28%22affiliation%5B%5D.1.label%22%2C+%22placeOfWork%22%29%0Aretain%28%22name%22%2C+%22placeOfWork%22%29&active-editor=fix

[fsteeg]

As an idea for a simple, yet configurable integration in the playground UI:

Provide each sample in a text file, load files in playground to populate 'Load sample' drop down with examples, using file name of the text file, e.g. marc_to_json. Examples can then be configured / maintained by adding / editing these text files.

[TobiasNx]

OAI-PMH - MODS -> JSON:

(could be a little nicer since the OAI PMH specific metadata is carried around.)

https://test.metafacture.org/playground/?flux=%22https%3A//duepublico2.uni-due.de/oer/oai%22%0A%7C+open-oaipmh%28metadataPrefix%3D%22mods%22%29%0A%7C+decode-xml%0A%7C+handle-generic-xml%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%3B&fix=copy_field%28%22preferredName%22%2C+%22name%22%29%0Acopy_field%28%22affiliation%5B%5D.1.label%22%2C+%22placeOfWork%22%29%0Aretain%28%22name%22%2C+%22placeOfWork%22%29&active-editor=fix

[fsteeg]

As a naming convention for the example files, maybe we could use something like <where-we-read-from>_<what-we-decode>_to_<what-we-encode>, e.g. http_json_to_json or oai-pmh_mods_to_json.

[katauber]

Deployed to https://test.metafacture.org/playground/

[katauber]

@TobiasNx I installed again to https://test.metafacture.org/playground/ with these changes:

1. When loading an example and process it and then switch to a new example the result box is not cleared.

AFter loading a new example, the result box is cleared now.

2. The MarcXML-to-JSON example does not seem to work, only if you select the morph-option. This is weird since the flux does not need any fix or morph.

Yes, you found a bug! I fixed this.

4. There should be some kind of tool tip concerning the PG_Data. It should be some explained.

I did not implement something for this. Where and when do you think is the best place for such a tooltip? I am not really sure, how to solve this. If we need this explained, we maybe need also explained how to use the fix and morph editor?

[TobiasNx]

+1 Now it seems to work.
Maybe @dr0i can say something about the Pica-to-Marc workflow that does not work, which is build from the examples in the runner.

Are they too old?

I updated and added some examples. @dr0i could you ave a look at https://github.com/metafacture/metafacture-playground/blob/25-examples/resources/examples/Web_Pica-to-MarcXML_(morph) why this workflow does not process? It is copied from:

https://github.com/metafacture/metafacture-core/tree/master/metafacture-runner/src/main/dist/examples/morph/pica-to-marcxml
See here in Playground

[dr0i]

@TobiasNx workflow seems fine - at least it is processed when executed with flux.sh. The logs at the server (only found nohup.out) just logs Processed flux file. - and that's it.
Have to dig a bit more how to log/debug the playground...

[dr0i]

Deployed the fix (metafacture/metafacture-core#435) to test: http://test.lobid.org/playground/.

[dr0i]

Note: changed metafacture-core-dependencies at productive server in project.clj to master-SNAPSHOT (and pulled the metafacture-core before and installed it locally to ~/.m2 ). So, if 25-examples will be merged into main the playground can be deployed by pulling and restarting it - the fix in metafacture-core will be part of the redeployment.

[dr0i]

@TobiasNx in the output the marc record's attribute type is missing, like <marc:record type="Bibliographic">. Don't know if this is kind of mandatory resp. if we should not provide it, as a good example. Probably there is something missing in the morph.

[TobiasNx]

As I stated here: metafacture/metafacture-core#402 (comment)

type is optional: https://www.loc.gov/standards/marcxml/xml/spy/spy.html#element_record_Link044CBAB8
it can have the following values: https://www.loc.gov/standards/marcxml/xml/spy/spy.html#simpleType_recordTypeType_Link01A148C8

Since it is not mandatory we do not need to do this, but we would need to add this as part of the morph.

[katauber]

Please re-review after functinal changes in code review @TobiasNx

[TobiasNx]

+1