Add support for Metamorph wildcards.

[blackwinter]

While Fix/Catmandu has its own variant of wildcards (which are only partially implemented currently), we also need Metamorph wildcards to be available (both are mutually compatible; see also WildcardTrie/SimpleRegexTrie).

An alternative might be to implement specialized fixes with MARC paths instead (see also MARCspec).

Functional review: @TobiasNx
Code review: @fsteeg

[TobiasNx]

I think it should be highlighted that the function of metafacture wildcards are different to the catmandu wildcards.

Catmandu Wildcards "are used to point to relative positions or many positions in a list"..

Metafactue are used to target multiple variations of path names not only the position in a list.

[blackwinter]

Metafactu[r]e [wildcards] are used to target multiple variations of path names not only the position in a list.

Indeed, thank you.

[TobiasNx]

I added a couple of functional test, they seem fine. Except for Test 2. This seems to be an odd behaviour concerning copy_field. Instead of overwriting the field "animal" it creates an object with two instances. I created a ticket for this: #99

https://gitlab.com/oersi/oersi-etl/-/tree/FunctionalReview/data/experimental/wildCardTest

[blackwinter]

Thanks! So that's a 👍 from you w.r.t. wildcard support?

[TobiasNx]

I added a sixt test. That also looks if a asteriks truncation also includes all the subordinated fields and values of arrays, but it seems not. Perhaps that is something that we do not need, but this was supported by metafacture morph.

Output is:

{
  "animals" : [ "dog", "cat", "zebra" ],
  "animal" : "bunny",
  "animols" : {
    "name" : "bird",
    "type" : "TEST"
  },
  "ANIMALS" : "dragon and unicorn",
  "stringimals" : [ "bunny" ]
}

Expected:

{
  "animals" : [ "dog", "cat", "zebra" ],
  "animal" : "bunny",
  "animols" : {
    "name" : "bird",
    "type" : "TEST"
  },
  "ANIMALS" : "dragon and unicorn",
  "stringimals" : [ "bunny", "dog", "cat", "zebra", "bird", "TEST" ]
}

https://gitlab.com/oersi/oersi-etl/-/blob/FunctionalReview/data/experimental/wildCardTest/wildcard_test6.flux

If that is something for a later stage or something we do not need I would say +1.

[blackwinter]

That also looks if a asteriks truncation also includes all the subordinated fields and values of arrays, but it seems not.

That's right, I didn't do this - more or less - intentionally. It would break compatibility with Fix/Catmandu (Metamorph-* would have a different meaning than Fix-*) and it's also not required for us ATM.

Perhaps that is something that we do not need, but this was supported by metafacture morph.

This will be our eternal dilemma ;) It's impossible to merge two mature projects of any kind without some losses. We can try to reconcile wherever possible as much as possible, but it's never going to be perfect.

[TobiasNx]

Perhaps that is something that we do not need, but this was supported by metafacture morph.

This will be our eternal dilemma ;) It's impossible to merge two mature projects of any kind without some losses. We can try to reconcile wherever possible as much as possible, but it's never going to be perfect.

It does not have to be an dilemma, sometimes is is good :D. It is something that I did not like in "old" metafacture morph.

[blackwinter]

FTR: Wildcard/regex matching does not (currently) extend to virtual fields (#63). They always have to be selected explicitly.