Skip to content
This repository has been archived by the owner on Jan 27, 2025. It is now read-only.

Add support for Metamorph wildcards. #97

Merged
merged 5 commits into from
Dec 22, 2021

Conversation

blackwinter
Copy link
Member

Delegates Value.Hash retrieval to SimpleRegexTrie for pattern matching.

All operations involving field patterns must be translated to concrete field names.

NOTE: Requires (a lot) more tests. But existing tests, including previously disabled ones, are already providing some coverage - with trivial (?) modifications to make them pass (not sure why they were testing the wrong order).

Resolves #89.

Delegates `Value.Hash` retrieval to `SimpleRegexTrie` for pattern matching.

All operations involving field patterns must be translated to concrete field names.
@blackwinter
Copy link
Member Author

The bulk of the effort went into transformFields(). There may still be some pitfalls left.

@blackwinter
Copy link
Member Author

Also, transformFields() and find()/findNested() are basically performing the same traversal operations. There may be room for consolidation.

Copy link
Member

@fsteeg fsteeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nice, so we're using the trie for pattern matching only, not for storing the actual values. I was expecting this to require a larger change, but this looks very good 👍

Is my understanding correct that right now, we're creating a new trie with a single value for every lookup? I think we could make the trie a member of Hash to avoid multiple instantiations.

I think the wrong order in the disabled tests was from a previous state, where we were not using LinkedHashMap for the record map.

@fsteeg fsteeg assigned blackwinter and unassigned fsteeg Dec 21, 2021
@blackwinter
Copy link
Member Author

blackwinter commented Dec 21, 2021

Is my understanding correct that right now, we're creating a new trie with a single value for every lookup? I think we could make the trie a member of Hash to avoid multiple instantiations.

Yes, your understanding is correct - and yes, we might be able to make it a member. I had a (hypothetical) case where that would've failed, but that might've been before I started checking for the actual pattern (trie.get(field).contains(pattern) instead of !trie.get(field).isEmpty()). I'll think about it a little more.

Cannot lead to false positives since the presence of the actual pattern is verified.
@blackwinter
Copy link
Member Author

Alright, I've changed the trie to a member (432ef88). Should be fine™ ;)

@blackwinter
Copy link
Member Author

Ah, nice, so we're using the trie for pattern matching only, not for storing the actual values.

FTR, we can't use SimpleRegexTrie as a key/value store because it's meant for the reverse use case as a Registry for NamedValueReceivers: It stores patterns (Metamorph lookups) and then retrieves values by concrete field name (metadata stream).

@blackwinter blackwinter merged commit b575105 into master Dec 22, 2021
@blackwinter blackwinter deleted the 89-addSupportForMetamorphWildcards branch December 22, 2021 10:57
blackwinter added a commit that referenced this pull request Apr 11, 2022
…207, #97)

`SimpleRegexTrie` is only used to determine whether a field name matches a pattern. The result of this computation only depends on these two values, not on the underlying data. So we can cache and reuse the result across the whole transformation process.

It might be possible to implement the matching in a more optimized way for this particular use case, but it's delegated to `SimpleRegexTrie` for compatibility and maintainability purposes.
blackwinter added a commit that referenced this pull request May 4, 2022
…s. (#97, 90cbde4)

Runs potential risk of diverging from `WildcardTrie`/`SimpleRegexTrie` implementation.
blackwinter added a commit that referenced this pull request May 13, 2022
Still a hotspot so we employ a bunch of additional strategies:

- Avoid `Stream` API in hot path.
- Sidestep `WildcardTrie` w.r.t. alternations.
- Guard `SimpleRegexTrie` matching with prefix match.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Metamorph wildcards.
2 participants