Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-14123: support kafka tombstone with USE_WRAPPER strategy #9604

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

babubabu
Copy link

@babubabu babubabu commented Dec 30, 2024

Add possibility to leave out the "value" attribute in flowfile wrapper to support publishing kafka tombstones. With this solution, we support a wrapper either completely without the "value" attribute or with "value: null". The JSON Reader and AVRO Set writer support null values out-of-the-box

Summary

NIFI-14123

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using mvn clean install -P contrib-check
    • JDK 21

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

@greyp9
Copy link
Contributor

greyp9 commented Jan 1, 2025

Hi @babubabu; thanks for the contribution! Looks like the "integration-tests" CI failed on this PR for the new test that was added. You can easily run this single test (after a full build) with a command line something like this:

nifi/nifi-extension-bundles/nifi-kafka-bundle % mvn verify -P integration-tests -pl nifi-kafka-3-integration -Dit.test=PublishKafkaWrapperX6IT

@babubabu
Copy link
Author

babubabu commented Jan 1, 2025

Hi @greyp9 ,
Thanks for your response. Yes I saw it, but couldn't fix it easily yesterday. I'm on it at 2nd Jan ;-)
Happy new year :-)

@babubabu babubabu force-pushed the NIFI-14123 branch 2 times, most recently from fd1580f to 27a26b9 Compare January 1, 2025 18:24
@babubabu
Copy link
Author

babubabu commented Jan 1, 2025

Hello @greyp9,
I fixed the test setup, now it should work ;-)
Can you please review it again?

Add possibility to leave out the "value" attribute in flowfile wrapper to support publishing kafka tombstones.
With this solution, we support a wrapper either completely without the "value" attribute or  with "value: null".
The JSON Reader and AVRO Set writer support null values out-of-the-box
@babubabu
Copy link
Author

babubabu commented Jan 2, 2025

Fixed checkstyle violations and rebased to current main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants