MINOR: Update Consumer and Producer JavaDocs for committing offsets #18336
+25
−9
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The consumer/producer JavaDocs still contain instruction for naively computing the offset to be committed. This PR updates the JavaDocs with regard to the improvements of KIP-1094.
AndrewJSchofield
requested changes
Dec 28, 2024
| * which is at position 5 has consumed records with offsets 0 through 4 and will next receive the record with offset 5. There | ||
| * are actually two notions of position relevant to the user of the consumer: | ||
| * which is at position 5 has consumed records with offsets 0 through 4 and will next receive the record with offset 5. | ||
| * Note that offsets are not guaranteed to be consecutive (eg., for compacted topic, or—independent of "read_committed" |
There was a problem hiding this comment.
I suggest the following the parentheses "(such as compacted topic or when records have been produced using transactions)".
| * which is at position 5 has consumed records with offsets 0 through 4 and will next receive the record with offset 5. | ||
| * Note that offsets are not guaranteed to be consecutive (eg., for compacted topic, or—independent of "read_committed" | ||
| * mode— transactional topics). For example, if the consumer did read a record with offset 4, but 5 is not an offset | ||
| * with a record, it's position might advance to 6 (or higher) directly. Similarly, if the consumer's position is 5, |
| @@ -984,7 +990,9 @@ public void commitSync(Duration timeout) { | |||
| * This commits offsets to Kafka. The offsets committed using this API will be used on the first fetch after every | |||
| * rebalance and also on startup. As such, if you need to store offsets in anything other than Kafka, this API | |||
| * should not be used. The committed offset should be the next message your application will consume, | |||
| * i.e. lastProcessedMessageOffset + 1. If automatic group management with {@link #subscribe(Collection)} is used, | |||
| * i.e. {@code nextRecordToBeProcessed.offset()} (or {@link ConsumerRecords#nextOffsets()}). | |||
| * You should also add the {@link ConsumerRecord#leaderEpoch()} (or {@code nextOffsets().get(...).leaderEpoch()}) | |||
There was a problem hiding this comment.
Maybe "You should also add the leader epoch as commit metadata, which can be obtained from {@link ConsumerRecord#leaderEpoch()} or {@link ConsumerRecords#nextOffsets}." I didn't find the nextOffsets().get(...).leaderEpoch() that easy to follow and the hyperlink to the ConsumerRecords seems nicer to me.
| @@ -1033,7 +1041,9 @@ public void commitSync(final Map<TopicPartition, OffsetAndMetadata> offsets) { | |||
| * This commits offsets to Kafka. The offsets committed using this API will be used on the first fetch after every | |||
| * rebalance and also on startup. As such, if you need to store offsets in anything other than Kafka, this API | |||
| * should not be used. The committed offset should be the next message your application will consume, | |||
| * i.e. lastProcessedMessageOffset + 1. If automatic group management with {@link #subscribe(Collection)} is used, | |||
| * i.e. {@code nextRecordToBeProcessed.offset()} (or {@link ConsumerRecords#nextOffsets()}). | |||
| * You should also add the {@link ConsumerRecord#leaderEpoch()} (or {@code nextOffsets().get(...).leaderEpoch()}) | |||
There was a problem hiding this comment.
Same point as above about nextOffsets().get(...).leaderEpoch().
| @@ -1117,7 +1127,9 @@ public void commitAsync(OffsetCommitCallback callback) { | |||
| * This commits offsets to Kafka. The offsets committed using this API will be used on the first fetch after every | |||
| * rebalance and also on startup. As such, if you need to store offsets in anything other than Kafka, this API | |||
| * should not be used. The committed offset should be the next message your application will consume, | |||
| * i.e. lastProcessedMessageOffset + 1. If automatic group management with {@link #subscribe(Collection)} is used, | |||
| * i.e. {@code nextRecordToBeProcessed.offset()} (or {@link ConsumerRecords#nextOffsets()}). | |||
| * You should also add the {@link ConsumerRecord#leaderEpoch()} (or {@code nextOffsets().get(...).leaderEpoch()}) | |||
| * be the next message your application will consume, i.e. lastProcessedMessageOffset + 1. | ||
| * be the next message your application will consume, i.e. {@code nextRecordToBeProcessed.offset()} | ||
| * (or {@link ConsumerRecords#nextOffsets()}). You should also add the {@link ConsumerRecord#leaderEpoch()} | ||
| * (or {@code nextOffsets().get(...).leaderEpoch()}) as commit metadata. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The consumer/producer JavaDocs still contain instruction for naively computing the offset to be committed.
This PR updates the JavaDocs with regard to the improvements of KIP-1094.