KAFKA-18465: Remove MetadataVersions older than 3.0-IV1 #18468
Conversation
ijuma
force-pushed
the
kafka-18465-remove-metadata-version-methods-for-versions-older-than-3.0
branch
from
1097696 to
c557e0b
Compare
| if (version < 3) | ||
| assertEquals(OffsetsForLeaderEpochRequest.DEBUGGING_REPLICA_ID, parsed.replicaId()); | ||
| else | ||
| assertEquals(replicaId, parsed.replicaId()); |
There was a problem hiding this comment.
Should we keep the test for version=4 ?
There was a problem hiding this comment.
The value of this test is a bit low, but I added it back.
| @@ -731,7 +731,6 @@ private void cancelClassicGroupSizeCounter() { | |||
| public void onLoaded(MetadataImage newImage) { | |||
| MetadataDelta emptyDelta = new MetadataDelta(newImage); | |||
| groupMetadataManager.onNewMetadataImage(newImage, emptyDelta); | |||
| offsetMetadataManager.onNewMetadataImage(newImage, emptyDelta); | |||
There was a problem hiding this comment.
I guess we don't need to clean up GroupMetadataManager.scala, as 4.0 uses a new coordinator by default?
There was a problem hiding this comment.
I updated that class too where it made sense. Did you spot additional areas of clean-up?
| @@ -238,7 +238,7 @@ public static CoordinatorRecord newGroupMetadataRecord( | |||
| ), | |||
| new ApiMessageAndVersion( | |||
| value, | |||
| metadataVersion.groupMetadataValueVersion() | |||
There was a problem hiding this comment.
Could you please remove variable MetadataVersion metadataVersion as it is unused now?
| ) { | ||
| short version = metadataVersion.offsetCommitValueVersion(offsetAndMetadata.expireTimestampMs.isPresent()); |
There was a problem hiding this comment.
It seems we need to use version=1 in order to keep "expireTimestamp". otherwise, in v3 expireTimestamp will be ignored in serialization.
There was a problem hiding this comment.
Yeah, this change is wrong - a few things will be reverted as a result.
There was a problem hiding this comment.
Fixed this one - still working through a few other things.
|
Thanks for the quick review @chia7712. I should have left a note, but I'm still working on this PR. I'll take a look at your comments soon. |
Apache Kafka 4.0 will only support KRaft and 3.0 was the first release where KRaft was in a reasonable state (even if not production ready yet). So, we can assume that Apache Kafka 4.0 will only communicate with brokers that are 3.0 or newer. A couple of notes: 1. KRaft was only marked as production-ready in 3.3, so we could go further and set the baseline to 3.3. I think we should have that discussion, but it made sense to start with the non controversial parts. 2. I have not removed the actual IBP definitions - leaving that for a separate PR.
ijuma
force-pushed
the
kafka-18465-remove-metadata-version-methods-for-versions-older-than-3.0
branch
from
c557e0b to
7449859
Compare
ijuma
changed the title
github-actions
bot
added
tools
performance
storage
|
I removed the relevant IBP constants too and fixed the known test failures. I'll check the full test results tomorrow and will do a file by file pass then too. |
Apache Kafka 4.0 will only support KRaft and 3.0-IV1 is the minimum version supported by KRaft. So, we can assume that Apache Kafka 4.0 will only communicate with brokers that are 3.0-IV1 or newer.
Note that KRaft was only marked as production-ready in 3.3, so we could go further and set the baseline to 3.3. I think we should have that discussion, but it made sense to start with the non controversial parts.
Committer Checklist (excluded from commit message)