This is an automated email from the ASF dual-hosted git repository. cmccabe pushed a change to branch KAFKA-16649 in repository https://gitbox.apache.org/repos/asf/kafka.git
discard f435287ac63 Fixes omit 5a7a9b7f213 KAFKA-16649: Remove lock from DynamicBrokerConfig.removeReconfigurable add 31355ef8f94 KAFKA-16475: add more tests to TopicImageNodeTest (#15735) add 4825c89d14e KAFKA-16588 broker shutdown hangs when log.segment.delete.delay.ms is zero (#15773) add d9c36299db7 KAFKA-16614 Disallow @ClusterTemplate("") (#15800) add 89d8045a15b KAFKA-16647 Remove setMetadataDirectory from BrokerNode/ControllerNode (#15833) add cdc4caa5787 KAFKA-14588 UserScramCredentialsCommandTest rewritten in Java (#15832) add a3f24149905 KAFKA-16624: Don't generate useless PartitionChangeRecord on older MV (#15810) add 87390f961f3 KAFKA-16572 allow defining number of disks per broker in ClusterTest (#15745) add 240243b91d6 KAFKA-10199: Accept only one task per element in output queue for failed tasks (#15849) add 2c0b8b69207 MINOR: ConsumerGroup#getOrMaybeCreateMember should not add the member to the group (#15847) add 1fd39150aa3 KAFKA-16655: Deflake ZKMigrationIntegrationTest.testDualWrite #15845 add 9b8aac22ec7 KAFKA-16427 KafkaConsumer#position() does not respect timeout when group protocol is CONSUMER (#15843) add bfe81d62297 KAFKA-16207; KRaft's internal log listener to update voter set (#15671) add 25118cec145 MINOR: remove redundant check in KafkaClusterTestKit (#15858) add 970ac078812 KAFKA-16659 KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER (#15853) add 41f5bf844df KAFKA-16223 Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest (2/3) (#15841) add 55a00be4e97 MINOR: Replaced Utils.join() with JDK API. (#15823) add 366aeab488c KAFKA-10199: Add remove operation with future to state updater (#15852) add 6a8977e2125 KAFKA-14588 [3/N] ConfigCommandTest rewritten in java (#15850) add 5c96ad61d95 KAFKA-16393 read/write sequence of buffers correctly (#15571) add 42754336e1f MINOR: Remove `ConsumerGroupPartitionMetadataValue.Epoch` field (#15854) add 0b4eaefd863 KAFKA-16646: KAFKA-16646Don't run cve scan job on forks (#15831) add aeca384641b KAFKA-16356: Remove class-name dispatch in RemoteLogMetadataSerde (#15620) add 4c4ae6e39c4 KAFKA-16608 Honour interrupted thread state on KafkaConsumer.poll (#15803) add fe8ccbc92c2 KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration (#15744) add 0df340d64d3 KAFKA-16470 kafka-dump-log --offsets-decoder should support new records (#15652) add 0de3b7c40b0 KAFKA-16593 Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions (#15766) add d76352e2151 MINOR: log newly created processId (#15851) add 459eaec666c KAFKA-16615; JoinGroup API for upgrading ConsumerGroup (#15798) add ea485a70611 KAFKA-16665: Allow to initialize newly assigned partition's positions without allowing fetching while callback runs (#15856) add cb35ddc5ca2 KAFKA-10199: Remove lost tasks in state updater with new remove (#15870) add 21bf715622e KAFKA-16307; Fix coordinator thread idle ratio (#15835) add 05df10449eb KAFKA-13328, KAFKA-13329 (2): Add custom preflight validation support for connector header, key, and value converters (#14309) add a4c6cefd10f KAFKA-14226: Introduce FieldPath abstraction and nested path support for ExtractField SMT (#15379) add 525b9b1d768 KAFKA-15018: Write connector tombstone offsets to secondary store before primary store (#13801) add a0f1658bb13 KAFKA-16678 Remove variable "unimplementedquorum" (#15879) add 5f933ac8403 MINOR: Correct connector scheduled rebalance logs (#15875) add 8655094e6c9 KAFKA-16511: Fix the leaking tiered segments during segment deletion (#15817) add 3b43edd7a1c MINOR: Remove dev_version parameter from streams tests (#15874) add f7b242f94e8 KAFKA-10199: Revoke tasks from state updater with new remove (#15871) add f74f596bc7d KAFKA-16640 Replace TestUtils#resource by scala.util.Using (#15881) add 2a5efe4a334 KAFKA-16685: Add parent exception to RLMTask warning logs (#15880) add c64a315fd55 MINOR: Made the supportedOperation variable name more verbose (#15892) add 29f3260a9c0 MINOR: Fix streams javadoc links (#15900) add 397d58b14c9 MINOR: use classic consumer with ZK mode for DeleteOffsetsConsumerGroupCommandIntegrationTest (#15872) add f4fdaa702a2 MINOR: Add missing RPCs to security.html (#15878) add 5a9ccb6b773 KAFKA-16445: Add PATCH method for connector config (#6934) add 8fd6596454e KAFKA-15045: (KIP-924) New interfaces and stubbed utility classes for pluggable TaskAssignors. (#15887) add 7e9ab4b2c68 KAFKA-16484 Support to define per broker/controller property by ClusterConfigProperty (#15715) add ff6d01c90fc KAFKA-15951: MissingSourceTopicException should include topic names (#15573) add 31528f581d0 KAFKA-15307: update/note deprecated configs (#14360) add 4bece0131fd KAFKA-14133 Move StreamTaskTest to Mockito (#14716) add 58c7369be2b KAFKA-16660 reduce the check interval to speedup DelegationTokenRequestsTest (#15907) add 510431a732d KAFKA-16688: Use helper method to shutdown ExecutorService (#15886) add 147ea55dfe2 MINOR: correct KAFKA_HEAP_OPTS server property in KafkaDockerWrapper(#15345) add 0cfc2983a58 KAFKA-16684: Fix flaky DedicatedMirrorIntegrationTest (#15906) add 4e4f7d3231b KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing (#14729) add ef7b48e66ab Allowing WriteTxnMarkers API to run with AlterCluster permissions (#15837) add 4dff60df678 MINOR: fix LogValidatorTest#checkNonCompressed (#15904) add 47841e0bb96 KAFKA-9401 Reduce contention for Fetch requests (#15836) add 643db430a70 KAFKA-16679 merge unit test down to the class of integration test (#15884) add 334d5d58bb7 KAFKA-16677 Replace ClusterType#ALL and ClusterType#DEFAULT by Array (#15897) add 8a9dd2beda9 KAFKA-16663; Cancel write timeout TimerTask on successful event completion (#15902) add cfffe4e2e85 KAFKA-10199: Handle assignment with new remove operation in state updater (#15882) add ee16eee5deb KAFKA-16587: Add subscription model information to group state (#15785) add 576facfdf22 KAFKA-16696 Removed the in-memory implementation of RSM and RLMM (#15911) add 6161fd0db28 KAFKA-14588 [2/N] ConfigCommandTest rewritten in java (#15873) add 5439914c32f KAFKA-10199: Shutdown with new remove operation in state updater (#15894) add f9169b7d3a9 KAFKA-16735; Deprecate offsets.commit.required.acks (#15931) add d61b34f2a7d KAFKA-14509: [3/4] Add integration test for consumerGroupDescribe API (#15727) add e18f61ce46a KAFKA-16695: Improve expired poll logging (#15909) add de105a8c140 KAFKA-16703 Close serverChannel in SocketServer if unable to bind to a port (#15923) add c1e7aa77dee MINOR: Move `group-coordinator` import control to its own file (#15934) add ccd83cafea6 KAFKA-16694; Remove Rack Awareness Code from the Server Side Assignors (#15903) add e1771448cf1 MINOR: Add coverage for MM2 heartbeats topic detection (#15932) add 0587a9af3d3 MINOR: Various cleanups in clients tests (#15877) add d59336a6156 MINOR: Use ClusterTemplate in ApiVersionsRequestTest (#15936) add 8ac32d6c194 KAFKA-16702 Fix producer leaks in KafkaLog4jAppenderTest (#15922) add 440f5f6c097 MINOR; Validate at least one control record (#15912) add 2958dcb9191 KAFKA-13115; Update doSend doc about possible blocking (#11023) add df5735dda52 MINOR: fix flaky testRecordThreadIdleRatioTwoThreads test (#15937) add 57d30d34509 KAFKA-16699: Have Streams treat InvalidPidMappingException like a ProducerFencedException (#15919) add 0e023e1f736 MINOR: Add classic member session timeout to ClassicMemberMetadata (#15921) add f0291ac74b8 KAFKA-15170: Fix rack-aware assignment in AbstractStickyAssignor (#13965) add 0c5e8d39660 KAFKA-15045: (KIP-924 pt. 2) Implement ApplicationState and KafkaStreamsState (#15920) add cb968845ecb KAFKA-15045: (KIP-924 pt. 3) Implement KafkaStreamsAssignment (#15944) add 74535493147 KAFKA-16671 enable test for ensureInternalEndpointIsSecured (#15868) add 34ec3fac156 MINOR: Fix warnings in streams javadoc (#15955) add 1e427c029e3 MINOR: Fix TargetAssignmentBuilderBenchmark (#15950) add eb5559a40e6 KAFKA-16686 Wait for given offset in TopicBasedRemoteLogMetadataManagerTest (#15885) add bf88013a281 MINOR: Rename `Record` to `CoordinatorRecord` (#15949) add d2e6c866326 KAFKA-10199: Remove queue-based remove from state updater (#15896) add a4952572dc1 MINOR: Change sys test describe topic parsing to improve extensibility (#15941) add c218c4e1b52 KAFKA-16287: Implement example tests for common rebalance callback (#15408) add 3f8d11f047b KAFKA-16577: New consumer fails with stop within allotted timeout in consumer_test.py system test (#15784) add bb3ff0f84a0 KAFKA-16759: Handle telemetry push response while terminating (#15957) add aca5d249d60 MINOR: revisit LogValidatorTest#checkRecompression (#15948) add ba19eedb90f KAFKA-7342: Migrate tests in remaining packages in o.a.k.streams (#15963) add ffb31e172ac MINOR: Remove usage of Stream API in CoordinatorRecordHelpers (#15969) add a1c2c68db11 KAFKA-16712 Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest (#15962) add 5da4b238d68 MINOR: Remove unused method in ToolsUtils (#15967) add dac569b9671 KAFKA-16668 Add tags support in ClusterTestExtension (#15861) add 7b1fe33d01c KAFKA-14588 [3/N] ConfigCommandTest rewritten in java (#15930) add 2c515946076 MINOR: rewrite TopicBasedRemoteLogMetadataManagerTest by ClusterTestExtensions (#15917) add 056d232f4e2 KAFKA-16526; Quorum state data version 1 (#15859) add fafa3c76dc9 KAFKA-15045: (KIP-924 pt. 4) Generify rack graph solving utilities (#15956) add c58c21cc208 KAFKA-16774: Delete flaky test since it is redundant (#15978) add f9db4fa19cc KAFKA-16787: Remove TRACE level logging from AsyncKafkaConsumer hot path (#15981) add b8c96389b47 KAFKA-16762: SyncGroup API for upgrading ConsumerGroup (#15954) add 7fea279ff99 KAFKA-16763 Upgrade to scala 2.12.19 and scala 2.13.14 (#15958) add 5b34574e862 MINOR: Refactor write timeout in CoordinatorRuntime (#15976) add 9896d079972 MINOR: Add debug enablement check when using log.debug (#15977) add 5fa48214448 MINOR: use try-with resources in ClientMetricsManagerTest (#15982) add 69fc4c5da41 MINOR: Migrate tests in o.a.k.streams to JUnit 5 (except KafkaStreamsTest) (#15942) add 92489995f3e KAFKA-16544 DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames should return null instead of throwing NPE (#15979) add 22f5113dba4 KAFKA-15723 KRaft support in ListOffsetsRequestTest (#15980) add 93a5efc4b72 KAFKA-16223 Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest (#15933) add 6aac009a2dc MINOR: Remove deprecated constructors from Connect's Kafka*BackingStore classes (#15865) add 3e15ab98ec6 KAFKA-16992: InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka (#15971) add 412b05df001 KAFKA-16789 Fix thread leak detection for event handler threads (#15984) add a0ca025f9d0 MINOR: Update upgrade documentation for 3.7.1 (#15866) add 81e60980218 KAFKA-16797 A bit cleanup of FeatureControlManager (#15997) add 95adb7bfbfc MINOR: ensure KafkaServerTestHarness::tearDown is always invoked (#15996) add 476d323f5a6 KAFKA-16197: Print Connect worker specific logs on poll timeout expiry (#15305) add b4c2d668014 KAFKA-16770; [1/N] Coalesce records into bigger batches (#15964) add c10bb58d1c9 KAFKA-14588 [4/N] ConfigCommandTest rewritten in java (#15839) add 52b4596dae2 KAFKA-16675: Refactored and new rebalance callbacks integration tests (#15965) add b5a013e4564 KAFKA-16513; Add test for WriteTxnMarkers with AlterCluster permission add 1dcdccf7366 MINOR: fix streams_broker_compatibility test (#16015) add 89083520efb KAFKA-16654 Refactor kafka.test.annotation.Type and ClusterTestExtensions (#15916) add 9fe3932e5c1 KAFKA-16784 Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to use ClusterTestExtensions (#15992) add affe8da54c9 KAFKA-7632: Support Compression Levels (KIP-390) (#15516) add 4cc99cbf3f3 KAFKA-16343: Add unit tests of foreignKeyJoin classes (#15564) add 6339e3a6bf5 KAFKA-15045: (KIP-924 pt. 6) Post process new assignment structure (#16002) add 7de58f7359a KAFKA-16739: Exclude protected members from aggregated release javadocs (#15940) add e93aae0664f KAFKA-16783: Migrate RemoteLogMetadataManagerTest to new test infra (#15983) add 271c04bd176 KAFKA-15444: Native docker image for Apache Kafka (KIP-974) (#15927) add 5552f5c26df KAFKA-15541: Add num-open-iterators metric (#15975) add 028e7a06dcd MINOR: fix incorrect formatter package in streams quickstart (#15991) add a98c9be6b00 KAFKA-15974: Enforce that event processing respects user-provided timeout (#15640) add e4e1116156d MINOR: Move Throttler to storage module (#16023) add a753172ad3e KAFKA-12399: Deprecate KafkaLog4jAppender (#15985) add ef2c5e41a54 KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState (#15972) add 27a6c156c49 KAFKA-15045: (KIP-924 pt. 7) Simplify requirements for rack aware graphs (#16004) add 06739d5aa02 KAFKA-15045: (KIP-924 pt. 8) Added TopicPartitionAssignmentInfo (#16024) add bef83ce89bb KAFKA-15541: Add iterator-duration metrics (#16028) add e692feed34c MINOR: fix flaky testRecordThreadIdleRatio (#15987) add 14b5c4d1e85 KAFKA-16793; Heartbeat API for upgrading ConsumerGroup (#15988) add ab0cc72499e MINOR: Move parseCsvList to server-common (#16029) add 8d117a188df KAFKA-16825: Update netty/jetty/jackson/zstd dependencies (#16038) add 5a4898450d0 KAFKA-15649: Handle directory failure timeout (#15697) add 0ba15ad4d1b KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… (#15910) add c3018ef55ec KAFKA-16804: Replace archivesBaseName with archivesName (#16016) add 4020307ac28 KAFKA-16795 Fix broken compatibility in kafka.tools.NoOpMessageFormatter, kafka.tools.DefaultMessageFormatter, and kafka.tools.LoggingMessageFormatter (#16020) add 93238ae312e KAFKA-15045: (KIP-924 pt. 10) Topic partition rack annotation simplified (#16034) add 11ad5e8bca2 MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions (#15469) add de3202832dc KAFKA-16828 RackAwareTaskAssignorTest failed (#16044) add 6941598f710 KAFKA-16826: Integrate Native Docker Image with github actions (#16045) add 520aa8665c8 KAFKA-16626; Lazily convert subscribed topic names to topic ids (#15970) add c5cd1908189 MINOR: Refactor SSL/SASL admin integration tests to not use a custom authorizer (#15377) add 0143c72e501 KAFKA-16815: Handle FencedInstanceId in HB response (#16047) add 2432a1866e7 KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images (#16027) add 90892ae99fb KAFKA-16516: Fix the controller node provider for broker to control channel add 4f55786a8a8 KAFKA-16515: Fix the ZK Metadata cache confusion between brokers and controllers add 8eea6b82638 MINOR: mention KAFKA-15905 in docs "Notable changes in 3.7.1" (#16070) add d585a494a48 KAFKA-16831: CoordinatorRuntime should initialize MemoryRecordsBuilder with max batch size write limit (#16059) add a8d166c00e6 KAFKA-16625; Reverse lookup map from topic partitions to members (#15974) add d9ee9c96dd5 KAFKA-15541: Use LongAdder instead of AtomicInteger (#16076) add 524ad1e14b1 KAFKA-16452: Don't throw OOORE when converting the offset to metadata (#15825) add da3304ecb6f KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE (#16072) add bac8df56ffd MINOR: fix typo in KAFKA-16515 add adab48df683 MINOR: Disable JDK 11 and 17 tests on PRs (#16051) add 91284d8d7b3 KAFKA-16709: abortAndPauseCleaning only when future log is not existed (#15951) add a57c05b73bd KAFKA-16805 Stop using a ClosureBackedAction to configure Spotbugs reports (#16081) add 699438b7f78 MINOR: Fix the config name in ProducerFailureHandlingTest (#16099) add 64f699aeea6 KAFKA-15853: Move general configs out of KafkaConfig (#16040) add a649bc457fa KAFKA-16711: Make sure to update highestOffsetInRemoteStorage after log dir change (#15947) add 4d04eb83ea2 KAFKA-16796 Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder (#16064) add 4eb60b5104c KAFKA-16507 Add KeyDeserializationException and ValueDeserializationException with record content (#15691) add 59ba5551420 KAFKA-15541: Add oldest-iterator-open-since-ms metric (#16041) add 56ee1392e8f KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor (#16052) add 8d243dfbd41 KAFKA-15045: (KIP-924 pt. 12) Wiring in new assignment configs and logic (#16074) add d64e3fbb2b7 KAFKA-15045: (KIP-924 pt. 13) AssignmentError calculation added (#16114) add 9562143f085 HOTFIX: remove unnecessary list creation (#16117) add eefd114c4ac KAFKA-16832; LeaveGroup API for upgrading ConsumerGroup (#16057) add 897cab2a614 KAFKA-16399: Add JBOD support in tiered storage (#15690) add b73f4798a48 KAFKA-16362: Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide (#15601) add 8d11d957950 KAFKA-16448: Add ProcessingExceptionHandler interface and implementations (#16090) new 32eb8c3d68d KAFKA-16649: Remove lock from DynamicBrokerConfig.removeReconfigurable This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (f435287ac63) \ N -- N -- N refs/heads/KAFKA-16649 (32eb8c3d68d) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/docker_build_and_test.yml | 1 + ...ml => docker_official_image_build_and_test.yml} | 8 +- .github/workflows/docker_promote.yml | 4 +- .github/workflows/docker_rc_release.yml | 3 +- .github/workflows/docker_scan.yml | 1 + ...ml => prepare_docker_official_image_source.yml} | 36 +- Jenkinsfile | 10 +- LICENSE-binary | 8 +- bin/kafka-run-class.sh | 2 +- bin/windows/kafka-run-class.bat | 2 +- build.gradle | 276 +- checkstyle/import-control-core.xml | 4 + checkstyle/import-control-group-coordinator.xml | 82 + checkstyle/import-control-metadata.xml | 10 +- checkstyle/import-control-storage.xml | 2 + checkstyle/import-control.xml | 48 +- checkstyle/suppressions.xml | 14 +- .../apache/kafka/clients/CommonClientConfigs.java | 3 +- .../apache/kafka/clients/FetchSessionHandler.java | 26 +- .../org/apache/kafka/clients/NodeApiVersions.java | 2 +- .../clients/admin/ConsumerGroupDescription.java | 4 +- .../kafka/clients/admin/DescribeTopicsResult.java | 1 + .../kafka/clients/admin/MemberAssignment.java | 4 +- .../kafka/clients/admin/SupportedVersionRange.java | 2 +- .../kafka/clients/admin/TopicDescription.java | 4 +- .../consumer/internals/AbstractCoordinator.java | 18 +- .../consumer/internals/AbstractStickyAssignor.java | 43 +- .../consumer/internals/AsyncKafkaConsumer.java | 240 +- .../clients/consumer/internals/CompletedFetch.java | 47 +- .../consumer/internals/ConsumerNetworkThread.java | 48 +- .../ConsumerRebalanceListenerInvoker.java | 12 +- .../clients/consumer/internals/FetchBuffer.java | 12 +- .../internals/HeartbeatRequestManager.java | 15 +- .../consumer/internals/LegacyKafkaConsumer.java | 6 +- .../consumer/internals/MembershipManagerImpl.java | 6 +- .../consumer/internals/SubscriptionState.java | 11 +- .../clients/consumer/internals/WakeupTrigger.java | 8 +- .../events/AbstractTopicMetadataEvent.java | 5 +- .../internals/events/AllTopicsMetadataEvent.java | 6 +- .../internals/events/ApplicationEventHandler.java | 13 +- .../events/ApplicationEventProcessor.java | 18 +- .../internals/events/BackgroundEventHandler.java | 7 +- .../consumer/internals/events/CommitEvent.java | 6 - .../events/CompletableApplicationEvent.java | 13 +- .../events/CompletableBackgroundEvent.java | 14 +- .../internals/events/CompletableEvent.java | 103 + .../internals/events/CompletableEventReaper.java | 157 + ...nsumerRebalanceListenerCallbackNeededEvent.java | 2 +- .../consumer/internals/events/EventProcessor.java | 124 +- .../events/FetchCommittedOffsetsEvent.java | 5 +- .../internals/events/LeaveOnCloseEvent.java | 6 +- .../internals/events/ListOffsetsEvent.java | 5 +- .../internals/events/ResetPositionsEvent.java | 6 +- .../consumer/internals/events/SyncCommitEvent.java | 5 +- .../internals/events/TopicMetadataEvent.java | 6 +- .../internals/events/UnsubscribeEvent.java | 6 +- .../internals/events/ValidatePositionsEvent.java | 6 +- .../kafka/clients/producer/KafkaProducer.java | 43 +- .../kafka/clients/producer/ProducerConfig.java | 18 + .../clients/producer/internals/ProducerBatch.java | 14 +- .../producer/internals/RecordAccumulator.java | 12 +- .../apache/kafka/common/TopicPartitionInfo.java | 9 +- .../apache/kafka/common/compress/Compression.java | 112 + .../kafka/common/compress/GzipCompression.java | 138 + .../{SnappyFactory.java => GzipOutputStream.java} | 50 +- ...ckInputStream.java => Lz4BlockInputStream.java} | 12 +- ...OutputStream.java => Lz4BlockOutputStream.java} | 61 +- .../kafka/common/compress/Lz4Compression.java | 108 + .../{SnappyFactory.java => NoCompression.java} | 40 +- .../kafka/common/compress/SnappyCompression.java | 91 + .../kafka/common/compress/ZstdCompression.java | 143 + .../apache/kafka/common/compress/ZstdFactory.java | 73 - .../org/apache/kafka/common/config/ConfigDef.java | 15 +- .../apache/kafka/common/config/TopicConfig.java | 8 + .../errors/RecordDeserializationException.java | 76 +- .../kafka/common/feature/BaseVersionRange.java | 11 +- .../common/feature/SupportedVersionRange.java | 2 +- .../kafka/common/network/SslTransportLayer.java | 4 +- .../common/record/AbstractLegacyRecordBatch.java | 3 +- .../kafka/common/record/CompressionType.java | 173 +- .../kafka/common/record/ControlRecordType.java | 10 +- .../kafka/common/record/ControlRecordUtils.java | 70 +- .../kafka/common/record/DefaultRecordBatch.java | 3 +- .../apache/kafka/common/record/MemoryRecords.java | 227 +- .../kafka/common/record/MemoryRecordsBuilder.java | 64 +- .../apache/kafka/common/record/RecordsUtil.java | 3 +- .../apache/kafka/common/requests/FetchRequest.java | 5 +- .../kafka/common/requests/LeaderAndIsrRequest.java | 3 +- .../kafka/common/requests/MetadataResponse.java | 7 +- .../kafka/common/requests/StopReplicaRequest.java | 3 +- .../common/requests/UpdateMetadataRequest.java | 4 +- .../common/telemetry/ClientTelemetryState.java | 4 +- .../internals/ClientTelemetryReporter.java | 16 +- .../telemetry/internals/ClientTelemetryUtils.java | 7 +- .../java/org/apache/kafka/common/utils/Timer.java | 9 + .../java/org/apache/kafka/common/utils/Utils.java | 44 +- .../common/message/KRaftVersionRecord.json | 17 +- .../resources/common/message/VotersRecord.json | 47 + .../kafka/clients/AddressChangeHostResolver.java | 4 +- .../org/apache/kafka/clients/ClientUtilsTest.java | 2 +- .../kafka/clients/ClusterConnectionStatesTest.java | 7 +- .../kafka/clients/FetchSessionHandlerTest.java | 15 +- .../apache/kafka/clients/InFlightRequestsTest.java | 2 +- .../org/apache/kafka/clients/MetadataTest.java | 22 +- ...tionTest.java => DescribeTopicsResultTest.java} | 31 +- .../kafka/clients/admin/TopicCollectionTest.java | 8 +- .../DeleteConsumerGroupOffsetsHandlerTest.java | 3 +- .../internals/DeleteConsumerGroupsHandlerTest.java | 3 +- .../DescribeConsumerGroupsHandlerTest.java | 3 +- .../admin/internals/FenceProducersHandlerTest.java | 2 - .../RemoveMembersFromConsumerGroupHandlerTest.java | 6 +- .../consumer/ConsumerGroupMetadataTest.java | 2 +- .../consumer/CooperativeStickyAssignorTest.java | 17 + .../kafka/clients/consumer/KafkaConsumerTest.java | 92 +- .../clients/consumer/RoundRobinAssignorTest.java | 11 +- .../internals/AbstractCoordinatorTest.java | 20 +- .../internals/AbstractPartitionAssignorTest.java | 2 +- .../internals/AbstractStickyAssignorTest.java | 27 +- .../consumer/internals/AsyncKafkaConsumerTest.java | 171 +- .../internals/CommitRequestManagerTest.java | 4 +- .../consumer/internals/CompletedFetchTest.java | 40 +- .../internals/ConsumerNetworkClientTest.java | 24 +- .../internals/ConsumerNetworkThreadTest.java | 82 +- .../consumer/internals/ConsumerTestBuilder.java | 30 +- .../consumer/internals/FetchCollectorTest.java | 4 +- .../internals/FetchRequestManagerTest.java | 102 +- .../clients/consumer/internals/FetcherTest.java | 102 +- .../internals/HeartbeatRequestManagerTest.java | 39 +- .../clients/consumer/internals/HeartbeatTest.java | 12 +- .../internals/NetworkClientDelegateTest.java | 3 +- .../consumer/internals/OffsetFetcherTest.java | 2 +- .../internals/OffsetForLeaderEpochClientTest.java | 12 +- .../internals/OffsetsRequestManagerTest.java | 2 +- .../consumer/internals/SubscriptionStateTest.java | 39 +- .../consumer/internals/WakeupTriggerTest.java | 34 + .../events/ApplicationEventProcessorTest.java | 28 +- .../events/CompletableEventReaperTest.java | 200 ++ .../metrics/HeartbeatMetricsManagerTest.java | 4 +- .../metrics/OffsetCommitMetricsManagerTest.java | 4 +- .../RebalanceCallbackMetricsManagerTest.java | 4 +- .../kafka/clients/producer/KafkaProducerTest.java | 18 +- .../producer/internals/ProducerBatchTest.java | 7 +- .../internals/ProducerInterceptorsTest.java | 2 +- .../producer/internals/ProducerMetadataTest.java | 10 +- .../producer/internals/RecordAccumulatorTest.java | 141 +- .../clients/producer/internals/SenderTest.java | 58 +- .../producer/internals/TransactionManagerTest.java | 38 +- .../apache/kafka/common/TopicPartitionTest.java | 7 +- .../kafka/common/compress/GzipCompressionTest.java | 84 + .../{KafkaLZ4Test.java => Lz4CompressionTest.java} | 108 +- .../kafka/common/compress/NoCompressionTest.java | 58 + .../common/compress/SnappyCompressionTest.java | 57 + .../kafka/common/compress/ZstdCompressionTest.java | 72 + .../apache/kafka/common/config/ConfigDefTest.java | 4 +- .../common/feature/SupportedVersionRangeTest.java | 24 +- .../common/header/internals/RecordHeadersTest.java | 8 +- .../apache/kafka/common/message/MessageTest.java | 2 +- .../kafka/common/message/RecordsSerdeTest.java | 4 +- .../apache/kafka/common/metrics/MetricsTest.java | 10 +- .../metrics/internals/IntGaugeSuiteTest.java | 3 +- .../apache/kafka/common/network/CertStores.java | 6 +- .../common/network/SaslChannelBuilderTest.java | 4 +- .../apache/kafka/common/network/SelectorTest.java | 4 +- .../kafka/common/network/SslSelectorTest.java | 7 +- .../common/network/SslTransportLayerTest.java | 104 +- .../network/SslVersionsTransportLayerTest.java | 2 +- .../kafka/common/protocol/MessageUtilTest.java | 2 +- .../kafka/common/protocol/SendBuilderTest.java | 4 +- .../kafka/common/protocol/types/TypeTest.java | 6 +- .../record/AbstractLegacyRecordBatchTest.java | 25 +- .../record/ByteBufferLogInputStreamTest.java | 17 +- .../record/CompressionRatioEstimatorTest.java | 4 +- .../kafka/common/record/CompressionTypeTest.java | 58 - .../kafka/common/record/ControlRecordTypeTest.java | 16 +- .../common/record/ControlRecordUtilsTest.java | 10 + .../common/record/DefaultRecordBatchTest.java | 55 +- .../kafka/common/record/DefaultRecordTest.java | 2 +- .../common/record/FileLogInputStreamTest.java | 31 +- .../kafka/common/record/FileRecordsTest.java | 56 +- .../record/LazyDownConversionRecordsTest.java | 8 +- .../common/record/MemoryRecordsBuilderTest.java | 99 +- .../kafka/common/record/MemoryRecordsTest.java | 53 +- .../common/record/RemoteLogInputStreamTest.java | 27 +- .../common/record/SimpleLegacyRecordTest.java | 4 +- .../common/record/UnalignedFileRecordsTest.java | 5 +- .../requests/AddPartitionsToTxnRequestTest.java | 60 +- .../common/requests/DeleteAclsRequestTest.java | 4 +- .../common/requests/DeleteGroupsResponseTest.java | 41 +- .../common/requests/DescribeAclsRequestTest.java | 4 +- .../common/requests/LeaveGroupRequestTest.java | 9 +- .../common/requests/LeaveGroupResponseTest.java | 13 +- .../common/requests/OffsetCommitRequestTest.java | 3 +- .../common/requests/OffsetFetchRequestTest.java | 2 +- .../kafka/common/requests/ProduceRequestTest.java | 30 +- .../kafka/common/requests/RequestResponseTest.java | 17 +- .../requests/TxnOffsetCommitRequestTest.java | 22 +- .../requests/WriteTxnMarkersRequestTest.java | 26 +- .../requests/WriteTxnMarkersResponseTest.java | 20 +- .../ClientAuthenticationFailureTest.java | 7 +- .../SaslAuthenticatorFailureDelayTest.java | 9 +- .../authenticator/SaslAuthenticatorTest.java | 18 +- .../security/authenticator/TestJaasConfig.java | 2 +- .../common/security/kerberos/KerberosNameTest.java | 10 +- .../common/security/kerberos/KerberosRuleTest.java | 21 +- .../internals/OAuthBearerSaslClientTest.java | 2 +- .../internals/OAuthBearerSaslServerTest.java | 5 - .../ExpiringCredentialRefreshingLoginTest.java | 10 +- .../internals/secured/RefreshingHttpsJwksTest.java | 2 +- .../unsecured/OAuthBearerUnsecuredJwsTest.java | 8 +- ...uthBearerUnsecuredLoginCallbackHandlerTest.java | 6 +- .../unsecured/OAuthBearerValidationUtilsTest.java | 4 +- .../scram/internals/ScramMessagesTest.java | 2 +- .../scram/internals/ScramSaslServerTest.java | 3 +- ...nNameLoggingTrustManagerFactoryWrapperTest.java | 2 +- .../security/ssl/SslPrincipalMapperTest.java | 8 +- .../security/ssl/mock/TestKeyManagerFactory.java | 4 +- .../common/serialization/SerializationTest.java | 5 +- .../internals/ClientTelemetryReporterTest.java | 19 + .../kafka/common/utils/AbstractIteratorTest.java | 2 +- .../apache/kafka/common/utils/ConfigUtilsTest.java | 13 +- .../org/apache/kafka/common/utils/Crc32CTest.java | 4 +- .../ImplicitLinkedHashMultiCollectionTest.java | 8 +- .../apache/kafka/common/utils/MockScheduler.java | 6 +- .../apache/kafka/common/utils/SanitizerTest.java | 2 +- .../kafka/common/utils/SecurityUtilsTest.java | 8 +- .../org/apache/kafka/common/utils/UtilsTest.java | 16 +- .../java/org/apache/kafka/test/MetricsBench.java | 2 +- .../org/apache/kafka/test/Microbenchmarks.java | 106 +- .../java/org/apache/kafka/connect/data/Values.java | 1161 ++++--- .../org/apache/kafka/connect/data/ValuesTest.java | 248 +- .../apache/kafka/connect/mirror/Checkpoint.java | 14 + .../kafka/connect/mirror/CheckpointStore.java | 203 ++ .../connect/mirror/MirrorCheckpointConfig.java | 1 + .../kafka/connect/mirror/MirrorCheckpointTask.java | 51 +- .../kafka/connect/mirror/OffsetSyncStore.java | 22 +- .../kafka/connect/mirror/CheckpointStoreTest.java | 97 + .../connect/mirror/MirrorCheckpointTaskTest.java | 125 +- .../connect/mirror/MirrorSourceConnectorTest.java | 8 +- .../kafka/connect/mirror/OffsetSyncStoreTest.java | 108 +- .../DedicatedMirrorIntegrationTest.java | 12 +- .../kafka/connect/runtime/AbstractHerder.java | 241 +- .../kafka/connect/runtime/ConnectorConfig.java | 6 +- .../org/apache/kafka/connect/runtime/Herder.java | 8 + .../apache/kafka/connect/runtime/WorkerInfo.java | 6 +- .../kafka/connect/runtime/WorkerSinkTask.java | 2 +- .../runtime/distributed/DistributedConfig.java | 5 +- .../runtime/distributed/DistributedHerder.java | 123 +- .../runtime/distributed/WorkerCoordinator.java | 6 + .../distributed/WorkerRebalanceListener.java | 6 + .../kafka/connect/runtime/isolation/Plugins.java | 6 +- .../runtime/rest/resources/ConnectorsResource.java | 13 + .../runtime/standalone/StandaloneHerder.java | 26 + .../storage/ConnectorOffsetBackingStore.java | 144 +- .../connect/storage/KafkaConfigBackingStore.java | 21 +- .../connect/storage/KafkaOffsetBackingStore.java | 20 +- .../connect/storage/KafkaStatusBackingStore.java | 28 +- .../apache/kafka/connect/util/ConnectUtils.java | 25 + .../apache/kafka/connect/util/KafkaBasedLog.java | 12 +- .../org/apache/kafka/connect/util/TopicAdmin.java | 3 +- .../connect/integration/BlockingConnectorTest.java | 2 +- .../integration/ConnectWorkerIntegrationTest.java | 82 + .../ConnectorValidationIntegrationTest.java | 90 +- .../SessionedProtocolIntegrationTest.java | 17 +- .../runtime/distributed/DistributedHerderTest.java | 127 + .../WorkerCoordinatorIncrementalTest.java | 3 + .../runtime/distributed/WorkerCoordinatorTest.java | 3 + .../rest/resources/ConnectorsResourceTest.java | 42 + .../runtime/standalone/StandaloneHerderTest.java | 73 + .../storage/ConnectorOffsetBackingStoreTest.java | 456 +++ .../KafkaConfigBackingStoreMockitoTest.java | 891 +++++- .../storage/KafkaConfigBackingStoreTest.java | 909 +----- .../storage/KafkaStatusBackingStoreFormatTest.java | 7 +- .../storage/KafkaStatusBackingStoreTest.java | 2 +- .../kafka/connect/util/ConnectUtilsTest.java | 20 + .../connect/util/clusters/ConnectAssertions.java | 2 +- .../connect/util/clusters/EmbeddedConnect.java | 37 + .../util/clusters/EmbeddedKafkaCluster.java | 5 +- .../org.apache.kafka.connect.storage.Converter | 4 +- ...rg.apache.kafka.connect.storage.HeaderConverter | 4 +- .../kafka/connect/transforms/ExtractField.java | 27 +- .../transforms/field/FieldSyntaxVersion.java | 90 + .../connect/transforms/field/SingleFieldPath.java | 242 ++ .../kafka/connect/transforms/ExtractFieldTest.java | 71 +- .../transforms/field/FieldPathNotationTest.java | 160 + .../transforms/field/FieldSyntaxVersionTest.java | 71 + .../transforms/field/SingleFieldPathTest.java | 117 + .../java/kafka/log/remote/RemoteLogManager.java | 96 +- core/src/main/java/kafka/server/NetworkUtils.java | 2 +- .../ReplicaAlterLogDirsTierStateMachine.java | 41 - .../server/ReplicaFetcherTierStateMachine.java | 280 -- .../main/java/kafka/server/TierStateMachine.java | 237 +- .../src/main/scala/kafka/admin/ConfigCommand.scala | 14 +- .../scala/kafka/admin/ZkSecurityMigrator.scala | 2 +- core/src/main/scala/kafka/cluster/Partition.scala | 2 +- .../coordinator/group/CoordinatorLoaderImpl.scala | 4 +- .../group/CoordinatorPartitionWriter.scala | 114 +- .../kafka/coordinator/group/GroupCoordinator.scala | 6 +- .../coordinator/group/GroupMetadataManager.scala | 97 +- .../coordinator/transaction/TransactionLog.scala | 5 +- .../transaction/TransactionStateManager.scala | 8 +- .../scala/kafka/docker/KafkaDockerWrapper.scala | 14 +- core/src/main/scala/kafka/log/LocalLog.scala | 19 +- core/src/main/scala/kafka/log/LogCleaner.scala | 13 +- core/src/main/scala/kafka/log/LogManager.scala | 9 +- core/src/main/scala/kafka/log/UnifiedLog.scala | 24 +- .../scala/kafka/metrics/KafkaMetricsConfig.scala | 8 +- .../main/scala/kafka/network/RequestChannel.scala | 5 +- .../main/scala/kafka/network/SocketServer.scala | 49 +- .../main/scala/kafka/raft/KafkaMetadataLog.scala | 21 +- core/src/main/scala/kafka/raft/RaftManager.scala | 54 +- .../kafka/security/authorizer/AclAuthorizer.scala | 2 +- core/src/main/scala/kafka/serializer/Decoder.scala | 7 +- .../kafka/server/AddPartitionsToTxnManager.scala | 20 +- .../kafka/server/BrokerLifecycleManager.scala | 42 +- .../src/main/scala/kafka/server/BrokerServer.scala | 50 +- .../main/scala/kafka/server/ControllerServer.scala | 2 +- .../src/main/scala/kafka/server/DelayedFetch.scala | 14 +- .../scala/kafka/server/DynamicBrokerConfig.scala | 26 +- .../src/main/scala/kafka/server/FetchSession.scala | 91 +- core/src/main/scala/kafka/server/KafkaApis.scala | 24 +- core/src/main/scala/kafka/server/KafkaBroker.scala | 3 + core/src/main/scala/kafka/server/KafkaConfig.scala | 281 +- .../main/scala/kafka/server/KafkaRaftServer.scala | 5 +- core/src/main/scala/kafka/server/KafkaServer.scala | 49 +- .../main/scala/kafka/server/MetadataCache.scala | 3 +- .../server/NodeToControllerChannelManager.scala | 29 +- .../kafka/server/ReplicaAlterLogDirsThread.scala | 2 +- .../scala/kafka/server/ReplicaFetcherThread.scala | 2 +- .../main/scala/kafka/server/ReplicaManager.scala | 75 +- .../src/main/scala/kafka/server/SharedServer.scala | 12 +- .../kafka/server/metadata/ZkMetadataCache.scala | 9 +- .../main/scala/kafka/tools/DumpLogSegments.scala | 215 +- .../main/scala/kafka/tools/TestRaftServer.scala | 14 +- core/src/main/scala/kafka/utils/CoreUtils.scala | 16 +- core/src/main/scala/kafka/utils/NotNothing.scala | 41 - core/src/main/scala/kafka/utils/Throttler.scala | 111 - core/src/main/scala/kafka/utils/ToolsUtils.scala | 32 - core/src/main/scala/kafka/zk/KafkaZkClient.scala | 64 +- .../kafka/admin/ConfigCommandIntegrationTest.java | 15 +- .../test/java/kafka/admin/ConfigCommandTest.java | 1999 ++++++++++++ .../admin/UserScramCredentialsCommandTest.java | 201 ++ .../kafka/log/remote/RemoteLogManagerTest.java | 201 +- core/src/test/java/kafka/test/ClusterConfig.java | 125 +- .../test/java/kafka/test/ClusterConfigTest.java | 38 +- core/src/test/java/kafka/test/ClusterInstance.java | 92 +- .../java/kafka/test/ClusterTestExtensionsTest.java | 155 +- .../test/annotation/ClusterConfigProperty.java | 21 + .../java/kafka/test/annotation/ClusterTest.java | 7 +- .../kafka/test/annotation/ClusterTestDefaults.java | 3 +- core/src/test/java/kafka/test/annotation/Type.java | 29 +- .../kafka/test/junit/ClusterTestExtensions.java | 150 +- .../test/junit/ClusterTestExtensionsUnitTest.java | 45 + core/src/test/java/kafka/test/junit/README.md | 41 +- .../test/junit/RaftClusterInvocationContext.java | 96 +- .../test/junit/ZkClusterInvocationContext.java | 82 +- core/src/test/java/kafka/testkit/BrokerNode.java | 19 +- .../test/java/kafka/testkit/BrokerNodeTest.java | 42 + .../test/java/kafka/testkit/ControllerNode.java | 44 +- .../java/kafka/testkit/KafkaClusterTestKit.java | 33 +- .../kafka/testkit/KafkaClusterTestKitTest.java | 25 +- core/src/test/java/kafka/testkit/TestKitNodes.java | 51 +- .../kafka/admin/BrokerApiVersionsCommandTest.scala | 5 +- .../api/AbstractAuthorizerIntegrationTest.scala | 11 +- .../kafka/api/AbstractConsumerTest.scala | 5 +- .../AdminClientWithPoliciesIntegrationTest.scala | 6 +- .../kafka/api/AuthorizerIntegrationTest.scala | 27 +- .../kafka/api/BaseAdminIntegrationTest.scala | 23 +- .../integration/kafka/api/BaseQuotaTest.scala | 6 +- .../kafka/api/ConsumerTopicCreationTest.scala | 7 +- .../kafka/api/CustomQuotaCallbackTest.scala | 6 +- .../DelegationTokenEndToEndAuthorizationTest.scala | 7 +- ...onTokenEndToEndAuthorizationWithOwnerTest.scala | 4 +- .../api/DescribeAuthorizedOperationsTest.scala | 6 +- .../kafka/api/EndToEndAuthorizationTest.scala | 70 +- .../kafka/api/GroupAuthorizerIntegrationTest.scala | 9 +- .../kafka/api/IntegrationTestHarness.scala | 22 +- .../kafka/api/PlaintextAdminIntegrationTest.scala | 118 +- .../kafka/api/PlaintextConsumerCallbackTest.scala | 171 ++ .../kafka/api/PlaintextConsumerTest.scala | 50 +- .../kafka/api/ProducerFailureHandlingTest.scala | 6 +- .../kafka/api/ProducerIdExpirationTest.scala | 4 +- .../kafka/api/SaslEndToEndAuthorizationTest.scala | 41 +- .../scala/integration/kafka/api/SaslSetup.scala | 6 +- .../kafka/api/SaslSslAdminIntegrationTest.scala | 172 +- .../kafka/api/SslAdminIntegrationTest.scala | 58 +- .../kafka/api/TransactionsBounceTest.scala | 6 +- .../kafka/api/TransactionsExpirationTest.scala | 4 +- .../integration/kafka/api/TransactionsTest.scala | 5 +- .../api/TransactionsWithMaxInFlightOneTest.scala | 4 +- .../transaction/ProducerIdsIntegrationTest.scala | 22 +- .../kafka/network/DynamicConnectionQuotaTest.scala | 5 +- .../network/DynamicNumNetworkThreadsTest.scala | 20 +- .../kafka/server/DelayedFetchTest.scala | 66 +- .../server/DynamicBrokerReconfigurationTest.scala | 18 +- .../kafka/server/IntegrationTestUtils.scala | 9 +- .../kafka/server/KRaftClusterTest.scala | 50 +- .../server/KafkaServerKRaftRegistrationTest.scala | 4 +- .../server/MetadataVersionIntegrationTest.scala | 16 +- .../kafka/server/QuorumTestHarness.scala | 28 +- .../kafka/server/RaftClusterSnapshotTest.scala | 15 +- .../kafka/zk/ZkMigrationIntegrationTest.scala | 153 +- .../scala/kafka/raft/KafkaMetadataLogTest.scala | 101 +- .../kafka/server/LocalLeaderEndPointTest.scala | 5 +- .../server/metadata/BrokerServerMetricsTest.scala | 12 +- .../test/scala/kafka/utils/ToolsUtilsTest.scala | 45 - .../scala/other/kafka/TestLinearWriteSpeed.scala | 37 +- .../scala/unit/kafka/admin/AclCommandTest.scala | 3 +- .../scala/unit/kafka/admin/ConfigCommandTest.scala | 1789 ----------- .../scala/unit/kafka/admin/DeleteTopicTest.scala | 8 +- .../admin/UserScramCredentialsCommandTest.scala | 142 - .../scala/unit/kafka/cluster/PartitionTest.scala | 23 +- .../controller/ControllerChannelManagerTest.scala | 4 +- .../AbstractCoordinatorConcurrencyTest.scala | 2 +- .../group/CoordinatorLoaderImplTest.scala | 41 +- .../group/CoordinatorPartitionWriterTest.scala | 350 +-- .../group/GroupMetadataManagerTest.scala | 221 +- .../TransactionCoordinatorConcurrencyTest.scala | 5 +- .../transaction/TransactionLogTest.scala | 5 +- .../transaction/TransactionStateManagerTest.scala | 15 +- .../kafka/integration/KafkaServerTestHarness.scala | 11 +- .../MetricsDuringTopicCreationDeletionTest.scala | 4 +- .../log/AbstractLogCleanerIntegrationTest.scala | 7 +- .../unit/kafka/log/BrokerCompressionTest.scala | 17 +- .../test/scala/unit/kafka/log/LocalLogTest.scala | 42 +- .../unit/kafka/log/LogCleanerIntegrationTest.scala | 11 +- .../kafka/log/LogCleanerLagIntegrationTest.scala | 6 +- .../unit/kafka/log/LogCleanerManagerTest.scala | 9 +- .../LogCleanerParameterizedIntegrationTest.scala | 32 +- .../test/scala/unit/kafka/log/LogCleanerTest.scala | 19 +- .../test/scala/unit/kafka/log/LogConfigTest.scala | 5 +- .../test/scala/unit/kafka/log/LogLoaderTest.scala | 31 +- .../test/scala/unit/kafka/log/LogManagerTest.scala | 48 +- .../test/scala/unit/kafka/log/LogSegmentTest.scala | 53 +- .../test/scala/unit/kafka/log/LogTestUtils.scala | 15 +- .../scala/unit/kafka/log/LogValidatorTest.scala | 368 ++- .../test/scala/unit/kafka/log/UnifiedLogTest.scala | 175 +- .../unit/kafka/network/SocketServerTest.scala | 24 +- .../scala/unit/kafka/raft/RaftManagerTest.scala | 9 +- .../delegation/DelegationTokenManagerTest.scala | 8 +- .../kafka/server/AbstractFetcherManagerTest.scala | 5 +- .../kafka/server/AbstractMetadataRequestTest.scala | 4 +- .../server/AddPartitionsToTxnManagerTest.scala | 34 +- .../server/AllocateProducerIdsRequestTest.scala | 10 +- ...rScramCredentialsRequestNotAuthorizedTest.scala | 6 +- .../AlterUserScramCredentialsRequestTest.scala | 10 +- .../unit/kafka/server/ApiVersionsRequestTest.scala | 167 +- .../server/AutoTopicCreationManagerTest.scala | 3 +- .../unit/kafka/server/BaseFetchRequestTest.scala | 3 +- .../scala/unit/kafka/server/BaseRequestTest.scala | 11 +- .../kafka/server/BrokerLifecycleManagerTest.scala | 6 +- .../unit/kafka/server/BrokerMetricNamesTest.scala | 3 +- .../server/BrokerRegistrationRequestTest.scala | 12 +- .../kafka/server/ClientQuotasRequestTest.scala | 5 +- .../server/ConsumerGroupDescribeRequestsTest.scala | 192 ++ .../server/ConsumerGroupHeartbeatRequestTest.scala | 25 +- .../server/ConsumerProtocolMigrationTest.scala | 16 +- .../unit/kafka/server/ControllerApisTest.scala | 9 +- .../kafka/server/ControllerMutationQuotaTest.scala | 4 +- .../kafka/server/DelegationTokenRequestsTest.scala | 15 +- .../kafka/server/DeleteGroupsRequestTest.scala | 4 +- ...leteTopicsRequestWithDeletionDisabledTest.scala | 4 +- .../kafka/server/DescribeClusterRequestTest.scala | 4 +- .../kafka/server/DescribeGroupsRequestTest.scala | 4 +- .../kafka/server/DescribeQuorumRequestTest.scala | 7 +- ...rScramCredentialsRequestNotAuthorizedTest.scala | 8 +- .../DescribeUserScramCredentialsRequestTest.scala | 10 +- .../kafka/server/DynamicBrokerConfigTest.scala | 34 +- .../kafka/server/DynamicConfigChangeTest.scala | 7 +- .../unit/kafka/server/EdgeCaseRequestTest.scala | 6 +- .../kafka/server/FetchRequestMaxBytesTest.scala | 3 +- .../scala/unit/kafka/server/FetchSessionTest.scala | 280 +- .../unit/kafka/server/FetcherThreadTestUtils.scala | 5 +- .../server/GroupCoordinatorBaseRequestTest.scala | 38 +- .../unit/kafka/server/HeartbeatRequestTest.scala | 4 +- .../unit/kafka/server/JoinGroupRequestTest.scala | 7 +- .../scala/unit/kafka/server/KafkaApisTest.scala | 97 +- .../scala/unit/kafka/server/KafkaConfigTest.scala | 218 +- .../server/KafkaMetricReporterClusterIdTest.scala | 7 +- .../kafka/server/KafkaMetricsReporterTest.scala | 5 +- .../unit/kafka/server/LeaveGroupRequestTest.scala | 4 +- .../unit/kafka/server/ListGroupsRequestTest.scala | 4 +- .../unit/kafka/server/ListOffsetsRequestTest.scala | 61 +- .../ListOffsetsRequestWithRemoteStoreTest.scala | 10 +- .../unit/kafka/server/LogDirFailureTest.scala | 30 +- .../unit/kafka/server/MockTierStateMachine.scala | 9 +- .../kafka/server/OffsetCommitRequestTest.scala | 4 +- .../kafka/server/OffsetDeleteRequestTest.scala | 4 +- .../unit/kafka/server/OffsetFetchRequestTest.scala | 8 +- .../unit/kafka/server/ProduceRequestTest.scala | 15 +- .../kafka/server/ReplicaFetcherThreadTest.scala | 5 +- .../kafka/server/ReplicaManagerQuotasTest.scala | 5 +- .../unit/kafka/server/ReplicaManagerTest.scala | 149 +- .../unit/kafka/server/ReplicationQuotasTest.scala | 6 +- .../scala/unit/kafka/server/RequestQuotaTest.scala | 13 +- .../kafka/server/SaslApiVersionsRequestTest.scala | 10 +- .../kafka/server/ServerGenerateBrokerIdTest.scala | 9 +- .../test/scala/unit/kafka/server/ServerTest.scala | 5 +- .../unit/kafka/server/SyncGroupRequestTest.scala | 4 +- ...achineTest.scala => TierStateMachineTest.scala} | 26 +- .../TopicIdWithOldInterBrokerProtocolTest.scala | 4 +- .../server/epoch/LeaderEpochIntegrationTest.scala | 3 +- .../unit/kafka/tools/DumpLogSegmentsTest.scala | 302 +- .../scala/unit/kafka/tools/StorageToolTest.scala | 4 +- .../scala/unit/kafka/utils/CoreUtilsTest.scala | 13 - .../test/scala/unit/kafka/utils/TestUtils.scala | 66 +- .../scala/unit/kafka/utils/ThrottlerTest.scala | 108 - .../scala/unit/kafka/zk/KafkaZkClientTest.scala | 26 +- .../zk/migration/ZkConfigMigrationClientTest.scala | 1 + docker/common.py | 10 +- docker/docker_build_test.py | 24 +- docker/docker_official_image_build_test.py | 87 + docker/docker_official_images/.gitkeep | 0 docker/docker_release.py | 13 +- docker/extract_docker_official_image_artifact.py | 77 + docker/generate_kafka_pr_template.py | 92 + docker/native/Dockerfile | 90 + docker/native/launch | 50 + docker/native/native-image-configs/jni-config.json | 35 + .../predefined-classes-config.json | 7 + .../native/native-image-configs/proxy-config.json | 5 + .../native-image-configs/reflect-config.json | 1275 ++++++++ .../native-image-configs/resource-config.json | 69 + .../native-image-configs/serialization-config.json | 128 + docker/prepare_docker_official_image_source.py | 68 + docker/test/docker_sanity_test.py | 10 +- docs/connect.html | 1 + docs/security.html | 48 + docs/streams/developer-guide/config-streams.html | 81 +- docs/streams/quickstart.html | 4 - docs/upgrade.html | 26 +- gradle.properties | 2 +- gradle/dependencies.gradle | 12 +- gradle/spotbugs-exclude.xml | 17 +- .../group/{Record.java => CoordinatorRecord.java} | 10 +- ...dHelpers.java => CoordinatorRecordHelpers.java} | 94 +- ...ecordSerde.java => CoordinatorRecordSerde.java} | 17 +- .../org/apache/kafka/coordinator/group/Group.java | 2 +- .../coordinator/group/GroupCoordinatorConfig.java | 17 +- .../coordinator/group/GroupCoordinatorService.java | 36 +- .../coordinator/group/GroupCoordinatorShard.java | 57 +- .../coordinator/group/GroupMetadataManager.java | 1377 +++++++-- .../coordinator/group/OffsetMetadataManager.java | 34 +- .../assignor/AbstractUniformAssignmentBuilder.java | 199 -- .../group/assignor/AssignmentMemberSpec.java | 9 +- .../assignor/GeneralUniformAssignmentBuilder.java | 142 +- ...titionAssignorException.java => GroupSpec.java} | 28 +- .../coordinator/group/assignor/GroupSpecImpl.java | 110 + .../OptimizedUniformAssignmentBuilder.java | 144 +- .../group/assignor/PartitionAssignor.java | 4 +- .../group/assignor/PartitionAssignorException.java | 2 +- .../coordinator/group/assignor/RangeAssignor.java | 61 +- ...ssignorException.java => SubscriptionType.java} | 28 +- .../group/assignor/UniformAssignor.java | 36 +- .../coordinator/group/classic/ClassicGroup.java | 16 +- .../coordinator/group/consumer/ConsumerGroup.java | 288 +- .../group/consumer/ConsumerGroupMember.java | 66 +- .../group/consumer/CurrentAssignmentBuilder.java | 45 +- .../group/consumer/TargetAssignmentBuilder.java | 113 +- .../kafka/coordinator/group/consumer/TopicIds.java | 176 ++ .../group/metrics/CoordinatorRuntimeMetrics.java | 6 +- .../metrics/GroupCoordinatorRuntimeMetrics.java | 33 +- .../group/runtime/CoordinatorLoader.java | 17 - .../group/runtime/CoordinatorRuntime.java | 234 +- .../group/runtime/CoordinatorTimer.java | 12 + .../coordinator/group/runtime/Deserializer.java | 23 +- .../group/runtime/MultiThreadedEventProcessor.java | 26 +- .../coordinator/group/runtime/PartitionWriter.java | 66 +- .../coordinator/group/runtime/Serializer.java | 22 +- .../message/ConsumerGroupMemberMetadataValue.json | 2 + .../ConsumerGroupPartitionMetadataValue.json | 2 - .../apache/kafka/coordinator/group/Assertions.java | 45 +- .../coordinator/group/AssignmentTestUtil.java | 30 + ...Test.java => CoordinatorRecordHelpersTest.java} | 82 +- ...deTest.java => CoordinatorRecordSerdeTest.java} | 36 +- ...{RecordTest.java => CoordinatorRecordTest.java} | 12 +- .../group/GroupCoordinatorConfigTest.java | 8 +- .../group/GroupCoordinatorServiceTest.java | 169 +- .../group/GroupCoordinatorShardTest.java | 116 +- .../group/GroupMetadataManagerTest.java | 3235 +++++++++++++++++--- .../group/GroupMetadataManagerTestContext.java | 204 +- .../coordinator/group/MockCoordinatorTimer.java | 13 + .../coordinator/group/MockPartitionAssignor.java | 14 +- ...ionAssignor.java => NoOpPartitionAssignor.java} | 29 +- .../group/OffsetMetadataManagerTest.java | 168 +- .../GeneralUniformAssignmentBuilderTest.java | 611 +--- .../group/assignor/GroupSpecImplTest.java | 84 + .../OptimizedUniformAssignmentBuilderTest.java | 675 +--- .../group/assignor/RangeAssignorTest.java | 251 +- .../group/consumer/ConsumerGroupBuilder.java | 20 +- .../group/consumer/ConsumerGroupMemberTest.java | 44 +- .../group/consumer/ConsumerGroupTest.java | 294 +- .../consumer/CurrentAssignmentBuilderTest.java | 37 + .../consumer/SubscribedTopicMetadataTest.java | 2 +- .../consumer/TargetAssignmentBuilderTest.java | 89 +- .../coordinator/group/consumer/TopicIdsTest.java | 190 ++ .../group/consumer/TopicMetadataTest.java | 4 +- .../GroupCoordinatorRuntimeMetricsTest.java | 22 +- .../group/runtime/CoordinatorResultTest.java | 8 +- .../group/runtime/CoordinatorRuntimeTest.java | 525 +++- .../group/runtime/InMemoryPartitionWriter.java | 194 +- .../runtime/MultiThreadedEventProcessorTest.java | 52 +- .../kafka/jmh/assignor/AssignorBenchmarkUtils.java | 77 + .../jmh/assignor/ClientSideAssignorBenchmark.java | 9 +- .../jmh/assignor/ServerSideAssignorBenchmark.java | 80 +- .../assignor/TargetAssignmentBuilderBenchmark.java | 41 +- .../kafka/jmh/common/FetchResponseBenchmark.java | 4 +- .../apache/kafka/jmh/connect/ValuesBenchmark.java | 297 ++ .../jmh/fetcher/ReplicaFetcherThreadBenchmark.java | 2 +- .../jmh/metadata/MetadataRequestBenchmark.java | 6 +- .../partition/PartitionMakeFollowerBenchmark.java | 4 +- .../jmh/producer/ProducerRequestBenchmark.java | 4 +- .../kafka/jmh/record/BaseRecordBatchBenchmark.java | 8 +- .../CompressedRecordBatchValidationBenchmark.java | 7 +- .../jmh/record/RecordBatchIterationBenchmark.java | 5 +- ...UncompressedRecordBatchValidationBenchmark.java | 7 +- .../apache/kafka/jmh/server/CheckpointBench.java | 2 +- .../kafka/jmh/server/PartitionCreationBench.java | 4 +- .../kafka/log4jappender/KafkaLog4jAppender.java | 20 +- .../log4jappender/KafkaLog4jAppenderTest.java | 13 + .../log4jappender/MockKafkaLog4jAppender.java | 1 + .../kafka/controller/FeatureControlManager.java | 4 +- .../kafka/controller/PartitionChangeBuilder.java | 89 +- .../apache/kafka/controller/QuorumController.java | 2 - .../kafka/metadata/util/BatchFileWriter.java | 4 +- .../controller/PartitionChangeBuilderTest.java | 213 +- .../kafka/image/node/TopicImageNodeTest.java | 43 +- .../kafka/image/publisher/SnapshotEmitterTest.java | 5 - .../org/apache/kafka/metalog/LocalLogManager.java | 47 +- .../kafka/metalog/LocalLogManagerTestEnv.java | 3 - .../java/org/apache/kafka/raft/CandidateState.java | 69 +- .../java/org/apache/kafka/raft/ControlRecord.java | 16 + .../java/org/apache/kafka/raft/ElectionState.java | 195 +- .../java/org/apache/kafka/raft/EpochState.java | 17 +- ...edStateStore.java => FileQuorumStateStore.java} | 129 +- .../java/org/apache/kafka/raft/FollowerState.java | 26 +- .../org/apache/kafka/raft/KafkaNetworkChannel.java | 5 +- .../org/apache/kafka/raft/KafkaRaftClient.java | 271 +- .../java/org/apache/kafka/raft/LeaderState.java | 16 +- .../java/org/apache/kafka/raft/NetworkChannel.java | 4 +- .../java/org/apache/kafka/raft/QuorumConfig.java | 102 +- .../java/org/apache/kafka/raft/QuorumState.java | 325 +- .../org/apache/kafka/raft/QuorumStateStore.java | 27 +- .../java/org/apache/kafka/raft/RaftClient.java | 5 - .../org/apache/kafka/raft/ReplicatedCounter.java | 36 +- .../java/org/apache/kafka/raft/ReplicatedLog.java | 24 +- .../java/org/apache/kafka/raft/ResignedState.java | 11 +- .../org/apache/kafka/raft/UnattachedState.java | 18 +- .../java/org/apache/kafka/raft/VotedState.java | 55 +- .../kafka/raft/internals/BatchAccumulator.java | 121 +- .../apache/kafka/raft/internals/BatchBuilder.java | 14 +- .../apache/kafka/raft/internals/CloseListener.java | 3 +- .../internals/KRaftControlRecordStateMachine.java | 270 ++ .../kafka/raft/internals/KafkaRaftMetrics.java | 28 +- .../apache/kafka/raft/internals/LogHistory.java | 119 + .../kafka/raft/internals/RecordsBatchReader.java | 1 + .../kafka/raft/internals/RecordsIterator.java | 12 +- .../apache/kafka/raft/internals/ReplicaKey.java | 54 +- .../kafka/raft/internals/TreeMapLogHistory.java | 76 + .../org/apache/kafka/raft/internals/VoterSet.java | 383 +++ .../kafka/raft/internals/VoterSetHistory.java | 124 + .../kafka/snapshot/FileRawSnapshotWriter.java | 18 +- .../kafka/snapshot/NotifyingRawSnapshotWriter.java | 86 + .../kafka/snapshot/RecordsSnapshotReader.java | 16 +- .../kafka/snapshot/RecordsSnapshotWriter.java | 226 +- .../resources/common/message/QuorumStateData.json | 20 +- .../org/apache/kafka/raft/CandidateStateTest.java | 95 +- .../org/apache/kafka/raft/ControlRecordTest.java | 6 +- .../org/apache/kafka/raft/ElectionStateTest.java | 102 + .../apache/kafka/raft/FileBasedStateStoreTest.java | 166 - .../kafka/raft/FileQuorumStateStoreTest.java | 223 ++ .../org/apache/kafka/raft/FollowerStateTest.java | 7 +- .../apache/kafka/raft/KafkaNetworkChannelTest.java | 21 +- .../kafka/raft/KafkaRaftClientSnapshotTest.java | 15 +- .../org/apache/kafka/raft/KafkaRaftClientTest.java | 58 +- .../org/apache/kafka/raft/LeaderStateTest.java | 7 +- .../test/java/org/apache/kafka/raft/MockLog.java | 8 +- .../java/org/apache/kafka/raft/MockLogTest.java | 40 +- .../org/apache/kafka/raft/MockNetworkChannel.java | 3 +- .../apache/kafka/raft/MockQuorumStateStore.java | 36 +- .../org/apache/kafka/raft/QuorumStateTest.java | 828 +++-- .../apache/kafka/raft/RaftClientTestContext.java | 95 +- .../apache/kafka/raft/RaftEventSimulationTest.java | 124 +- .../org/apache/kafka/raft/ResignedStateTest.java | 8 +- .../org/apache/kafka/raft/UnattachedStateTest.java | 16 +- .../java/org/apache/kafka/raft/VotedStateTest.java | 60 +- .../kafka/raft/internals/BatchAccumulatorTest.java | 200 +- .../kafka/raft/internals/BatchBuilderTest.java | 7 +- .../KRaftControlRecordStateMachineTest.java | 375 +++ .../kafka/raft/internals/KafkaRaftMetricsTest.java | 121 +- .../kafka/raft/internals/RecordsIteratorTest.java | 206 +- .../raft/internals/TreeMapLogHistoryTest.java | 104 + .../kafka/raft/internals/VoterSetHistoryTest.java | 216 ++ .../apache/kafka/raft/internals/VoterSetTest.java | 260 ++ .../apache/kafka/snapshot/FileRawSnapshotTest.java | 7 +- .../snapshot/NotifyingRawSnapshotWriterTest.java | 123 + .../kafka/snapshot/RecordsSnapshotWriterTest.java | 227 ++ .../org/apache/kafka/snapshot/SnapshotsTest.java | 2 +- .../org/apache/kafka/queue/KafkaEventQueue.java | 5 +- .../kafka/server/config/ServerLogConfigs.java | 7 +- .../server/config/ServerTopicConfigSynonyms.java | 3 + .../kafka/server/record/BrokerCompressionType.java | 48 +- .../java/org/apache/kafka/server/util/Csv.java | 16 + .../kafka/server/util/timer/SystemTimer.java | 3 +- .../apache/kafka/queue/KafkaEventQueueTest.java | 445 +-- .../server/record/BrokerCompressionTypeTest.java | 29 +- .../java/org/apache/kafka/server/util/CsvTest.java | 22 +- .../apache/kafka/server/util/timer/MockTimer.java | 4 + .../org/apache/kafka/server/config/Defaults.java | 60 - .../config/DelegationTokenManagerConfigs.java | 41 + .../kafka/server/config/ReplicationConfigs.java | 3 +- .../apache/kafka/server/config/ServerConfigs.java | 139 + .../kafka/server/ClientMetricsManagerTest.java | 327 +- .../TopicBasedRemoteLogMetadataManager.java | 27 +- .../serialization/RemoteLogMetadataSerde.java | 70 +- .../kafka/storage/internals/log/FetchParams.java | 6 + .../kafka/storage/internals/log/LogConfig.java | 44 +- .../storage/internals/log/LogOffsetMetadata.java | 16 +- .../kafka/storage/internals/log/LogSegment.java | 18 +- .../kafka/storage/internals/log/LogValidator.java | 27 +- .../kafka/storage/internals/utils/Throttler.java | 102 + .../storage/RemoteLogMetadataManagerTestUtils.java | 124 + .../TopicBasedRemoteLogMetadataManagerHarness.java | 103 +- ...ogMetadataManagerMultipleSubscriptionsTest.java | 243 +- ...icBasedRemoteLogMetadataManagerRestartTest.java | 2 +- .../TopicBasedRemoteLogMetadataManagerTest.java | 246 +- .../storage/InmemoryRemoteLogMetadataManager.java | 183 -- .../storage/InmemoryRemoteStorageManager.java | 183 -- .../storage/InmemoryRemoteStorageManagerTest.java | 251 -- .../log/remote/storage/LocalTieredStorageTest.java | 4 +- .../storage/RemoteLogMetadataManagerTest.java | 85 +- .../internals/log/LogOffsetMetadataTest.java | 82 + .../storage/internals/utils/ThrottlerTest.java | 115 + .../tiered/storage/TieredStorageTestBuilder.java | 17 + .../tiered/storage/TieredStorageTestContext.java | 19 +- .../tiered/storage/TieredStorageTestHarness.java | 2 +- .../tiered/storage/actions/AlterLogDirAction.java | 80 + .../storage/actions/EraseBrokerStorageAction.java | 16 +- .../tiered/storage/actions/ExpectLeaderAction.java | 10 +- .../storage/integration/AlterLogDirTest.java | 68 + ...FetchFromLeaderWithCorruptedCheckpointTest.java | 84 + .../TransactionsWithTieredStoreTest.java | 2 +- .../tiered/storage/utils/BrokerLocalStorage.java | 56 +- .../storage/utils/TieredStorageTestUtils.java | 8 + .../org/apache/kafka/streams/StreamsConfig.java | 36 +- .../kafka/streams/errors/ErrorHandlerContext.java | 133 + .../LogAndContinueProcessingExceptionHandler.java | 46 + .../LogAndFailProcessingExceptionHandler.java | 46 + .../streams/errors/ProcessingExceptionHandler.java | 56 + .../internals/DefaultErrorHandlerContext.java | 90 + .../streams/kstream/internals/KStreamImplJoin.java | 6 +- .../kstream/internals/KStreamKStreamJoin.java | 160 +- .../internals/KStreamKStreamJoinLeftSide.java | 71 + .../internals/KStreamKStreamJoinRightSide.java | 70 + .../streams/processor/api/RecordMetadata.java | 7 +- .../processor/assignment/ApplicationState.java | 52 + .../processor/assignment/AssignmentConfigs.java | 159 + .../assignment/KafkaStreamsAssignment.java | 129 + .../processor/assignment/KafkaStreamsState.java | 108 + .../streams/processor/assignment/ProcessId.java | 30 +- .../processor/assignment/TaskAssignmentUtils.java | 93 + .../streams/processor/assignment/TaskAssignor.java | 100 + .../streams/processor/assignment/TaskInfo.java | 56 + .../processor/assignment/TaskTopicPartition.java | 56 + .../assignment/assignors/StickyTaskAssignor.java | 466 +++ .../processor/internals/DefaultStateUpdater.java | 290 +- .../internals/InternalTopologyBuilder.java | 9 + .../processor/internals/PendingUpdateAction.java | 92 - .../processor/internals/RecordCollectorImpl.java | 2 + .../processor/internals/RepartitionTopics.java | 7 + .../processor/internals/StateDirectory.java | 4 +- .../streams/processor/internals/StateUpdater.java | 119 +- .../internals/StreamsPartitionAssignor.java | 309 +- .../processor/internals/StreamsProducer.java | 8 +- .../internals/StreamsRebalanceListener.java | 4 +- .../streams/processor/internals/TaskAndAction.java | 34 +- .../streams/processor/internals/TaskManager.java | 415 +-- .../kafka/streams/processor/internals/Tasks.java | 104 +- .../streams/processor/internals/TasksRegistry.java | 28 +- .../internals/assignment/ApplicationStateImpl.java | 83 + .../assignment/AssignorConfiguration.java | 32 + .../internals/assignment/AssignorError.java | 23 +- .../BalanceSubtopologyGraphConstructor.java | 58 +- .../internals/assignment/ClientState.java | 43 + .../internals/assignment/DefaultTaskInfo.java | 62 + .../assignment/DefaultTaskTopicPartition.java | 99 + .../assignment/KafkaStreamsStateImpl.java | 164 + .../assignment/MinTrafficGraphConstructor.java | 20 +- .../assignment/RackAwareGraphConstructor.java | 30 +- .../RackAwareGraphConstructorFactory.java | 15 +- .../assignment/RackAwareTaskAssignor.java | 44 +- .../processor/internals/assignment/RackUtils.java | 157 + .../streams/state/internals/LeftOrRightValue.java | 15 - .../streams/state/internals/MeteredIterator.java | 18 +- .../state/internals/MeteredKeyValueStore.java | 48 +- .../MeteredMultiVersionedKeyQueryIterator.java | 40 +- .../state/internals/MeteredSessionStore.java | 64 +- .../internals/MeteredTimestampedKeyValueStore.java | 17 +- .../internals/MeteredVersionedKeyValueStore.java | 9 +- .../state/internals/MeteredWindowStore.java | 64 +- .../internals/MeteredWindowStoreIterator.java | 36 +- .../internals/MeteredWindowedKeyValueIterator.java | 36 +- .../state/internals/TimestampedKeyAndJoinSide.java | 22 +- .../TimestampedKeyAndJoinSideDeserializer.java | 5 +- .../state/internals/metrics/StateStoreMetrics.java | 67 + .../apache/kafka/streams/KafkaStreamsWrapper.java | 1 - .../org/apache/kafka/streams/KeyValueTest.java | 38 +- .../apache/kafka/streams/StreamsBuilderTest.java | 26 +- .../apache/kafka/streams/StreamsConfigTest.java | 64 +- .../org/apache/kafka/streams/TopologyTest.java | 61 +- ...ighAvailabilityTaskAssignorIntegrationTest.java | 8 +- .../integration/RestoreIntegrationTest.java | 3 - .../integration/utils/EmbeddedKafkaCluster.java | 6 +- .../streams/integration/utils/KafkaEmbedded.java | 7 +- .../kafka/streams/internals/ApiUtilsTest.java | 6 +- .../kafka/streams/kstream/JoinWindowsTest.java | 8 +- .../kafka/streams/kstream/MaterializedTest.java | 8 +- .../apache/kafka/streams/kstream/NamedTest.java | 6 +- .../apache/kafka/streams/kstream/PrintedTest.java | 12 +- .../kstream/RepartitionTopicNamingTest.java | 6 +- .../kstream/SessionWindowedDeserializerTest.java | 8 +- .../kstream/SessionWindowedSerializerTest.java | 8 +- .../kafka/streams/kstream/SessionWindowsTest.java | 8 +- .../kafka/streams/kstream/SlidingWindowsTest.java | 6 +- .../kafka/streams/kstream/SuppressedTest.java | 2 +- .../kstream/TimeWindowedDeserializerTest.java | 8 +- .../kstream/TimeWindowedSerializerTest.java | 8 +- .../kafka/streams/kstream/TimeWindowsTest.java | 10 +- .../streams/kstream/UnlimitedWindowsTest.java | 8 +- .../apache/kafka/streams/kstream/WindowTest.java | 8 +- .../kafka/streams/kstream/WindowedSerdesTest.java | 10 +- .../kstream/internals/KStreamKStreamJoinTest.java | 3 +- .../ForeignTableJoinProcessorSupplierTests.java | 203 ++ .../ResponseJoinProcessorSupplierTest.java | 4 +- ... => SubscriptionJoinProcessorSupplierTest.java} | 2 +- .../SubscriptionSendProcessorSupplierTest.java | 353 +++ .../processor/FailOnInvalidTimestampTest.java | 6 +- .../LogAndSkipOnInvalidTimestampTest.java | 2 +- .../kafka/streams/processor/ReadOnlyStoreTest.java | 2 +- .../UsePartitionTimeOnInvalidTimestampTest.java | 4 +- .../processor/WallclockTimestampExtractorTest.java | 2 +- .../internals/DefaultStateUpdaterTest.java | 674 ++-- .../processor/internals/RecordCollectorTest.java | 16 + .../processor/internals/StreamTaskTest.java | 552 ++-- .../processor/internals/StreamThreadTest.java | 71 +- .../processor/internals/StreamsProducerTest.java | 22 + .../processor/internals/TaskAndActionTest.java | 38 +- .../processor/internals/TaskManagerTest.java | 1308 +++----- .../streams/processor/internals/TasksTest.java | 205 -- .../assignment/KafkaStreamsStateTest.java | 91 + .../assignment/RackAwareGraphConstructorTest.java | 8 +- .../streams/query/MultiVersionedKeyQueryTest.java | 6 +- .../kafka/streams/query/PositionBoundTest.java | 10 +- .../apache/kafka/streams/query/PositionTest.java | 8 +- .../kafka/streams/query/StateQueryResultTest.java | 2 +- .../kafka/streams/query/VersionedKeyQueryTest.java | 6 +- .../apache/kafka/streams/state/HostInfoTest.java | 10 +- .../kafka/streams/state/NoOpWindowStore.java | 2 +- .../kafka/streams/state/StateSerdesTest.java | 8 +- .../org/apache/kafka/streams/state/StoresTest.java | 6 +- .../kafka/streams/state/StreamsMetadataTest.java | 6 +- .../state/internals/MeteredKeyValueStoreTest.java | 91 +- .../state/internals/MeteredSessionStoreTest.java | 88 +- .../MeteredTimestampedKeyValueStoreTest.java | 90 +- .../MeteredVersionedKeyValueStoreTest.java | 106 + .../state/internals/MeteredWindowStoreTest.java | 91 +- .../TimestampedKeyAndJoinSideSerializerTest.java | 6 +- .../streams/tests/RelationalSmokeTestTest.java | 9 +- .../kafka/streams/tests/SystemTestUtilTest.java | 16 +- tests/kafkatest/services/kafka/kafka.py | 17 +- ...test.py => transactions_mixed_versions_test.py} | 210 +- tests/kafkatest/tests/core/transactions_test.py | 48 +- ...ctions_test.py => transactions_upgrade_test.py} | 230 +- .../streams/streams_application_upgrade_test.py | 7 +- .../streams/streams_broker_compatibility_test.py | 1 + .../tests/streams/streams_upgrade_test.py | 15 +- tests/kafkatest/utils/transactions_utils.py | 55 + .../apache/kafka/tools/MetadataQuorumCommand.java | 5 +- .../apache/kafka/tools/TransactionsCommand.java | 2 +- .../org/apache/kafka/tools/VerifiableConsumer.java | 5 +- .../tools/consumer/ConsoleConsumerOptions.java | 21 +- .../tools/consumer/group/ConsumerGroupCommand.java | 64 +- .../group/ConsumerGroupCommandOptions.java | 12 +- .../kafka/tools/consumer/group/GroupState.java | 5 +- .../tools/reassign/ReassignPartitionsCommand.java | 4 +- .../kafka/tools/DeleteRecordsCommandTest.java | 23 +- .../org/apache/kafka/tools/FeatureCommandTest.java | 82 +- .../org/apache/kafka/tools/GetOffsetShellTest.java | 3 +- .../kafka/tools/LeaderElectionCommandTest.java | 3 +- .../kafka/tools/MetadataQuorumCommandTest.java | 39 +- .../kafka/tools/ReplicaVerificationToolTest.java | 4 +- .../tools/consumer/ConsoleConsumerOptionsTest.java | 27 + .../group/ConsumerGroupCommandTestUtils.java | 116 + .../consumer/group/ConsumerGroupServiceTest.java | 13 +- .../consumer/group/DeleteConsumerGroupsTest.java | 494 +-- ...OffsetsConsumerGroupCommandIntegrationTest.java | 114 +- .../consumer/group/DescribeConsumerGroupTest.java | 91 +- .../group/ResetConsumerGroupOffsetTest.java | 5 +- .../java/org/apache/kafka/tools/api/Decoder.java | 12 +- .../org/apache/kafka/tools/api/DefaultDecoder.java | 15 +- .../org/apache/kafka/tools/api/IntegerDecoder.java | 15 +- .../org/apache/kafka/tools/api/LongDecoder.java | 15 +- .../org/apache/kafka/tools/api/StringDecoder.java | 15 +- .../java/org/apache/kafka/trogdor/agent/Agent.java | 3 +- .../apache/kafka/trogdor/basic/BasicPlatform.java | 9 +- .../apache/kafka/trogdor/common/WorkerUtils.java | 3 +- .../kafka/trogdor/coordinator/TaskManager.java | 8 +- .../trogdor/fault/ProcessStopFaultWorker.java | 4 +- .../kafka/trogdor/workload/RoundTripWorker.java | 4 +- .../kafka/trogdor/basic/BasicPlatformTest.java | 4 +- .../trogdor/common/CapturingCommandRunner.java | 5 +- .../kafka/trogdor/common/MiniTrogdorCluster.java | 3 +- .../kafka/trogdor/coordinator/CoordinatorTest.java | 3 +- 912 files changed, 40233 insertions(+), 20675 deletions(-) copy .github/workflows/{docker_build_and_test.yml => docker_official_image_build_and_test.yml} (87%) copy .github/workflows/{docker_build_and_test.yml => prepare_docker_official_image_source.yml} (57%) create mode 100644 checkstyle/import-control-group-coordinator.xml create mode 100644 clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CompletableEventReaper.java create mode 100644 clients/src/main/java/org/apache/kafka/common/compress/Compression.java create mode 100644 clients/src/main/java/org/apache/kafka/common/compress/GzipCompression.java copy clients/src/main/java/org/apache/kafka/common/compress/{SnappyFactory.java => GzipOutputStream.java} (51%) rename clients/src/main/java/org/apache/kafka/common/compress/{KafkaLZ4BlockInputStream.java => Lz4BlockInputStream.java} (95%) rename clients/src/main/java/org/apache/kafka/common/compress/{KafkaLZ4BlockOutputStream.java => Lz4BlockOutputStream.java} (85%) create mode 100644 clients/src/main/java/org/apache/kafka/common/compress/Lz4Compression.java rename clients/src/main/java/org/apache/kafka/common/compress/{SnappyFactory.java => NoCompression.java} (57%) create mode 100644 clients/src/main/java/org/apache/kafka/common/compress/SnappyCompression.java create mode 100644 clients/src/main/java/org/apache/kafka/common/compress/ZstdCompression.java delete mode 100644 clients/src/main/java/org/apache/kafka/common/compress/ZstdFactory.java copy raft/src/main/resources/common/message/QuorumStateData.json => clients/src/main/resources/common/message/KRaftVersionRecord.json (58%) create mode 100644 clients/src/main/resources/common/message/VotersRecord.json copy clients/src/test/java/org/apache/kafka/clients/admin/{TopicCollectionTest.java => DescribeTopicsResultTest.java} (51%) create mode 100644 clients/src/test/java/org/apache/kafka/clients/consumer/internals/events/CompletableEventReaperTest.java create mode 100644 clients/src/test/java/org/apache/kafka/common/compress/GzipCompressionTest.java rename clients/src/test/java/org/apache/kafka/common/compress/{KafkaLZ4Test.java => Lz4CompressionTest.java} (72%) create mode 100644 clients/src/test/java/org/apache/kafka/common/compress/NoCompressionTest.java create mode 100644 clients/src/test/java/org/apache/kafka/common/compress/SnappyCompressionTest.java create mode 100644 clients/src/test/java/org/apache/kafka/common/compress/ZstdCompressionTest.java delete mode 100644 clients/src/test/java/org/apache/kafka/common/record/CompressionTypeTest.java create mode 100644 connect/mirror/src/main/java/org/apache/kafka/connect/mirror/CheckpointStore.java create mode 100644 connect/mirror/src/test/java/org/apache/kafka/connect/mirror/CheckpointStoreTest.java create mode 100644 connect/runtime/src/test/java/org/apache/kafka/connect/storage/ConnectorOffsetBackingStoreTest.java create mode 100644 connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/FieldSyntaxVersion.java create mode 100644 connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java create mode 100644 connect/transforms/src/test/java/org/apache/kafka/connect/transforms/field/FieldPathNotationTest.java create mode 100644 connect/transforms/src/test/java/org/apache/kafka/connect/transforms/field/FieldSyntaxVersionTest.java create mode 100644 connect/transforms/src/test/java/org/apache/kafka/connect/transforms/field/SingleFieldPathTest.java delete mode 100644 core/src/main/java/kafka/server/ReplicaAlterLogDirsTierStateMachine.java delete mode 100644 core/src/main/java/kafka/server/ReplicaFetcherTierStateMachine.java delete mode 100644 core/src/main/scala/kafka/utils/NotNothing.scala delete mode 100644 core/src/main/scala/kafka/utils/Throttler.scala create mode 100644 core/src/test/java/kafka/admin/ConfigCommandTest.java create mode 100644 core/src/test/java/kafka/admin/UserScramCredentialsCommandTest.java create mode 100644 core/src/test/java/kafka/test/junit/ClusterTestExtensionsUnitTest.java create mode 100644 core/src/test/java/kafka/testkit/BrokerNodeTest.java create mode 100644 core/src/test/scala/integration/kafka/api/PlaintextConsumerCallbackTest.scala delete mode 100644 core/src/test/scala/kafka/utils/ToolsUtilsTest.scala delete mode 100644 core/src/test/scala/unit/kafka/admin/ConfigCommandTest.scala delete mode 100644 core/src/test/scala/unit/kafka/admin/UserScramCredentialsCommandTest.scala create mode 100644 core/src/test/scala/unit/kafka/server/ConsumerGroupDescribeRequestsTest.scala rename core/src/test/scala/unit/kafka/server/{ReplicaFetcherTierStateMachineTest.scala => TierStateMachineTest.scala} (93%) delete mode 100755 core/src/test/scala/unit/kafka/utils/ThrottlerTest.scala create mode 100644 docker/docker_official_image_build_test.py create mode 100644 docker/docker_official_images/.gitkeep create mode 100644 docker/extract_docker_official_image_artifact.py create mode 100644 docker/generate_kafka_pr_template.py create mode 100644 docker/native/Dockerfile create mode 100755 docker/native/launch create mode 100644 docker/native/native-image-configs/jni-config.json create mode 100644 docker/native/native-image-configs/predefined-classes-config.json create mode 100644 docker/native/native-image-configs/proxy-config.json create mode 100644 docker/native/native-image-configs/reflect-config.json create mode 100644 docker/native/native-image-configs/resource-config.json create mode 100644 docker/native/native-image-configs/serialization-config.json create mode 100644 docker/prepare_docker_official_image_source.py rename group-coordinator/src/main/java/org/apache/kafka/coordinator/group/{Record.java => CoordinatorRecord.java} (91%) rename group-coordinator/src/main/java/org/apache/kafka/coordinator/group/{RecordHelpers.java => CoordinatorRecordHelpers.java} (87%) rename group-coordinator/src/main/java/org/apache/kafka/coordinator/group/{RecordSerde.java => CoordinatorRecordSerde.java} (91%) copy group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/{PartitionAssignorException.java => GroupSpec.java} (60%) create mode 100644 group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GroupSpecImpl.java copy group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/{PartitionAssignorException.java => SubscriptionType.java} (59%) create mode 100644 group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/TopicIds.java copy clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/LeaveOnCloseEvent.java => group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/Deserializer.java (62%) copy clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/LeaveOnCloseEvent.java => group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/Serializer.java (69%) rename group-coordinator/src/test/java/org/apache/kafka/coordinator/group/{RecordHelpersTest.java => CoordinatorRecordHelpersTest.java} (89%) rename group-coordinator/src/test/java/org/apache/kafka/coordinator/group/{RecordSerdeTest.java => CoordinatorRecordSerdeTest.java} (89%) rename group-coordinator/src/test/java/org/apache/kafka/coordinator/group/{RecordTest.java => CoordinatorRecordTest.java} (84%) copy group-coordinator/src/test/java/org/apache/kafka/coordinator/group/{MockPartitionAssignor.java => NoOpPartitionAssignor.java} (60%) create mode 100644 group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/GroupSpecImplTest.java create mode 100644 group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/TopicIdsTest.java create mode 100644 jmh-benchmarks/src/main/java/org/apache/kafka/jmh/assignor/AssignorBenchmarkUtils.java create mode 100644 jmh-benchmarks/src/main/java/org/apache/kafka/jmh/connect/ValuesBenchmark.java rename raft/src/main/java/org/apache/kafka/raft/{FileBasedStateStore.java => FileQuorumStateStore.java} (65%) create mode 100644 raft/src/main/java/org/apache/kafka/raft/internals/KRaftControlRecordStateMachine.java create mode 100644 raft/src/main/java/org/apache/kafka/raft/internals/LogHistory.java rename group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AssignmentSpec.java => raft/src/main/java/org/apache/kafka/raft/internals/ReplicaKey.java (55%) create mode 100644 raft/src/main/java/org/apache/kafka/raft/internals/TreeMapLogHistory.java create mode 100644 raft/src/main/java/org/apache/kafka/raft/internals/VoterSet.java create mode 100644 raft/src/main/java/org/apache/kafka/raft/internals/VoterSetHistory.java create mode 100644 raft/src/main/java/org/apache/kafka/snapshot/NotifyingRawSnapshotWriter.java create mode 100644 raft/src/test/java/org/apache/kafka/raft/ElectionStateTest.java delete mode 100644 raft/src/test/java/org/apache/kafka/raft/FileBasedStateStoreTest.java create mode 100644 raft/src/test/java/org/apache/kafka/raft/FileQuorumStateStoreTest.java create mode 100644 raft/src/test/java/org/apache/kafka/raft/internals/KRaftControlRecordStateMachineTest.java create mode 100644 raft/src/test/java/org/apache/kafka/raft/internals/TreeMapLogHistoryTest.java create mode 100644 raft/src/test/java/org/apache/kafka/raft/internals/VoterSetHistoryTest.java create mode 100644 raft/src/test/java/org/apache/kafka/raft/internals/VoterSetTest.java create mode 100644 raft/src/test/java/org/apache/kafka/snapshot/NotifyingRawSnapshotWriterTest.java create mode 100644 raft/src/test/java/org/apache/kafka/snapshot/RecordsSnapshotWriterTest.java delete mode 100644 server/src/main/java/org/apache/kafka/server/config/Defaults.java create mode 100644 server/src/main/java/org/apache/kafka/server/config/DelegationTokenManagerConfigs.java create mode 100644 server/src/main/java/org/apache/kafka/server/config/ServerConfigs.java create mode 100644 storage/src/main/java/org/apache/kafka/storage/internals/utils/Throttler.java create mode 100644 storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataManagerTestUtils.java delete mode 100644 storage/src/test/java/org/apache/kafka/server/log/remote/storage/InmemoryRemoteLogMetadataManager.java delete mode 100644 storage/src/test/java/org/apache/kafka/server/log/remote/storage/InmemoryRemoteStorageManager.java delete mode 100644 storage/src/test/java/org/apache/kafka/server/log/remote/storage/InmemoryRemoteStorageManagerTest.java create mode 100644 storage/src/test/java/org/apache/kafka/storage/internals/log/LogOffsetMetadataTest.java create mode 100644 storage/src/test/java/org/apache/kafka/storage/internals/utils/ThrottlerTest.java create mode 100644 storage/src/test/java/org/apache/kafka/tiered/storage/actions/AlterLogDirAction.java create mode 100644 storage/src/test/java/org/apache/kafka/tiered/storage/integration/AlterLogDirTest.java create mode 100644 storage/src/test/java/org/apache/kafka/tiered/storage/integration/FetchFromLeaderWithCorruptedCheckpointTest.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/errors/ErrorHandlerContext.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/errors/LogAndContinueProcessingExceptionHandler.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/errors/LogAndFailProcessingExceptionHandler.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/errors/ProcessingExceptionHandler.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/errors/internals/DefaultErrorHandlerContext.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinLeftSide.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinRightSide.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/ApplicationState.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/AssignmentConfigs.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/KafkaStreamsAssignment.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/KafkaStreamsState.java copy raft/src/test/java/org/apache/kafka/raft/MockQuorumStateStore.java => streams/src/main/java/org/apache/kafka/streams/processor/assignment/ProcessId.java (63%) create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/TaskAssignmentUtils.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/TaskAssignor.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/TaskInfo.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/TaskTopicPartition.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java delete mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/internals/PendingUpdateAction.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ApplicationStateImpl.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/DefaultTaskInfo.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/DefaultTaskTopicPartition.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/KafkaStreamsStateImpl.java create mode 100644 streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackUtils.java copy clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/LeaveOnCloseEvent.java => streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredIterator.java (66%) create mode 100644 streams/src/test/java/org/apache/kafka/streams/kstream/internals/foreignkeyjoin/ForeignTableJoinProcessorSupplierTests.java rename streams/src/test/java/org/apache/kafka/streams/kstream/internals/foreignkeyjoin/{ForeignTableJoinProcessorSupplierTest.java => SubscriptionJoinProcessorSupplierTest.java} (99%) create mode 100644 streams/src/test/java/org/apache/kafka/streams/kstream/internals/foreignkeyjoin/SubscriptionSendProcessorSupplierTest.java create mode 100644 streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/KafkaStreamsStateTest.java copy tests/kafkatest/tests/core/{transactions_test.py => transactions_mixed_versions_test.py} (52%) copy tests/kafkatest/tests/core/{transactions_test.py => transactions_upgrade_test.py} (52%) create mode 100644 tests/kafkatest/utils/transactions_utils.py create mode 100644 tools/src/test/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommandTestUtils.java copy raft/src/main/java/org/apache/kafka/raft/internals/CloseListener.java => tools/tools-api/src/main/java/org/apache/kafka/tools/api/Decoder.java (81%) copy clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/LeaveOnCloseEvent.java => tools/tools-api/src/main/java/org/apache/kafka/tools/api/DefaultDecoder.java (74%) copy clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/LeaveOnCloseEvent.java => tools/tools-api/src/main/java/org/apache/kafka/tools/api/IntegerDecoder.java (73%) copy clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/LeaveOnCloseEvent.java => tools/tools-api/src/main/java/org/apache/kafka/tools/api/LongDecoder.java (74%) copy clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/AllTopicsMetadataEvent.java => tools/tools-api/src/main/java/org/apache/kafka/tools/api/StringDecoder.java (70%)