Hello,
I think you can start by reading `clients` module, starting from Producer and
Consumer and AdminClient, there are many related blogs which can be found by
google but may be outdated since Kafka are updating very quickly, a good new is
that Kafka have very comprehensive test cases and you can run them locally.
Then you can read `core` and other modules.
Here are some of my experience:
1. Build kafka using gradle build tool
2. KafkaProducer
2.1 KafkaProducer API
2.2 KafkaProducer class
2.2.1 ProducerInterceptor
2.2.2 ProcuderMetadata
2.2.3 Serializer and Deserializer
2.2.4 Partitioner
2.3 RecordAccumulator
2.3.1 MemoryRecords
2.3.4 RecordBatch
2.3.3 BufferPool
2.3.4 RecordAccumulator
2.4 Sender
2.4.1 Request
2.4.2 Selector
2.4.3 InFlightRequests
2.4.4 MetadataUpdater
2.4.5 NetworkClient
3. KafkaConsumer
3.1 KafkaConsumer API
3.2 delivery guarantee semantic
3.3 Consumer group rebalance
3.4 KafkaConsumer
3.4.1 ConsumerNetworkClient
3.4.2 SubscriptionState
3.4.3 ConsumerCoordinator
3.4.4 PartitionAssignor
3.4.5 Heartbeat Request
3.4.6 Reblance request
3.4.7 offset request
3.4.8 Fetcher
3.4.9 overall
4. Core module
4.1 network
4.1.1 reactor pattern
4.1.2 SockerServer
4.1.3 AbstractServerThread
4.1.4 Acceptor
4.1.5 Processor
4.1.6 RequestChannel
4.2 API
4.2.1 KafkaRequestHandler
4.2.2 KafkaApis
4.3 Log
4.3.1 document
4.3.2 FileMessageSet
4.3.3 ButeBufferMessageSet
4.3.4 OffsetIndex
4.3.5 LogSegment
4.3.6 Log
4.3.7 LogManager
4.4. DelayedOperationPugatory
4.4.1 TimingWheel
4.4.2 SystemTimer
4.4.3 DelayedOperation
4.4.4 DelayedOperationPugatory
4.4.5 DelayedFetch
4.4.6 DelayedProcuce
4.5 replica
4.5.1 Replica
4.5.2 Partition
4.5.3 ReplicaManager
4.6 kafkaController (deprecated, using KRaft instead)
4.6.1 ControllerChannelManager
4.6.2 ControllerContext
4.6.3 ControllerBrokerRequestBatch
4.6.4 PartitionStateMachine
4.6.5 PartitionLeaderSelectir
4.6.6 REplicateStateMachine
4.6.7 Zookeeper Listener
4.6.8 KafkaController startup and failover
4.6.9 ControlledShutdownRequest
4.7 GroupCoordinator
4.7.1 GroupMetadataManager
4.7.2 GroupCoordinator
5. Acl and authentication
6. Jmx
7. Kafka tool
8. raft/metadata/stream/connect modules
Some classes may have deprecated or renamed, you can using git to check their
change history.
--
Best,
Ziming
> On Jan 29, 2023, at 21:35, Nelson Bighetti wrote:
>
> Hello everyone! :)
>
> I've just started reading the code but it appears a little overwhelming to
> me. Is there any documentation describing the directory structure? Or maybe
> could anybody suggest a good starting point to read the code?
>
> Thanks in advance.
