Yingquan He created KAFKA-19743:
-----------------------------------
Summary: [DISCUSS] Exploring Compile-time AST Generation as an
Alternative Approach for Kafka Protocol Classes
Key: KAFKA-19743
URL: https://issues.apache.org/jira/browse/KAFKA-19743
Project: Kafka
Issue Type: Wish
Reporter: Yingquan He
h2. 1. Background & Motivation
Currently, Kafka uses a JSON template-based pre-generation approach to create
protocol classes under {{{}org.apache.kafka.common.message{}}}. This solution
is mature, stable, and provides excellent multi-language support.
However, I've been considering whether we could provide an alternative
*compile-time code generation* approach for the Java ecosystem, similar to the
AST modification technology used by Lombok. This approach might offer better
development experience in certain scenarios.
h2. 2. Proposal Overview
{*}Core Idea{*}: Use annotation processors to directly modify the AST during
compilation, dynamically generating complete implementations of protocol
classes instead of pre-generating Java source files.
{*}Example Implementation Concept{*}:
{code:java}
@KafkaMessage(apiKey = 1, version = "0.1.0")
public class FetchRequestSpec {
@ProtocolField(type = "string", order = 1)
private String groupId;
@ProtocolField(type = "int16", order = 2)
private short acks;
// More fields...
}{code}
h2. 3. Comparative Analysis
||Aspect||Current Template Approach||Compile-time AST Approach||Hybrid
Possibility||
|*Development Experience*|Requires viewing generated source files|Cleaner
source code, Lombok-like|AST during development, pre-gen for release|
|*Build Process*|Explicit pre-generation step|Integrated into standard
compilation|Configurable generation strategy|
|*Debugging Support*|✅ Full source-level debugging|⚠️ Requires IDE plugin
support|Source output on demand|
|*Multi-language Support*|✅ Single definition, multiple outputs|❌
Java-only|Preserve existing multi-language capabilities|
|*Performance Impact*|One-time generation at build time|Regeneration on each
compilation|Smart caching mechanism|
h2. 4. Potential Value
* {*}Faster Development Iteration{*}: Immediate compilation after protocol
definition changes, no pre-generation step required
* {*}Source Code Cleanliness{*}: Avoid polluting codebase with大量 generated
boilerplate code
* {*}Type Safety{*}: Stricter protocol constraint checking at compile time
* {*}Alignment with Modern Java Patterns{*}: Similar to Quarkus, Micronaut's
compile-time processing philosophy
h2. 5. Feasibility Considerations
I understand the potential challenges this approach might face:
* Compatibility with existing multi-language ecosystem
* Ensuring debugging experience
* Increased build system complexity
* Community learning curve
Therefore, I suggest we could:
# {*}Offer as Optional Approach{*}: Doesn't disrupt existing workflow, serves
as alternative during development
# {*}Incremental Implementation{*}: Start with small scope or new modules for
experimentation
# {*}Toolchain Support{*}: Provide source output mode to ensure debugging
capability
h2. 6. Next Steps & Recommendations
If the community finds this direction interesting, I'm willing to:
* Create a Proof of Concept (POC) implementation to demonstrate concrete
results
* Participate in relevant discussions and design work
* Assist with performance benchmarking and compatibility validation
Thank you for taking the time to review this proposal. I look forward to
further discussing this idea with the community, and whether it is ultimately
adopted or not, I appreciate the opportunity to share my thoughts with the
community.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
