[
https://issues.apache.org/jira/browse/KAFKA-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated KAFKA-10732:
------------------------------------
Description:
During the adoption of Kafka in large enterprises, it's important to guarantee
data in some topic conforms to the specific format.
When data are written and read by the different applications developed by the
different teams it's hard to guarantee data format using only custom SerDe,
because malicious applications can use different SerDe.
The data format can be enforced only on the broker side.
I propose to introduce the new public interface for it RecordsPolicy:
{noformat}
public interface RecordsPolicy extends Configurable, AutoCloseable {
void validate(String topic, Records records) throws PolicyViolationException;
}
{noformat}
and a two new configuration options:
* {{records.policy.class.name: String}} - sets class name of the implementation
of RecordsPolicy for the specific topic.
* {{records.policy.enabled: Boolean}} - enable or disable records policy for
the topic
If {{records.policy.enabled=true}} than an instance of the RecordsPolicy should
check each Records batch before applying data to the log.
If {{PolicyViolationException}} thrown from the validate method then no data
added to the log and the client receives an error.
was:
During the adoption of Kafka in large enterprises, it's important to guarantee
data in some topic conforms to the specific format.
When data are written and read by the different applications developed by the
different teams it's hard to guarantee data format using only custom SerDe,
because malicious applications can use different SerDe.
The data format can be enforced only on the broker side.
I propose to introduce the new public interface for it RecordsPolicy:
{noformat}
public interface RecordsPolicy extends Configurable, AutoCloseable {
void validate(String topic, Records records) throws PolicyViolationException;
}
{noformat}
and a two new configuration options:
* {{records.policy.class.name: String}} - sets class name of the implementation
of RecordsPolicy for the specific topic.
* {{records.policy.enabled: Boolean}} - enable or disable records policy for
the topic
> API to ensure Records policy on the broker
> ------------------------------------------
>
> Key: KAFKA-10732
> URL: https://issues.apache.org/jira/browse/KAFKA-10732
> Project: Kafka
> Issue Type: Improvement
> Reporter: Nikolay Izhikov
> Assignee: Nikolay Izhikov
> Priority: Major
>
> During the adoption of Kafka in large enterprises, it's important to
> guarantee data in some topic conforms to the specific format.
> When data are written and read by the different applications developed by the
> different teams it's hard to guarantee data format using only custom SerDe,
> because malicious applications can use different SerDe.
> The data format can be enforced only on the broker side.
> I propose to introduce the new public interface for it RecordsPolicy:
> {noformat}
> public interface RecordsPolicy extends Configurable, AutoCloseable {
> void validate(String topic, Records records) throws
> PolicyViolationException;
> }
> {noformat}
> and a two new configuration options:
> * {{records.policy.class.name: String}} - sets class name of the
> implementation of RecordsPolicy for the specific topic.
> * {{records.policy.enabled: Boolean}} - enable or disable records policy for
> the topic
> If {{records.policy.enabled=true}} than an instance of the RecordsPolicy
> should check each Records batch before applying data to the log.
> If {{PolicyViolationException}} thrown from the validate method then no data
> added to the log and the client receives an error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
