[
https://issues.apache.org/jira/browse/KAFKA-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Chanaud updated KAFKA-13511:
-----------------------------------
Description:
Currently, the SMT TimestampConverter can convert Timestamp from either source
String, Long or Date into target String, Long or Date.
The problem is that Long source or target is required to be epoch in
milliseconds.
In many cases, epoch is represented with different precisions. This leads to
several Jira tickets :
* KAFKA-12364
* KAFKA-10561
I propose to add a new config to TimestampConverter called "epoch.precision"
which defaults to "millis" so as to not impact existing code, and allows for
more precisions : seconds, millis, micros.
{code:json}
"transforms": "TimestampConverter",
"transforms.TimestampConverter.type":
"org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.TimestampConverter.field": "event_date",
"transforms.TimestampConverter.epoch.precision": "micros",
"transforms.TimestampConverter.target.type": "Timestamp"
{code}
Exactly like "format" field which is used as input when the source in String
and output when the target.type is string, this new field would be used as
input when the field is Long, and as output when the target.type is "unix"
was:
Currently, the SMT TimestampConverter can convert Timestamp from either source
String, Long or Date into target String, Long or Date.
The problem is that Long source or target is required to be epoch in
milliseconds.
In many cases, epoch is represented with different precisions. This leads to
several Jira tickets :
* KAFKA-12364
* KAFKA-10561
I propose to add a new config to TimestampConverter called "epoch.precision"
which defaults to "millis" so as to not impact existing code, and allows for
more precisions : seconds, millis, micros.
{code:json}
"transforms": "TimestampConverter",
"transforms.TimestampConverter.type":
"org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.TimestampConverter.field": "event_date",
"transforms.TimestampConverter.epoch.precision": "micros",
"transforms.TimestampConverter.target.type": "Timestamp"
{code}
> Update TimestampConverter SMT to support unix epoch as millis, micros, and
> seconds
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-13511
> URL: https://issues.apache.org/jira/browse/KAFKA-13511
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Julien Chanaud
> Priority: Minor
>
> Currently, the SMT TimestampConverter can convert Timestamp from either
> source String, Long or Date into target String, Long or Date.
> The problem is that Long source or target is required to be epoch in
> milliseconds.
> In many cases, epoch is represented with different precisions. This leads to
> several Jira tickets :
> * KAFKA-12364
> * KAFKA-10561
> I propose to add a new config to TimestampConverter called "epoch.precision"
> which defaults to "millis" so as to not impact existing code, and allows for
> more precisions : seconds, millis, micros.
> {code:json}
> "transforms": "TimestampConverter",
> "transforms.TimestampConverter.type":
> "org.apache.kafka.connect.transforms.TimestampConverter$Value",
> "transforms.TimestampConverter.field": "event_date",
> "transforms.TimestampConverter.epoch.precision": "micros",
> "transforms.TimestampConverter.target.type": "Timestamp"
> {code}
> Exactly like "format" field which is used as input when the source in String
> and output when the target.type is string, this new field would be used as
> input when the field is Long, and as output when the target.type is "unix"
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
