Hi All,
In order to support application integration without central coordinator like
Flink and Kafka Stream, transaction table need to be supported in SDK, and a
new type of segment called Online Segment is proposed.
Since it is hard to describe the motivation and design in a good format in
Hi Jacky,
Its a good idea to support writing transactional table from SDK. But we need
to add following limitations as well
1. It can work on file systems which can take append lock like HDFS.
2. Compaction, delete segment cannot be done on online segments till it is
converted to the transactio
Hi
Good idea, thank you started this discussion.
Agree with Ravi comments, we need to double-check some limitations after
introducing the feature.
Flink and Kafka integration can be discussed later.
For using SDK to write new data to the existing carbondata table , some
questions:
1.How to ensu
Hi Jacky,
Carbon should support transactional table in SDK before
ApacheFlink-Carbondata Integration.After having online segment, I can use
this feature to implement ApacheFlink-CarbonData integration.Therefore, can
I participate in the development of this feature,facilitating the
integration of Ap
> 在 2018年12月7日，下午11:05，ravipesala 写道：
>
> Hi Jacky,
>
> Its a good idea to support writing transactional table from SDK. But we need
> to add following limitations as well
> 1. It can work on file systems which can take append lock like HDFS.
Likun: yes, since we need to overwrite table stat
Hi Nicholas,
Yes, this is a feature required for flink-carbon to write to transactional
table. You are welcomed to participate in this. I think you can contribute by
reviewing the design doc in CARBONDATA-3152 firstly, after we settle down the
API we can open sub-tasks for this ticket.
Regar
> 在 2018年12月8日，下午3:53，Liang Chen 写道：
>
> Hi
>
> Good idea, thank you started this discussion.
>
> Agree with Ravi comments, we need to double-check some limitations after
> introducing the feature.
>
> Flink and Kafka integration can be discussed later.
> For using SDK to write new data to
Hi Jacky,
I have already reviewed the design doc in CARBONDATA-3152.What is the
current progress of supporting transactional table in SDK？In my option,
creating online segement firstly is necessary for now.
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.c
