--Original Mail --
Sender:Joern Kottmann
Send Date:Tue Dec 7 16:58:03 2021
Recipients:Yun Gao
CC:vtygoss , Alexander Preuß ,
user@flink.apache.org
Subject:Re: Re: Re: how to run streaming process after batch process is
completed?
Hello,
One
Hello,
One of the applications Spire [1] is using Flink for is to process AIS [2]
data collected by our satellites and from other sources. AIS is
transmitting a ships' static and dynamic information, such as names,
callsigns or positions. One of the challenges processing AIS data is that
there
@flink.apache.org
Subject:Re: Re: how to run streaming process after batch process is completed?
Hello,
Are there plans to support checkpoints for batch mode? I currently load the
state back via the DataStream API, but this gets more and more complicated and
doesn't always lead to a perfect state restore
//cwiki.apache.org/confluence/display/FLINK/FLIP-150%3A+Introduce+Hybrid+Source
>
> --Original Mail --
> *Sender:*vtygoss
> *Send Date:*Wed Dec 1 17:52:17 2021
> *Recipients:*Alexander Preuß
> *CC:*user@flink.apache.org
> *Subject:*Re: how to run
2021
Recipients:Alexander Preuß
CC:user@flink.apache.org
Subject:Re: how to run streaming process after batch process is completed?
Hi Alexander,
This is my ideal data pipeline.
- 1. Sqoop transfer bounded data from database to hive. And I think flink batch
process is more efficient than
Hi Alexander,
This is my ideal data pipeline.
- 1. Sqoop transfer bounded data from database to hive. And I think flink batch
process is more efficient than streaming process, so i want to process this
bounded data in batch mode and write result in HiveTable2.
- 2. There ares some tools to
Hi Vtygoss,
Can you explain a bit more about your ideal pipeline? Is the batch data
bounded data or could you also process it in streaming execution mode? And
is the streaming data derived from the batch data or do you just want to
ensure that the batch has been finished before running the
Hi, community!
By Flink, I want to unify batch process and streaming process in data
production pipeline. Batch process is used to process inventory data, then
streaming process is used to process incremental data. But I meet a problem,
there is no state in batch and the result is error if i
