Hi Chesnay
Currently that is what i have done, reading the schema from database in
order to create a new table in jdbc database and writing the rows coming
from jdbcinputformat.
Overall i am trying to implement the solution which reads the streaming
data from one source which either could be coming from kafka, Jdbc,
Hive, Hdfs and writing those streaming data to output source which is
again could be any of those.
For a simple use case i have just taken one scenario using jdbc in and
jdbc out, Since the jdbc input source returns the datastream of Row and
to write them into jdbc database we have to create a table which
requires schema.
Thanks
Punit
On 02/08/2017 08:22 AM, Chesnay Schepler wrote:
Hello,
I don't understand why you explicitly need the schema since the batch
JDBCInput-/Outputformats don't require it.
That's kind of the nice thing about Rows.
Would be cool if you could tell us what you're planning to do with the
schema :)
In any case, to get the schema within the plan then you will have to
query the DB and build it yourself. Note that this
is executed on the client.
Regards,
Chesnay
On 08.02.2017 00:39, Punit Tandel wrote:
Hi Robert
Thanks for the response, So in near future release of the flink
version , is this functionality going to be implemented ?
Thanks
On 02/07/2017 04:12 PM, Robert Metzger wrote:
Currently, there is no streaming JDBC connector.
Check out this thread from last year:
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/JDBC-Streaming-Connector-td10508.html
On Mon, Feb 6, 2017 at 5:00 PM, Ufuk Celebi <u...@apache.org
<mailto:u...@apache.org>> wrote:
I'm not sure how well this works for the streaming API. Looping in
Chesnay, who worked on this.
On Mon, Feb 6, 2017 at 11:09 AM, Punit Tandel
<punit.tan...@ericsson.com> wrote:
> Hi ,
>
> I was looking into flink streaming api and trying to implement
the solution
> for reading the data from jdbc database and writing them to
jdbc databse
> again.
>
> At the moment i can see the datastream is returning Row from
the database.
> dataStream.getType().getGenericParameters() retuning an empty
list of
> collection.
>
> I am right now manually creating a database connection and
getting the
> schema from ResultMetadata and constructing the schema for the
table which
> is a bit heavy operation.
>
> So is there any other way to get the schema for the table in
order to create
> a new table and write those records in the database ?
>
> Please let me know
>
> Thanks
> Punit
