Flink scala data sink code from CSV to postgres.

2016-09-23 Thread Jagan
Hi Guys,

We have a requirement like loading data from CSV file to postgress database
In Flink scala.

We need very basic script how to read data from file and write it to
postgress database.

Could you please some one help me on that guys
-- 
Regards.
Jagan.


Re: Flink Scala - data load from CSV to postgress database.

2016-09-23 Thread Jagan
Hi Flavio,

Thanks this script is very useful, however we need Flink scala script to
load data from csv to postgress.

if we have following java equivalent code in scala that will be really
helpful...thanks.

myResult.output(
// build and configure OutputFormat
JDBCOutputFormat.buildJDBCOutputFormat()
.setDrivername(" ")
.setDBUrl(" ")
.setQuery("insert into persons (name, age, height)
values (?,?,?)")
.finish()
);



On 23 September 2016 at 08:35, Flavio Pompermaier 
wrote:

> Hi Jagan, I have a Java version of such a Job if you want (that requires
> table-api in order to handle null values), I hope this could help!
> Obviously you'll have to set all parameters properly..(e.g.
> jdbcOutputFormat ,
> fieldTypes, etc...).
>
>
> public static void main(String[] args) throws Exception {
> String path = "file:/tmp/myFile.csv";
> String rowDelim = CsvInputFormat.DEFAULT_LINE_DELIMITER;
> String fieldDelim = CsvInputFormat.DEFAULT_FIELD_DELIMITER;
> String[] fieldNames = "Column 1,Column 2,Column
> 3,Column4".split(fieldDelim);
>
> Character quoteCharacter = '"';
> boolean ignoreFirstLine = Boolean.TRUE;
> String ignoreComments = null;
> boolean lenient = Boolean.FALSE;
>
> TypeInformation[] fieldTypes = new TypeInformation[] {
> BasicTypeInfo.STRING_TYPE_INFO,
> BasicTypeInfo.INT_TYPE_INFO,
> BasicTypeInfo.STRING_TYPE_INFO,
> BasicTypeInfo.STRING_TYPE_INFO
> };
>
> ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
>
> CsvTableSource csvTableSource = new
> CsvTableSource(path,fieldNames,fieldTypes,fieldDelim,
>rowDelim, quoteCharacter, ignoreFirstLine, ignoreComments, lenient);
> DataSet csvDataSet = csvTableSource.getDataSet(env);
> JDBCOutputFormat jdbcOutputFormat =
> JDBCOutputFormat.buildJDBCOutputFormat()
> .setDrivername("org.postgresql.Driver")
> .setDBUrl("jdbc:postgresql://localhost/test?user=xxx&password=xxx")
> .setQuery("insert into %s (id, title, author, price, qty) values
> (?,?,?,?,?)") .finish(); csvDataSet.output(jdbcOutputFormat);
> }
>
> Best,
> Flavio
>
> On Fri, Sep 23, 2016 at 12:52 AM, Jagan  wrote:
>
> > Hi Guys,
> >
> > We have requirement like – loading data from local CSV file to Postgress
> > database using Flink Scala…
> >
> > Do you have any sample Flink scala code for this?
> >
> > We have tried and searched in Google/Flinkweb website for data load, we
> > haven’t found any sample code for this requisite.
> >
> > Code: Flink Scala.
> >
> > Load: from CSV to local postgres database.
> >
> > Thanks
> >
> > Jagan
> >
> > On 22 September 2016 at 20:37, Jagan  wrote:
> >
> > > Hi Team,
> > >
> > >   Will you be able to guide me on this? Is this a known issue that
> we
> > > can't implement  dataload in flink scala ?
> > >
> > >data load from csv to postgress or any relational database in
> > Flink
> > > Scala
> > >
> > > Thanks
> > >
> > > Jagan.
> > >
> > > On 22 September 2016 at 20:15, Jagan  wrote:
> > >
> > >> Thanks Suneel,
> > >>
> > >> but client want to implement the data load in Flink Scala..
> > >>
> > >>
> > >> On 22 September 2016 at 20:07, Suneel Marthi 
> > wrote:
> > >>
> > >>> Couldn't u use SQLLoader or something for doing that?
> > >>>
> > >>> http://stackoverflow.com/questions/2987433/how-to-import-csv
> > >>> -file-data-into-a-postgresql-table
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Sep 22, 2016 at 3:01 PM, Jagan  wrote:
> > >>>
> > >>> > Hi Guys,
> > >>> >
> > >>> > We have a requirement like – loading data from local CSV file to
> > >>> Postgress
> > >>> > database using Flink Scala…We have tried number of ways all failed
> > >>> >
> > >>> > Do you have any example for this? With dependency libraries to
> > >>> understand
> > >>> > how to load data from CSV to postgres
> > >>> >
> > >>> > We have tried and searched in Google/Flinkweb website for data
> load,
> > we
> > >>> > haven’t fou

Re: Flink Scala - data load from CSV to postgress database.

2016-09-22 Thread Jagan
Hi Guys,

We have requirement like – loading data from local CSV file to Postgress
database using Flink Scala…

Do you have any sample Flink scala code for this?

We have tried and searched in Google/Flinkweb website for data load, we
haven’t found any sample code for this requisite.

Code: Flink Scala.

Load: from CSV to local postgres database.

Thanks

Jagan

On 22 September 2016 at 20:37, Jagan  wrote:

> Hi Team,
>
>   Will you be able to guide me on this? Is this a known issue that we
> can't implement  dataload in flink scala ?
>
>data load from csv to postgress or any relational database in Flink
> Scala
>
> Thanks
>
> Jagan.
>
> On 22 September 2016 at 20:15, Jagan  wrote:
>
>> Thanks Suneel,
>>
>> but client want to implement the data load in Flink Scala..
>>
>>
>> On 22 September 2016 at 20:07, Suneel Marthi  wrote:
>>
>>> Couldn't u use SQLLoader or something for doing that?
>>>
>>> http://stackoverflow.com/questions/2987433/how-to-import-csv
>>> -file-data-into-a-postgresql-table
>>>
>>>
>>>
>>> On Thu, Sep 22, 2016 at 3:01 PM, Jagan  wrote:
>>>
>>> > Hi Guys,
>>> >
>>> > We have a requirement like – loading data from local CSV file to
>>> Postgress
>>> > database using Flink Scala…We have tried number of ways all failed
>>> >
>>> > Do you have any example for this? With dependency libraries to
>>> understand
>>> > how to load data from CSV to postgres
>>> >
>>> > We have tried and searched in Google/Flinkweb website for data load, we
>>> > haven’t found any sample code for this requisite.
>>> >
>>> > Code: Flink Scala.
>>> >
>>> > Load: from CSV to local postgres database.
>>> >
>>> > Thanks
>>> >
>>> > Jagan
>>> >
>>> > 0044-7411239688
>>> >
>>> >
>>> >
>>> > --
>>> > Regards.
>>> > Jagan.
>>> >
>>> > 
>>> > -
>>> > *  The Good You Do Today, People will often forget tomorrow: Do
>>> Good
>>> > Anyway 
>>> >
>>>
>>
>>
>>
>> --
>> Regards.
>> Jagan.
>>
>> 
>> -
>> *  The Good You Do Today, People will often forget tomorrow: Do Good
>> Anyway 
>>
>
>
>
> --
> Regards.
> Jagan.
>
> 
> -
> *  The Good You Do Today, People will often forget tomorrow: Do Good
> Anyway 
>



-- 
Regards.
Jagan.

-
*  The Good You Do Today, People will often forget tomorrow: Do Good
Anyway 


Flink Scala data load from CSV to postgress database.

2016-09-22 Thread Jagan
Hi Guys,

We have requirement like – loading data from local CSV file to Postgress
database using Flink Scala…We have tried number of ways all faile

Do you have any example for this? With dependency libraries to understand
how to load data from CSV to postgres

We have tried and searched in Google/Flinkweb website for data load, we
haven’t found any sample code for this requisite.

Code: Flink Scala.

Load: from CSV to local postgres database.

Thanks

Jagan

0044-7411239688


Re: Flink Scala - data load from CSV to postgress database.

2016-09-22 Thread Jagan
Hi Team,

  Will you be able to guide me on this? Is this a known issue that we
can't implement  dataload in flink scala ?

   data load from csv to postgress or any relational database in Flink
Scala

Thanks

Jagan.

On 22 September 2016 at 20:15, Jagan  wrote:

> Thanks Suneel,
>
> but client want to implement the data load in Flink Scala..
>
>
> On 22 September 2016 at 20:07, Suneel Marthi  wrote:
>
>> Couldn't u use SQLLoader or something for doing that?
>>
>> http://stackoverflow.com/questions/2987433/how-to-import-
>> csv-file-data-into-a-postgresql-table
>>
>>
>>
>> On Thu, Sep 22, 2016 at 3:01 PM, Jagan  wrote:
>>
>> > Hi Guys,
>> >
>> > We have a requirement like – loading data from local CSV file to
>> Postgress
>> > database using Flink Scala…We have tried number of ways all failed
>> >
>> > Do you have any example for this? With dependency libraries to
>> understand
>> > how to load data from CSV to postgres
>> >
>> > We have tried and searched in Google/Flinkweb website for data load, we
>> > haven’t found any sample code for this requisite.
>> >
>> > Code: Flink Scala.
>> >
>> > Load: from CSV to local postgres database.
>> >
>> > Thanks
>> >
>> > Jagan
>> >
>> > 0044-7411239688
>> >
>> >
>> >
>> > --
>> > Regards.
>> > Jagan.
>> >
>> > 
>> > -
>> > *  The Good You Do Today, People will often forget tomorrow: Do Good
>> > Anyway 
>> >
>>
>
>
>
> --
> Regards.
> Jagan.
>
> 
> -
> *  The Good You Do Today, People will often forget tomorrow: Do Good
> Anyway 
>



-- 
Regards.
Jagan.

-
*  The Good You Do Today, People will often forget tomorrow: Do Good
Anyway 


Re: Flink Scala - data load from CSV to postgress database.

2016-09-22 Thread Jagan
Thanks Suneel,

but client want to implement the data load in Flink Scala..


On 22 September 2016 at 20:07, Suneel Marthi  wrote:

> Couldn't u use SQLLoader or something for doing that?
>
> http://stackoverflow.com/questions/2987433/how-to-
> import-csv-file-data-into-a-postgresql-table
>
>
>
> On Thu, Sep 22, 2016 at 3:01 PM, Jagan  wrote:
>
> > Hi Guys,
> >
> > We have a requirement like – loading data from local CSV file to
> Postgress
> > database using Flink Scala…We have tried number of ways all failed
> >
> > Do you have any example for this? With dependency libraries to understand
> > how to load data from CSV to postgres
> >
> > We have tried and searched in Google/Flinkweb website for data load, we
> > haven’t found any sample code for this requisite.
> >
> > Code: Flink Scala.
> >
> > Load: from CSV to local postgres database.
> >
> > Thanks
> >
> > Jagan
> >
> > 0044-7411239688
> >
> >
> >
> > --
> > Regards.
> > Jagan.
> >
> > ----
> > -
> > *  The Good You Do Today, People will often forget tomorrow: Do Good
> > Anyway 
> >
>



-- 
Regards.
Jagan.

-
*  The Good You Do Today, People will often forget tomorrow: Do Good
Anyway 


Fwd: Flink Scala - data load from CSV to postgress database.

2016-09-22 Thread Jagan
Hi Guys,

We have a requirement like – loading data from local CSV file to Postgress
database using Flink Scala…We have tried number of ways all failed

Do you have any example for this? With dependency libraries to understand
how to load data from CSV to postgres

We have tried and searched in Google/Flinkweb website for data load, we
haven’t found any sample code for this requisite.

Code: Flink Scala.

Load: from CSV to local postgres database.

Thanks

Jagan

0044-7411239688



-- 
Regards.
Jagan.

-
*  The Good You Do Today, People will often forget tomorrow: Do Good
Anyway