Hi Jagan, I have a Java version of such a Job if you want (that requires
table-api in order to handle null values), I hope this could help!
Obviously you'll have to set all parameters properly..(e.g. jdbcOutputFormat ,
fieldTypes, etc...).
public static void main(String[] args) throws Exception {
String path = "file:/tmp/myFile.csv";
String rowDelim = CsvInputFormat.DEFAULT_LINE_DELIMITER;
String fieldDelim = CsvInputFormat.DEFAULT_FIELD_DELIMITER;
String[] fieldNames = "Column 1,Column 2,Column
3,Column4".split(fieldDelim);
Character quoteCharacter = '"';
boolean ignoreFirstLine = Boolean.TRUE;
String ignoreComments = null;
boolean lenient = Boolean.FALSE;
TypeInformation<?>[] fieldTypes = new TypeInformation<?>[] {
BasicTypeInfo.STRING_TYPE_INFO,
BasicTypeInfo.INT_TYPE_INFO,
BasicTypeInfo.STRING_TYPE_INFO,
BasicTypeInfo.STRING_TYPE_INFO
};
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
CsvTableSource csvTableSource = new
CsvTableSource(path,fieldNames,fieldTypes,fieldDelim,
rowDelim, quoteCharacter, ignoreFirstLine, ignoreComments, lenient);
DataSet<Row> csvDataSet = csvTableSource.getDataSet(env);
JDBCOutputFormat jdbcOutputFormat =
JDBCOutputFormat.buildJDBCOutputFormat()
.setDrivername("org.postgresql.Driver")
.setDBUrl("jdbc:postgresql://localhost/test?user=xxx&password=xxx")
.setQuery("insert into %s (id, title, author, price, qty) values
(?,?,?,?,?)") .finish(); csvDataSet.output(jdbcOutputFormat);
}
Best,
Flavio
On Fri, Sep 23, 2016 at 12:52 AM, Jagan <jaga...@gmail.com> wrote:
> Hi Guys,
>
> We have requirement like – loading data from local CSV file to Postgress
> database using Flink Scala…
>
> Do you have any sample Flink scala code for this?
>
> We have tried and searched in Google/Flinkweb website for data load, we
> haven’t found any sample code for this requisite.
>
> Code: Flink Scala.
>
> Load: from CSV to local postgres database.
>
> Thanks
>
> Jagan
>
> On 22 September 2016 at 20:37, Jagan <jaga...@gmail.com> wrote:
>
> > Hi Team,
> >
> > Will you be able to guide me on this? Is this a known issue that we
> > can't implement dataload in flink scala ?
> >
> > data load from csv to postgress or any relational database in
> Flink
> > Scala
> >
> > Thanks
> >
> > Jagan.
> >
> > On 22 September 2016 at 20:15, Jagan <jaga...@gmail.com> wrote:
> >
> >> Thanks Suneel,
> >>
> >> but client want to implement the data load in Flink Scala..
> >>
> >>
> >> On 22 September 2016 at 20:07, Suneel Marthi <smar...@apache.org>
> wrote:
> >>
> >>> Couldn't u use SQLLoader or something for doing that?
> >>>
> >>> http://stackoverflow.com/questions/2987433/how-to-import-csv
> >>> -file-data-into-a-postgresql-table
> >>>
> >>>
> >>>
> >>> On Thu, Sep 22, 2016 at 3:01 PM, Jagan <jaga...@gmail.com> wrote:
> >>>
> >>> > Hi Guys,
> >>> >
> >>> > We have a requirement like – loading data from local CSV file to
> >>> Postgress
> >>> > database using Flink Scala…We have tried number of ways all failed
> >>> >
> >>> > Do you have any example for this? With dependency libraries to
> >>> understand
> >>> > how to load data from CSV to postgres
> >>> >
> >>> > We have tried and searched in Google/Flinkweb website for data load,
> we
> >>> > haven’t found any sample code for this requisite.
> >>> >
> >>> > Code: Flink Scala.
> >>> >
> >>> > Load: from CSV to local postgres database.
> >>> >
> >>> > Thanks
> >>> >
> >>> > Jagan
> >>> >
> >>> > 0044-7411239688
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Regards.
> >>> > Jagan.
> >>> >
> >>> > ------------------------------------------------------------
> >>> > ---------------------------------------------------------
> >>> > ***** The Good You Do Today, People will often forget tomorrow: Do
> >>> Good
> >>> > Anyway ****
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> Regards.
> >> Jagan.
> >>
> >> ------------------------------------------------------------
> >> ---------------------------------------------------------
> >> ***** The Good You Do Today, People will often forget tomorrow: Do Good
> >> Anyway ****
> >>
> >
> >
> >
> > --
> > Regards.
> > Jagan.
> >
> > ------------------------------------------------------------
> > ---------------------------------------------------------
> > ***** The Good You Do Today, People will often forget tomorrow: Do Good
> > Anyway ****
> >
>
>
>
> --
> Regards.
> Jagan.
>
> ------------------------------------------------------------
> ---------------------------------------------------------
> ***** The Good You Do Today, People will often forget tomorrow: Do Good
> Anyway ****
>
