Dataframe insertInto(tableName: String): Unit :Failure Scenario

2016-08-05 Thread java bigdata
Hi Team, I'm using Dataframe.insertInto(tableName: String): Unit api for hive table insertions . I would like to know what happens when out of 500 records, 100 records are inserted and failure occurs at 101th record. Will 100 records be inserted into Hive or will they rolled back , and any failur

Re: Dataframe Transformation with Inner fields in Complex Datatypes.

2016-07-19 Thread java bigdata
> Hi > > withColumn adds the column. If you want different name, please use > .alias() function. > > On Mon, Jul 18, 2016 at 2:16 AM, java bigdata > wrote: > >> Hi Team, >> >> I am facing a major issue while transforming dataframe containing complex >> d

Dataframe Transformation with Inner fields in Complex Datatypes.

2016-07-17 Thread java bigdata
Hi Team, I am facing a major issue while transforming dataframe containing complex datatype columns. I need to update the inner fields of complex datatype, for eg: converting one inner field to UPPERCASE letters, and return the same dataframe with new upper case values in it. Below is my issue des