[jira] [Commented] (SPARK-19442) Unable to add column to the dataset using Dataset.WithColumn() api

2017-02-14 Thread Navya Krishnappa (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867357#comment-15867357
 ] 

Navya Krishnappa commented on SPARK-19442:
--

Thank you [~hyukjin.kwon]. 

It is satisfied my requirement. I could create a new column with blank values. 
:) 

> Unable to add column to the dataset using Dataset.WithColumn() api
> --
>
> Key: SPARK-19442
> URL: https://issues.apache.org/jira/browse/SPARK-19442
> Project: Spark
>  Issue Type: Bug
>  Components: Java API
>Affects Versions: 2.0.2
>Reporter: Navya Krishnappa
>
> When I'm creating a new column using Dataset.WithColumn() api, Analysis 
> Exception is thrown.
> Dataset.WithColumn() api: 
> Dataset.withColumn("newColumnName', new 
> org.apache.spark.sql.Column("newColumnName").cast("int"));
> Stacktrace: 
> cannot resolve '`NewColumn`' given input columns: [abc,xyz ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19442) Unable to add column to the dataset using Dataset.WithColumn() api

2017-02-14 Thread Hyukjin Kwon (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867277#comment-15867277
 ] 

Hyukjin Kwon commented on SPARK-19442:
--

ping [~Navya Krishnappa], would this satisfy your demand? 

> Unable to add column to the dataset using Dataset.WithColumn() api
> --
>
> Key: SPARK-19442
> URL: https://issues.apache.org/jira/browse/SPARK-19442
> Project: Spark
>  Issue Type: Bug
>  Components: Java API
>Affects Versions: 2.0.2
>Reporter: Navya Krishnappa
>
> When I'm creating a new column using Dataset.WithColumn() api, Analysis 
> Exception is thrown.
> Dataset.WithColumn() api: 
> Dataset.withColumn("newColumnName', new 
> org.apache.spark.sql.Column("newColumnName").cast("int"));
> Stacktrace: 
> cannot resolve '`NewColumn`' given input columns: [abc,xyz ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19442) Unable to add column to the dataset using Dataset.WithColumn() api

2017-02-13 Thread Hyukjin Kwon (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865256#comment-15865256
 ] 

Hyukjin Kwon commented on SPARK-19442:
--

How about something like this?

{code}
import org.apache.spark.sql.functions._

spark.range(10).withColumn("newColumnName", lit(null).cast("int")).show();
{code}

> Unable to add column to the dataset using Dataset.WithColumn() api
> --
>
> Key: SPARK-19442
> URL: https://issues.apache.org/jira/browse/SPARK-19442
> Project: Spark
>  Issue Type: Bug
>  Components: Java API
>Affects Versions: 2.0.2
>Reporter: Navya Krishnappa
>
> When I'm creating a new column using Dataset.WithColumn() api, Analysis 
> Exception is thrown.
> Dataset.WithColumn() api: 
> Dataset.withColumn("newColumnName', new 
> org.apache.spark.sql.Column("newColumnName").cast("int"));
> Stacktrace: 
> cannot resolve '`NewColumn`' given input columns: [abc,xyz ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19442) Unable to add column to the dataset using Dataset.WithColumn() api

2017-02-13 Thread Navya Krishnappa (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865135#comment-15865135
 ] 

Navya Krishnappa commented on SPARK-19442:
--

If the source file has 3 columns

NameAge Address
Abc10   Bangalore
Xyz10  Bangalore

After adding new column say "State". Resultant dataset should be 
 
NameAge AddressState
Abc10   Bangalore
Xyz10  Bangalore



> Unable to add column to the dataset using Dataset.WithColumn() api
> --
>
> Key: SPARK-19442
> URL: https://issues.apache.org/jira/browse/SPARK-19442
> Project: Spark
>  Issue Type: Bug
>  Components: Java API
>Affects Versions: 2.0.2
>Reporter: Navya Krishnappa
>
> When I'm creating a new column using Dataset.WithColumn() api, Analysis 
> Exception is thrown.
> Dataset.WithColumn() api: 
> Dataset.withColumn("newColumnName', new 
> org.apache.spark.sql.Column("newColumnName").cast("int"));
> Stacktrace: 
> cannot resolve '`NewColumn`' given input columns: [abc,xyz ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19442) Unable to add column to the dataset using Dataset.WithColumn() api

2017-02-13 Thread Hyukjin Kwon (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863601#comment-15863601
 ] 

Hyukjin Kwon commented on SPARK-19442:
--

Could I please ask a example like I did if I misunderstood?

> Unable to add column to the dataset using Dataset.WithColumn() api
> --
>
> Key: SPARK-19442
> URL: https://issues.apache.org/jira/browse/SPARK-19442
> Project: Spark
>  Issue Type: Bug
>  Components: Java API
>Affects Versions: 2.0.2
>Reporter: Navya Krishnappa
>
> When I'm creating a new column using Dataset.WithColumn() api, Analysis 
> Exception is thrown.
> Dataset.WithColumn() api: 
> Dataset.withColumn("newColumnName', new 
> org.apache.spark.sql.Column("newColumnName").cast("int"));
> Stacktrace: 
> cannot resolve '`NewColumn`' given input columns: [abc,xyz ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19442) Unable to add column to the dataset using Dataset.WithColumn() api

2017-02-13 Thread Navya Krishnappa (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863216#comment-15863216
 ] 

Navya Krishnappa commented on SPARK-19442:
--

Thank you for your response. 

It is working as expected. I could able to add a new column to the data set.

> Unable to add column to the dataset using Dataset.WithColumn() api
> --
>
> Key: SPARK-19442
> URL: https://issues.apache.org/jira/browse/SPARK-19442
> Project: Spark
>  Issue Type: Bug
>  Components: Java API
>Affects Versions: 2.0.2
>Reporter: Navya Krishnappa
>
> When I'm creating a new column using Dataset.WithColumn() api, Analysis 
> Exception is thrown.
> Dataset.WithColumn() api: 
> Dataset.withColumn("newColumnName', new 
> org.apache.spark.sql.Column("newColumnName").cast("int"));
> Stacktrace: 
> cannot resolve '`NewColumn`' given input columns: [abc,xyz ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org