[jira] [Commented] (SPARK-18704) CrossValidator should preserve more tuning statistics

2016-12-04 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721523#comment-15721523 ] yuhao yang commented on SPARK-18704: Glad to have your attention. In https://github.

[jira] [Issue Comment Deleted] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18539: Comment: was deleted (was: Below is the test case you can try. {noformat} Seq("parquet").foreach { for

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721507#comment-15721507 ] Xiao Li commented on SPARK-18539: - Below is the test case you can try. {noformat} Se

[jira] [Comment Edited] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721499#comment-15721499 ] Xiao Li edited comment on SPARK-18539 at 12/5/16 7:42 AM: -- The e

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721499#comment-15721499 ] Xiao Li commented on SPARK-18539: - The error is from Parquet. {noformat} 16/11/22 17:43:4

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721488#comment-15721488 ] Reynold Xin commented on SPARK-18539: - Why don't we fix the parquet reader so it can

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721481#comment-15721481 ] Xiao Li commented on SPARK-18539: - The default of `spark.sql.parquet.mergeSchema` is fals

[jira] [Commented] (SPARK-18704) CrossValidator should preserve more tuning statistics

2016-12-04 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721448#comment-15721448 ] Nick Pentreath commented on SPARK-18704: Yeah, I like this idea. I've also been f

[jira] [Updated] (SPARK-18715) Fix wrong AIC calculation in Binomial GLM

2016-12-04 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wayne Zhang updated SPARK-18715: Description: The AIC calculation in Binomial GLM seems to be wrong when there are weights. The res

[jira] [Commented] (SPARK-18531) Apache Spark FPGrowth algorithm implementation fails with java.lang.StackOverflowError

2016-12-04 Thread Saleem Ansari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721440#comment-15721440 ] Saleem Ansari commented on SPARK-18531: --- [~yuhaoyan] Thanks for your suggestion. H

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Vitaly Gerasimov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721430#comment-15721430 ] Vitaly Gerasimov commented on SPARK-18539: -- I think this is another reason why w

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Vitaly Gerasimov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721423#comment-15721423 ] Vitaly Gerasimov commented on SPARK-18539: -- If we can neglect the performance wh

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721414#comment-15721414 ] Xiao Li commented on SPARK-18539: - FYI, I checked the other formats, CSV and JSON work as

[jira] [Updated] (SPARK-18715) Fix wrong AIC calculation in Binomial GLM

2016-12-04 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wayne Zhang updated SPARK-18715: Summary: Fix wrong AIC calculation in Binomial GLM (was: Correct AIC calculation in Binomial GLM)

[jira] [Resolved] (SPARK-18642) Spark SQL: Catalyst is scanning undesired columns

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-18642. --- Resolution: Fixed Fix Version/s: 2.0.0 I close this issue as a `Fixed` because of the

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721392#comment-15721392 ] Xiao Li commented on SPARK-18539: - Yeah. It is very slow when you have many many small pa

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Vitaly Gerasimov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721385#comment-15721385 ] Vitaly Gerasimov commented on SPARK-18539: -- Hmm.. How it works when we use schem

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721376#comment-15721376 ] Xiao Li commented on SPARK-18539: - Basically, we have to know whether a column exists or

[jira] [Created] (SPARK-18715) Correct AIC calculation in Binomial GLM

2016-12-04 Thread Wayne Zhang (JIRA)
Wayne Zhang created SPARK-18715: --- Summary: Correct AIC calculation in Binomial GLM Key: SPARK-18715 URL: https://issues.apache.org/jira/browse/SPARK-18715 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Vitaly Gerasimov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721372#comment-15721372 ] Vitaly Gerasimov commented on SPARK-18539: -- If I turn off `spark.sql.parquet.fil

[jira] [Commented] (SPARK-18349) Update R API documentation on ml model summary

2016-12-04 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721360#comment-15721360 ] Miao Wang commented on SPARK-18349: --- Will submit PR by tomorrow. Thanks! > Update R AP

[jira] [Assigned] (SPARK-18714) SparkSession.time - a simple timer function

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18714: Assignee: Reynold Xin (was: Apache Spark) > SparkSession.time - a simple timer function >

[jira] [Commented] (SPARK-18714) SparkSession.time - a simple timer function

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721349#comment-15721349 ] Apache Spark commented on SPARK-18714: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-18714) SparkSession.time - a simple timer function

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18714: Assignee: Apache Spark (was: Reynold Xin) > SparkSession.time - a simple timer function >

[jira] [Created] (SPARK-18714) Add a simple time function to SparkSession

2016-12-04 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18714: --- Summary: Add a simple time function to SparkSession Key: SPARK-18714 URL: https://issues.apache.org/jira/browse/SPARK-18714 Project: Spark Issue Type: New Feat

[jira] [Updated] (SPARK-18714) SparkSession.time - a simple timer function

2016-12-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18714: Summary: SparkSession.time - a simple timer function (was: Add a simple time function to SparkSess

[jira] [Closed] (SPARK-18712) keep the order of sql expression and support short circuit

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-18712. - Resolution: Won't Fix I'm closing this issue. You can reopen this if you have still another opini

[jira] [Commented] (SPARK-18712) keep the order of sql expression and support short circuit

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721324#comment-15721324 ] Dongjoon Hyun commented on SPARK-18712: --- Hi, [~yahsuan]. Thank you for making issue

[jira] [Assigned] (SPARK-18705) Docs for one-pass solver for linear regression with L1 and elastic-net penalties

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18705: Assignee: Apache Spark > Docs for one-pass solver for linear regression with L1 and elasti

[jira] [Commented] (SPARK-18705) Docs for one-pass solver for linear regression with L1 and elastic-net penalties

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721321#comment-15721321 ] Apache Spark commented on SPARK-18705: -- User 'sethah' has created a pull request for

[jira] [Assigned] (SPARK-18705) Docs for one-pass solver for linear regression with L1 and elastic-net penalties

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18705: Assignee: (was: Apache Spark) > Docs for one-pass solver for linear regression with L1

[jira] [Resolved] (SPARK-18702) input_file_block_start and input_file_block_length function

2016-12-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18702. - Resolution: Fixed Fix Version/s: 2.2.0 > input_file_block_start and input_file_block_lengt

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721316#comment-15721316 ] Xiao Li commented on SPARK-18539: - Could you turn off `spark.sql.parquet.filterPushdown`?

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721310#comment-15721310 ] Dongjoon Hyun commented on SPARK-18539: --- The use case makes sense. I see now! > Ca

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Vitaly Gerasimov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721300#comment-15721300 ] Vitaly Gerasimov commented on SPARK-18539: -- Thank you for your reply. I think we

[jira] [Created] (SPARK-18713) using SparkR build step wise regression model (glm)

2016-12-04 Thread Prasann modi (JIRA)
Prasann modi created SPARK-18713: Summary: using SparkR build step wise regression model (glm) Key: SPARK-18713 URL: https://issues.apache.org/jira/browse/SPARK-18713 Project: Spark Issue Typ

[jira] [Commented] (SPARK-18251) DataSet API | RuntimeException: Null value appeared in non-nullable field when holding Option Case Class

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721228#comment-15721228 ] koert kuipers commented on SPARK-18251: --- i think all of these arguments are very va

[jira] [Updated] (SPARK-18712) keep the order of sql expression and support short circuit

2016-12-04 Thread yahsuan, chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yahsuan, chang updated SPARK-18712: --- Summary: keep the order of sql expression and support short circuit (was: keep the order of

[jira] [Updated] (SPARK-18712) keep the order of sql expression

2016-12-04 Thread yahsuan, chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yahsuan, chang updated SPARK-18712: --- Description: The following python code fails with spark 2.0.2, but works with spark 1.5.2 {c

[jira] [Updated] (SPARK-18712) keep the order of sql expression

2016-12-04 Thread yahsuan, chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yahsuan, chang updated SPARK-18712: --- Description: The following python code fails with spark 2.0.2, but works with spark 1.5.2 {c

[jira] [Updated] (SPARK-18712) keep the order of sql expression

2016-12-04 Thread yahsuan, chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yahsuan, chang updated SPARK-18712: --- Summary: keep the order of sql expression (was: support short circuit for sql expression) >

[jira] [Updated] (SPARK-18712) support short circuit for sql expression

2016-12-04 Thread yahsuan, chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yahsuan, chang updated SPARK-18712: --- Description: The following python code fails with spark 2.0.2, but works with spark 1.5.2 {c

[jira] [Updated] (SPARK-18712) support short circuit for sql expression

2016-12-04 Thread yahsuan, chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yahsuan, chang updated SPARK-18712: --- Description: The following python code fails with spark 2.0.2, but works with spark 1.5.2 {c

[jira] [Resolved] (SPARK-18643) SparkR hangs at session start when installed as a package without SPARK_HOME set

2016-12-04 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-18643. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request

[jira] [Updated] (SPARK-18712) support short circuit for sql expression

2016-12-04 Thread yahsuan, chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yahsuan, chang updated SPARK-18712: --- Description: The following python code failed with spark 2.0.2, but worked with spark 1.5.2

[jira] [Created] (SPARK-18712) support short circuit for sql expression

2016-12-04 Thread yahsuan, chang (JIRA)
yahsuan, chang created SPARK-18712: -- Summary: support short circuit for sql expression Key: SPARK-18712 URL: https://issues.apache.org/jira/browse/SPARK-18712 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-18711) NPE in generated SpecificMutableProjection for Aggregator

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721175#comment-15721175 ] koert kuipers commented on SPARK-18711: --- simplified: {noformat} case class Holder

[jira] [Updated] (SPARK-18711) NPE in generated SpecificMutableProjection for Aggregator

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-18711: -- Description: this is a bug in the branch-2.1, but i don't think it was in 2.1.0-rc1 code (contr

[jira] [Updated] (SPARK-18711) NPE in generated SpecificMutableProjection for Aggregator

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-18711: -- Description: this is a bug in the branch-2.1, but i don't think it was in 2.1.0-rc1 code: {nofo

[jira] [Created] (SPARK-18711) NPE in generated SpecificMutableProjection for Aggregator

2016-12-04 Thread koert kuipers (JIRA)
koert kuipers created SPARK-18711: - Summary: NPE in generated SpecificMutableProjection for Aggregator Key: SPARK-18711 URL: https://issues.apache.org/jira/browse/SPARK-18711 Project: Spark I

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721065#comment-15721065 ] Dongjoon Hyun commented on SPARK-18539: --- Thank you so much! > Cannot filter by non

[jira] [Created] (SPARK-18710) Add offset to GeneralizedLinearRegression models

2016-12-04 Thread Wayne Zhang (JIRA)
Wayne Zhang created SPARK-18710: --- Summary: Add offset to GeneralizedLinearRegression models Key: SPARK-18710 URL: https://issues.apache.org/jira/browse/SPARK-18710 Project: Spark Issue Type: Ne

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720964#comment-15720964 ] Xiao Li commented on SPARK-18539: - The parquet filter push-down of Spark 2.x is different

[jira] [Commented] (SPARK-18705) Docs for one-pass solver for linear regression with L1 and elastic-net penalties

2016-12-04 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720892#comment-15720892 ] Seth Hendrickson commented on SPARK-18705: -- Yeah, I'll do it today :) > Docs fo

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720726#comment-15720726 ] Dongjoon Hyun commented on SPARK-18539: --- It looks like the predicates are pushed do

[jira] [Comment Edited] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-04 Thread Jakub Nowacki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720710#comment-15720710 ] Jakub Nowacki edited comment on SPARK-18699 at 12/4/16 10:26 PM: --

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-04 Thread Jakub Nowacki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720710#comment-15720710 ] Jakub Nowacki commented on SPARK-18699: --- While I don't argue that some other packag

[jira] [Closed] (SPARK-18707) Can spark support exactly once based kafka ? Due to these following question?

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-18707. - Resolution: Invalid Sorry, but JIRA is not for questions. It seems to me a usage question. Please

[jira] [Commented] (SPARK-18709) Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame with incompatible types for fields.

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720691#comment-15720691 ] Dongjoon Hyun commented on SPARK-18709: --- Also, I updated the fix version to 2.0.0 a

[jira] [Updated] (SPARK-18709) Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame with incompatible types for fields.

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18709: -- Fix Version/s: (was: 2.0.2) 2.0.0 > Automatic null conversion bug (inste

[jira] [Commented] (SPARK-18709) Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame with incompatible types for fields.

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720687#comment-15720687 ] Dongjoon Hyun commented on SPARK-18709: --- Hi, [~amogh.91]. I removed the target vers

[jira] [Updated] (SPARK-18709) Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame with incompatible types for fields.

2016-12-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18709: -- Target Version/s: (was: 1.6.3) > Automatic null conversion bug (instead of throwing error) wh

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720610#comment-15720610 ] koert kuipers commented on SPARK-15810: --- here is an example where (None,) gets read

[jira] [Updated] (SPARK-18709) Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame with incompatible types for fields.

2016-12-04 Thread Amogh Param (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amogh Param updated SPARK-18709: Summary: Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame w

[jira] [Created] (SPARK-18709) Failure to throw error and automatic null conversion bug when creating a Spark Datarame with incompatible types for fields.

2016-12-04 Thread Amogh Param (JIRA)
Amogh Param created SPARK-18709: --- Summary: Failure to throw error and automatic null conversion bug when creating a Spark Datarame with incompatible types for fields. Key: SPARK-18709 URL: https://issues.apache.org/

[jira] [Comment Edited] (SPARK-15810) Aggregator doesn't play nice with Option

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720557#comment-15720557 ] koert kuipers edited comment on SPARK-15810 at 12/4/16 8:36 PM: ---

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720557#comment-15720557 ] koert kuipers commented on SPARK-15810: --- when i code Aggregator up generically as a

[jira] [Updated] (SPARK-18534) Datasets Aggregation with Maps

2016-12-04 Thread Anton Okolnychyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-18534: - Description: There is a problem with user-defined aggregations in the Dataset API in Spar

[jira] [Updated] (SPARK-18534) Datasets Aggregation with Maps

2016-12-04 Thread Anton Okolnychyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-18534: - Description: There is a problem with user-defined aggregations in the Dataset API in Spar

[jira] [Assigned] (SPARK-16609) Single function for parsing timestamps/dates

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16609: Assignee: (was: Apache Spark) > Single function for parsing timestamps/dates > ---

[jira] [Assigned] (SPARK-16609) Single function for parsing timestamps/dates

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16609: Assignee: Apache Spark > Single function for parsing timestamps/dates > --

[jira] [Commented] (SPARK-16609) Single function for parsing timestamps/dates

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720535#comment-15720535 ] Apache Spark commented on SPARK-16609: -- User 'anabranch' has created a pull request

[jira] [Updated] (SPARK-18534) Datasets Aggregation with Maps

2016-12-04 Thread Anton Okolnychyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-18534: - Description: There is a problem with user-defined aggregations in the Dataset API in Spar

[jira] [Commented] (SPARK-18534) Datasets Aggregation with Maps

2016-12-04 Thread Anton Okolnychyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720522#comment-15720522 ] Anton Okolnychyi commented on SPARK-18534: -- [~yhuai], it seems that you should b

[jira] [Updated] (SPARK-18534) Datasets Aggregation with Maps

2016-12-04 Thread Anton Okolnychyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-18534: - Affects Version/s: 1.6.2 > Datasets Aggregation with Maps > -

[jira] [Commented] (SPARK-18534) Datasets Aggregation with Maps

2016-12-04 Thread Anton Okolnychyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720504#comment-15720504 ] Anton Okolnychyi commented on SPARK-18534: -- I have done a small investigation to

[jira] [Updated] (SPARK-18701) Poisson GLM fails due to wrong initialization

2016-12-04 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wayne Zhang updated SPARK-18701: Shepherd: Sean Owen (was: sean corkum) Issue Type: Bug (was: New Feature) > Poisson GLM fai

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720404#comment-15720404 ] koert kuipers commented on SPARK-15810: --- similar test: {noformat} val agg1 = new

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720397#comment-15720397 ] koert kuipers commented on SPARK-15810: --- to continue the previous example: wrapping

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720387#comment-15720387 ] koert kuipers commented on SPARK-15810: --- next, again with spark 2.1.x branch i want

[jira] [Commented] (SPARK-18708) Improve documentation in SparkContext.scala file

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720383#comment-15720383 ] Apache Spark commented on SPARK-18708: -- User 'Mironor' has created a pull request fo

[jira] [Assigned] (SPARK-18708) Improve documentation in SparkContext.scala file

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18708: Assignee: (was: Apache Spark) > Improve documentation in SparkContext.scala file > ---

[jira] [Assigned] (SPARK-18708) Improve documentation in SparkContext.scala file

2016-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18708: Assignee: Apache Spark > Improve documentation in SparkContext.scala file > --

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-12-04 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720377#comment-15720377 ] koert kuipers commented on SPARK-15810: --- i am redoing tests with options and aggreg

[jira] [Created] (SPARK-18708) Improve documentation in SparkContext.scala file

2016-12-04 Thread Bedrytski Aliaksandr (JIRA)
Bedrytski Aliaksandr created SPARK-18708: Summary: Improve documentation in SparkContext.scala file Key: SPARK-18708 URL: https://issues.apache.org/jira/browse/SPARK-18708 Project: Spark

[jira] [Commented] (SPARK-18682) Batch Source for Kafka

2016-12-04 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720146#comment-15720146 ] Cody Koeninger commented on SPARK-18682: Isn't this a duplicate of https://issues

[jira] [Issue Comment Deleted] (SPARK-18682) Batch Source for Kafka

2016-12-04 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger updated SPARK-18682: --- Comment: was deleted (was: Isn't this a duplicate of https://issues.apache.org/jira/browse/S

[jira] [Commented] (SPARK-18682) Batch Source for Kafka

2016-12-04 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720145#comment-15720145 ] Cody Koeninger commented on SPARK-18682: Isn't this a duplicate of https://issues

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720128#comment-15720128 ] Hyukjin Kwon commented on SPARK-18699: -- Thank you for cc'ing me. Yup, I noticed this

[jira] [Closed] (SPARK-18706) Can spark support exactly once based kafka ? Due to these following question.

2016-12-04 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj closed SPARK-18706. --- Resolution: Duplicate > Can spark support exactly once based kafka ? Due to these following question. >

[jira] [Commented] (SPARK-18706) Can spark support exactly once based kafka ? Due to these following question.

2016-12-04 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720087#comment-15720087 ] hustfxj commented on SPARK-18706: - 1. If a task complete the operation, it will notify dr

[jira] [Created] (SPARK-18706) Can spark support exactly once based kafka ? Due to these following question.

2016-12-04 Thread hustfxj (JIRA)
hustfxj created SPARK-18706: --- Summary: Can spark support exactly once based kafka ? Due to these following question. Key: SPARK-18706 URL: https://issues.apache.org/jira/browse/SPARK-18706 Project: Spark

[jira] [Created] (SPARK-18707) Can spark support exactly once based kafka ? Due to these following question?

2016-12-04 Thread hustfxj (JIRA)
hustfxj created SPARK-18707: --- Summary: Can spark support exactly once based kafka ? Due to these following question? Key: SPARK-18707 URL: https://issues.apache.org/jira/browse/SPARK-18707 Project: Spark

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720069#comment-15720069 ] Takeshi Yamamuro commented on SPARK-18699: -- Anyway, we can easily fix this like

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720033#comment-15720033 ] Takeshi Yamamuro commented on SPARK-18699: -- Additionally, in our basic stance, i

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15719993#comment-15719993 ] Takeshi Yamamuro commented on SPARK-18699: -- `DROPMALFORMED` works well in this q

[jira] [Resolved] (SPARK-18661) Creating a partitioned datasource table should not scan all files for table

2016-12-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18661. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0 > Creating a partitio

[jira] [Commented] (SPARK-18322) ML, Graph 2.1 QA: Update user guide for new features & APIs

2016-12-04 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15719982#comment-15719982 ] Yanbo Liang commented on SPARK-18322: - I have checked major changes during this relea

[jira] [Commented] (SPARK-18705) Docs for one-pass solver for linear regression with L1 and elastic-net penalties

2016-12-04 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15719964#comment-15719964 ] Yanbo Liang commented on SPARK-18705: - [~sethah] Do you have time to work on this? Th

[jira] [Created] (SPARK-18705) Docs for one-pass solver for linear regression with L1 and elastic-net penalties

2016-12-04 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-18705: --- Summary: Docs for one-pass solver for linear regression with L1 and elastic-net penalties Key: SPARK-18705 URL: https://issues.apache.org/jira/browse/SPARK-18705 Projec

  1   2   >