[jira] [Created] (SPARK-22303) [SQL] Getting java.sql.SQLException: Unsupported type 101 for BINARY_DOUBLE

2017-10-17 Thread Kohki Nishio (JIRA)
Kohki Nishio created SPARK-22303: Summary: [SQL] Getting java.sql.SQLException: Unsupported type 101 for BINARY_DOUBLE Key: SPARK-22303 URL: https://issues.apache.org/jira/browse/SPARK-22303 Project:

[jira] [Resolved] (SPARK-22278) Expose current event time watermark and current processing time in GroupState

2017-10-17 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-22278. --- Resolution: Fixed Fix Version/s: (was: 2.2.0) 3.0.0 Issue

[jira] [Commented] (SPARK-22302) Remove manual backports for subprocess.check_output and check_call

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208722#comment-16208722 ] Apache Spark commented on SPARK-22302: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-22302) Remove manual backports for subprocess.check_output and check_call

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22302: Assignee: (was: Apache Spark) > Remove manual backports for subprocess.check_output

[jira] [Assigned] (SPARK-22302) Remove manual backports for subprocess.check_output and check_call

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22302: Assignee: Apache Spark > Remove manual backports for subprocess.check_output and

[jira] [Created] (SPARK-22302) Remove manual backports for subprocess.check_output and check_call

2017-10-17 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-22302: Summary: Remove manual backports for subprocess.check_output and check_call Key: SPARK-22302 URL: https://issues.apache.org/jira/browse/SPARK-22302 Project: Spark

[jira] [Commented] (SPARK-22295) Chi Square selector not recognizing field in Data frame

2017-10-17 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208708#comment-16208708 ] Peng Meng commented on SPARK-22295: --- Hi [~cheburakshu] ， thanks for reporting this bug and helpful

[jira] [Commented] (SPARK-17902) collect() ignores stringsAsFactors

2017-10-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208685#comment-16208685 ] Hyukjin Kwon commented on SPARK-17902: -- Hi [~falaki] and [~shivaram], I was thinking a just simple

[jira] [Comment Edited] (SPARK-17902) collect() ignores stringsAsFactors

2017-10-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208685#comment-16208685 ] Hyukjin Kwon edited comment on SPARK-17902 at 10/18/17 1:29 AM: Hi

[jira] [Commented] (SPARK-22204) Explain output for SQL with commands shows no optimization

2017-10-17 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208676#comment-16208676 ] Andrew Ash commented on SPARK-22204: One way to work around this issue could be by getting the child

[jira] [Commented] (SPARK-22301) Add rule to Optimizer for In with empty list of values

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208640#comment-16208640 ] Apache Spark commented on SPARK-22301: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22301) Add rule to Optimizer for In with empty list of values

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22301: Assignee: (was: Apache Spark) > Add rule to Optimizer for In with empty list of

[jira] [Assigned] (SPARK-22301) Add rule to Optimizer for In with empty list of values

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22301: Assignee: Apache Spark > Add rule to Optimizer for In with empty list of values >

[jira] [Created] (SPARK-22301) Add rule to Optimizer for In with empty list of values

2017-10-17 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22301: --- Summary: Add rule to Optimizer for In with empty list of values Key: SPARK-22301 URL: https://issues.apache.org/jira/browse/SPARK-22301 Project: Spark Issue

[jira] [Commented] (SPARK-22289) Cannot save LogisticRegressionClassificationModel with bounds on coefficients

2017-10-17 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208614#comment-16208614 ] yuhao yang commented on SPARK-22289: Thanks for the reply. I'll start compose a PR. > Cannot save

[jira] [Commented] (SPARK-13030) Change OneHotEncoder to Estimator

2017-10-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208607#comment-16208607 ] Joseph K. Bradley commented on SPARK-13030: --- Does multi-column support need to be put in this

[jira] [Commented] (SPARK-22289) Cannot save LogisticRegressionClassificationModel with bounds on coefficients

2017-10-17 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208601#comment-16208601 ] Yanbo Liang commented on SPARK-22289: - +1 for option 2. Please feel free to send a PR. Thanks. >

[jira] [Comment Edited] (SPARK-22283) withColumn should replace multiple instances with a single one

2017-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208585#comment-16208585 ] Liang-Chi Hsieh edited comment on SPARK-22283 at 10/17/17 11:43 PM:

[jira] [Commented] (SPARK-22283) withColumn should replace multiple instances with a single one

2017-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208585#comment-16208585 ] Liang-Chi Hsieh commented on SPARK-22283: - [~kitbellew] I didn't mean you're doing select. I

[jira] [Commented] (SPARK-22249) UnsupportedOperationException: empty.reduceLeft when caching a dataframe

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208462#comment-16208462 ] Apache Spark commented on SPARK-22249: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Resolved] (SPARK-22050) Allow BlockUpdated events to be optionally logged to the event log

2017-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22050. Resolution: Fixed Assignee: Michael Mior Fix Version/s: 2.3.0 > Allow

[jira] [Updated] (SPARK-22298) SparkUI executor URL encode appID

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22298: -- Issue Type: Improvement (was: Bug) > SparkUI executor URL encode appID >

[jira] [Resolved] (SPARK-22271) Describe results in "null" for the value of "mean" of a numeric variable

2017-10-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22271. - Resolution: Fixed Assignee: Huaxin Gao Fix Version/s: 2.3.0 2.2.1 >

[jira] [Updated] (SPARK-22296) CodeGenerator - failed to compile when constructor has scala.collection.mutable.Seq vs.

2017-10-17 Thread Randy Tidd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randy Tidd updated SPARK-22296: --- Summary: CodeGenerator - failed to compile when constructor has scala.collection.mutable.Seq vs.

[jira] [Assigned] (SPARK-22300) Update ORC to 1.4.1

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22300: Assignee: (was: Apache Spark) > Update ORC to 1.4.1 > --- > >

[jira] [Assigned] (SPARK-22300) Update ORC to 1.4.1

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22300: Assignee: Apache Spark > Update ORC to 1.4.1 > --- > >

[jira] [Commented] (SPARK-22300) Update ORC to 1.4.1

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208180#comment-16208180 ] Apache Spark commented on SPARK-22300: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-22296) CodeGenerator - failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

2017-10-17 Thread Randy Tidd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randy Tidd updated SPARK-22296: --- Description: This is with Scala 2.11. We have a case class that has a constructor with 85 args, the

[jira] [Updated] (SPARK-22296) CodeGenerator - failed to compile when constructor has scala.collection.mutable.Seq vs. scala.collection.Seq

2017-10-17 Thread Randy Tidd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randy Tidd updated SPARK-22296: --- Summary: CodeGenerator - failed to compile when constructor has scala.collection.mutable.Seq vs.

[jira] [Created] (SPARK-22300) Update ORC to 1.4.1

2017-10-17 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-22300: - Summary: Update ORC to 1.4.1 Key: SPARK-22300 URL: https://issues.apache.org/jira/browse/SPARK-22300 Project: Spark Issue Type: Bug Components:

[jira] [Assigned] (SPARK-21840) Allow multiple SparkSubmit invocations in same JVM without polluting system properties

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21840: Assignee: (was: Apache Spark) > Allow multiple SparkSubmit invocations in same JVM

[jira] [Commented] (SPARK-22298) SparkUI executor URL encode appID

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208172#comment-16208172 ] Apache Spark commented on SPARK-22298: -- User 'alexnaspo' has created a pull request for this issue:

[jira] [Commented] (SPARK-21840) Allow multiple SparkSubmit invocations in same JVM without polluting system properties

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208171#comment-16208171 ] Apache Spark commented on SPARK-21840: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21840) Allow multiple SparkSubmit invocations in same JVM without polluting system properties

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21840: Assignee: Apache Spark > Allow multiple SparkSubmit invocations in same JVM without

[jira] [Assigned] (SPARK-22298) SparkUI executor URL encode appID

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22298: Assignee: Apache Spark > SparkUI executor URL encode appID >

[jira] [Assigned] (SPARK-22298) SparkUI executor URL encode appID

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22298: Assignee: (was: Apache Spark) > SparkUI executor URL encode appID >

[jira] [Created] (SPARK-22299) Use OFFSET and LIMIT for JDBC DataFrameReader striping

2017-10-17 Thread Zack Behringer (JIRA)
Zack Behringer created SPARK-22299: -- Summary: Use OFFSET and LIMIT for JDBC DataFrameReader striping Key: SPARK-22299 URL: https://issues.apache.org/jira/browse/SPARK-22299 Project: Spark

[jira] [Created] (SPARK-22298) SparkUI executor URL encode appID

2017-10-17 Thread Alexander Naspo (JIRA)
Alexander Naspo created SPARK-22298: --- Summary: SparkUI executor URL encode appID Key: SPARK-22298 URL: https://issues.apache.org/jira/browse/SPARK-22298 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-22297) Flaky test: BlockManagerSuite "Shuffle registration timeout and maxAttempts conf"

2017-10-17 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-22297: -- Summary: Flaky test: BlockManagerSuite "Shuffle registration timeout and maxAttempts conf" Key: SPARK-22297 URL: https://issues.apache.org/jira/browse/SPARK-22297

[jira] [Created] (SPARK-22296) CodeGenerator - failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

2017-10-17 Thread Randy Tidd (JIRA)
Randy Tidd created SPARK-22296: -- Summary: CodeGenerator - failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java' Key: SPARK-22296 URL:

[jira] [Resolved] (SPARK-22295) Chi Square selector not recognizing field in Data frame

2017-10-17 Thread Cheburakshu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheburakshu resolved SPARK-22295. - Resolution: Invalid > Chi Square selector not recognizing field in Data frame >

[jira] [Updated] (SPARK-22295) Chi Square selector not recognizing field in Data frame

2017-10-17 Thread Cheburakshu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheburakshu updated SPARK-22295: Description: ChiSquare selector is not recognizing the field 'class' which is present in the data

[jira] [Updated] (SPARK-22295) Chi Square selector not recognizing field in Data frame

2017-10-17 Thread Cheburakshu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheburakshu updated SPARK-22295: Description: ChiSquare selector is not recognizing the field 'class' which is present in the data

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208078#comment-16208078 ] Apache Spark commented on SPARK-18016: -- User 'bdrillard' has created a pull request for this issue:

[jira] [Updated] (SPARK-22295) Chi Square selector not recognizing field in Data frame

2017-10-17 Thread Cheburakshu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheburakshu updated SPARK-22295: Description: ChiSquare selector is not recognizing the field 'class' which is present in the data

[jira] [Created] (SPARK-22295) Chi Square selector not recognizing field in Data frame

2017-10-17 Thread Cheburakshu (JIRA)
Cheburakshu created SPARK-22295: --- Summary: Chi Square selector not recognizing field in Data frame Key: SPARK-22295 URL: https://issues.apache.org/jira/browse/SPARK-22295 Project: Spark Issue

[jira] [Updated] (SPARK-22249) UnsupportedOperationException: empty.reduceLeft when caching a dataframe

2017-10-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22249: Component/s: (was: PySpark) SQL > UnsupportedOperationException: empty.reduceLeft

[jira] [Commented] (SPARK-21213) Support collecting partition-level statistics: rowCount and sizeInBytes

2017-10-17 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208027#comment-16208027 ] Ruslan Dautkhanov commented on SPARK-21213: --- Would the partition-level stats be compatible with

[jira] [Commented] (SPARK-22283) withColumn should replace multiple instances with a single one

2017-10-17 Thread Albert Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208020#comment-16208020 ] Albert Meltzer commented on SPARK-22283: [~cjm] thank you for finding the new implementation,

[jira] [Commented] (SPARK-22283) withColumn should replace multiple instances with a single one

2017-10-17 Thread Albert Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208011#comment-16208011 ] Albert Meltzer commented on SPARK-22283: [~viirya] I'm not doing select, I'm trying to replace

[jira] [Updated] (SPARK-21459) Some aggregation functions change the case of nested field names

2017-10-17 Thread David Allsopp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Allsopp updated SPARK-21459: -- Description: When working with DataFrames with nested schemas, the behavior of the

[jira] [Comment Edited] (SPARK-21459) Some aggregation functions change the case of nested field names

2017-10-17 Thread David Allsopp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207270#comment-16207270 ] David Allsopp edited comment on SPARK-21459 at 10/17/17 4:54 PM: - Just

[jira] [Updated] (SPARK-22181) ReplaceExceptWithFilter if one or both of the datasets are fully derived out of Filters from a same parent

2017-10-17 Thread Sathiya Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathiya Kumar updated SPARK-22181: -- Description: While applying Except operator between two datasets, if one or both of the

[jira] [Commented] (SPARK-22250) Be less restrictive on type checking

2017-10-17 Thread Fernando Pereira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207798#comment-16207798 ] Fernando Pereira commented on SPARK-22250: -- I did some tests and even though verifySchema=False

[jira] [Commented] (SPARK-22283) withColumn should replace multiple instances with a single one

2017-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207780#comment-16207780 ] Liang-Chi Hsieh commented on SPARK-22283: - When joined result has duplicate column name, you

[jira] [Assigned] (SPARK-22062) BlockManager does not account for memory consumed by remote fetches

2017-10-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22062: --- Assignee: Saisai Shao > BlockManager does not account for memory consumed by remote fetches

[jira] [Resolved] (SPARK-22062) BlockManager does not account for memory consumed by remote fetches

2017-10-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22062. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19476

[jira] [Comment Edited] (SPARK-19606) Support constraints in spark-dispatcher

2017-10-17 Thread Pascal GILLET (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207722#comment-16207722 ] Pascal GILLET edited comment on SPARK-19606 at 10/17/17 2:39 PM: - * _If

[jira] [Comment Edited] (SPARK-19606) Support constraints in spark-dispatcher

2017-10-17 Thread Pascal GILLET (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207722#comment-16207722 ] Pascal GILLET edited comment on SPARK-19606 at 10/17/17 2:38 PM: - * _If

[jira] [Commented] (SPARK-19606) Support constraints in spark-dispatcher

2017-10-17 Thread Pascal GILLET (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207722#comment-16207722 ] Pascal GILLET commented on SPARK-19606: --- * _If "spark.mesos.constraints" is passed with the job

[jira] [Commented] (SPARK-20396) groupBy().apply() with pandas udf in pyspark

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207711#comment-16207711 ] Apache Spark commented on SPARK-20396: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-22288) Tricky interaction between closure-serialization and inheritance results in confusing failure

2017-10-17 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207706#comment-16207706 ] Ryan Williams commented on SPARK-22288: --- Makes sense, fine with me to "Won't Fix". bq. You can

[jira] [Commented] (SPARK-22277) Chi Square selector garbling Vector content.

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207618#comment-16207618 ] Apache Spark commented on SPARK-22277: -- User 'mpjlu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22277) Chi Square selector garbling Vector content.

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22277: Assignee: (was: Apache Spark) > Chi Square selector garbling Vector content. >

[jira] [Assigned] (SPARK-22277) Chi Square selector garbling Vector content.

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22277: Assignee: Apache Spark > Chi Square selector garbling Vector content. >

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2017-10-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207573#comment-16207573 ] Steve Loughran commented on SPARK-2984: --- bq. multiple batches writing to same location

[jira] [Assigned] (SPARK-22287) SPARK_DAEMON_MEMORY not honored by MesosClusterDispatcher

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22287: Assignee: (was: Apache Spark) > SPARK_DAEMON_MEMORY not honored by

[jira] [Commented] (SPARK-22287) SPARK_DAEMON_MEMORY not honored by MesosClusterDispatcher

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207541#comment-16207541 ] Apache Spark commented on SPARK-22287: -- User 'pmackles' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22287) SPARK_DAEMON_MEMORY not honored by MesosClusterDispatcher

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22287: Assignee: Apache Spark > SPARK_DAEMON_MEMORY not honored by MesosClusterDispatcher >

[jira] [Commented] (SPARK-21551) pyspark's collect fails when getaddrinfo is too slow

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207540#comment-16207540 ] Apache Spark commented on SPARK-21551: -- User 'FRosner' has created a pull request for this issue:

[jira] [Commented] (SPARK-21551) pyspark's collect fails when getaddrinfo is too slow

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207539#comment-16207539 ] Apache Spark commented on SPARK-21551: -- User 'FRosner' has created a pull request for this issue:

[jira] [Commented] (SPARK-21551) pyspark's collect fails when getaddrinfo is too slow

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207538#comment-16207538 ] Apache Spark commented on SPARK-21551: -- User 'FRosner' has created a pull request for this issue:

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2017-10-17 Thread Soumitra Sulav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207519#comment-16207519 ] Soumitra Sulav commented on SPARK-2984: --- I'm facing the same issues with Spark 2.0.2 on DC/OS with

[jira] [Commented] (SPARK-22294) Reset spark.driver.bindAddress when starting a Checkpoint

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207509#comment-16207509 ] Apache Spark commented on SPARK-22294: -- User 'ssaavedra' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22294) Reset spark.driver.bindAddress when starting a Checkpoint

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22294: Assignee: (was: Apache Spark) > Reset spark.driver.bindAddress when starting a

[jira] [Assigned] (SPARK-22294) Reset spark.driver.bindAddress when starting a Checkpoint

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22294: Assignee: Apache Spark > Reset spark.driver.bindAddress when starting a Checkpoint >

[jira] [Created] (SPARK-22294) Reset spark.driver.bindAddress when starting a Checkpoint

2017-10-17 Thread Santiago Saavedra (JIRA)
Santiago Saavedra created SPARK-22294: - Summary: Reset spark.driver.bindAddress when starting a Checkpoint Key: SPARK-22294 URL: https://issues.apache.org/jira/browse/SPARK-22294 Project: Spark

[jira] [Commented] (SPARK-21459) Some aggregation functions change the case of nested field names

2017-10-17 Thread David Allsopp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207270#comment-16207270 ] David Allsopp commented on SPARK-21459: --- Just trying to see when this problem was resolved: * It is

[jira] [Assigned] (SPARK-22224) Override toString of KeyValueGroupedDataset & RelationalGroupedDataset

2017-10-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-4: --- Assignee: Kent Yao > Override toString of KeyValueGroupedDataset & RelationalGroupedDataset

[jira] [Resolved] (SPARK-22224) Override toString of KeyValueGroupedDataset & RelationalGroupedDataset

2017-10-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-4. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19363

[jira] [Commented] (SPARK-21551) pyspark's collect fails when getaddrinfo is too slow

2017-10-17 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207220#comment-16207220 ] Frank Rosner commented on SPARK-21551: -- Do you guys mind if I backport this also to 2.0.x, 2.1.x,

[jira] [Commented] (SPARK-18649) sc.textFile(my_file).collect() raises socket.timeout on large files

2017-10-17 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207217#comment-16207217 ] Frank Rosner commented on SPARK-18649: -- Looks like in SPARK-21551 they increased the hard coded

[jira] [Commented] (SPARK-16599) java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask(BlockInfoManager.scala:343)

2017-10-17 Thread Pranav Singhania (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207204#comment-16207204 ] Pranav Singhania commented on SPARK-16599: -- [~srowen] I have observed this happening with my

[jira] [Commented] (SPARK-22284) Code of class \"org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection\" grows beyond 64 KB

2017-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207183#comment-16207183 ] Liang-Chi Hsieh commented on SPARK-22284: - Btw, we have used {{UnsafeProjection}} in many places

[jira] [Commented] (SPARK-22277) Chi Square selector garbling Vector content.

2017-10-17 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207146#comment-16207146 ] Peng Meng commented on SPARK-22277: --- This seems is a bug. If no one is working on it. I can work on it.

[jira] [Commented] (SPARK-22289) Cannot save LogisticRegressionClassificationModel with bounds on coefficients

2017-10-17 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207133#comment-16207133 ] Nick Pentreath commented on SPARK-22289: I think option (2) is the more general fix here. >

[jira] [Resolved] (SPARK-22249) UnsupportedOperationException: empty.reduceLeft when caching a dataframe

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22249. --- Resolution: Fixed Resolved by https://github.com/apache/spark/pull/19494 >

[jira] [Assigned] (SPARK-22249) UnsupportedOperationException: empty.reduceLeft when caching a dataframe

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-22249: - Assignee: Marco Gaido Fix Version/s: 2.3.0 2.2.1 >

[jira] [Resolved] (SPARK-19317) UnsupportedOperationException: empty.reduceLeft in LinearSeqOptimized

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19317. --- Resolution: Duplicate > UnsupportedOperationException: empty.reduceLeft in LinearSeqOptimized >

[jira] [Assigned] (SPARK-20992) Link to Nomad scheduler backend in docs

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20992: - Assignee: Ben Barnard > Link to Nomad scheduler backend in docs >

[jira] [Resolved] (SPARK-20992) Link to Nomad scheduler backend in docs

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20992. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19354

[jira] [Commented] (SPARK-22288) Tricky interaction between closure-serialization and inheritance results in confusing failure

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207116#comment-16207116 ] Sean Owen commented on SPARK-22288: --- I think this is a Java serialization question, not Spark. Still

[jira] [Commented] (SPARK-22289) Cannot save LogisticRegressionClassificationModel with bounds on coefficients

2017-10-17 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207115#comment-16207115 ] yuhao yang commented on SPARK-22289: cc [~yanboliang] [~dbtsai] > Cannot save

[jira] [Resolved] (SPARK-21459) Some aggregation functions change the case of nested field names

2017-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21459. --- Resolution: Cannot Reproduce > Some aggregation functions change the case of nested field names >

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UTF from HDFS

2017-10-17 Thread Bang Xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207098#comment-16207098 ] Bang Xiao commented on SPARK-21697: --- in the case describe above, i added "spark jars : file:///xxx.jar"

[jira] [Resolved] (SPARK-22264) History server will be unavailable if there is an event log file with large size

2017-10-17 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang resolved SPARK-22264. -- Resolution: Duplicate > History server will be unavailable if there is an event log file with large >

[jira] [Assigned] (SPARK-22293) Avoid unnecessary traversal in ResolveReferences

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22293: Assignee: (was: Apache Spark) > Avoid unnecessary traversal in ResolveReferences >

[jira] [Assigned] (SPARK-22293) Avoid unnecessary traversal in ResolveReferences

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22293: Assignee: Apache Spark > Avoid unnecessary traversal in ResolveReferences >

[jira] [Commented] (SPARK-22293) Avoid unnecessary traversal in ResolveReferences

2017-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207093#comment-16207093 ] Apache Spark commented on SPARK-22293: -- User 'ConeyLiu' has created a pull request for this issue:

[jira] [Created] (SPARK-22293) Avoid unnecessary traversal in ResolveReferences

2017-10-17 Thread Xianyang Liu (JIRA)
Xianyang Liu created SPARK-22293: Summary: Avoid unnecessary traversal in ResolveReferences Key: SPARK-22293 URL: https://issues.apache.org/jira/browse/SPARK-22293 Project: Spark Issue Type:

  1   2   >