[jira] [Commented] (SPARK-10659) DataFrames and SparkSQL saveAsParquetFile does not preserve REQUIRED (not nullable) flag in schema

[ https://issues.apache.org/jira/browse/SPARK-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14805072#comment-14805072 ] Vladimir Picka commented on SPARK-10659: Thanks so much. I will put it to test on our use case.

[jira] [Updated] (SPARK-10681) DateTimeUtils needs a method to parse string to SQL's timestamp value

[ https://issues.apache.org/jira/browse/SPARK-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10681: -- Issue Type: Improvement (was: Bug) Dumb question, but isn't this just {{...getTime * 1000}}? does it

[jira] [Commented] (SPARK-10677) UnsafeExternalSorter should atomically release and acquire

[ https://issues.apache.org/jira/browse/SPARK-10677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876138#comment-14876138 ] Andrew Or commented on SPARK-10677: --- Closing as Won't Fix because the ShuffleMemoryManager actually

[jira] [Resolved] (SPARK-10677) UnsafeExternalSorter should atomically release and acquire

[ https://issues.apache.org/jira/browse/SPARK-10677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10677. --- Resolution: Won't Fix > UnsafeExternalSorter should atomically release and acquire >

[jira] [Updated] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10611: --- Fix Version/s: 1.5.1 > Configuration object thread safety issue in NewHadoopRDD >

[jira] [Resolved] (SPARK-4280) In dynamic allocation, add option to never kill executors with cached blocks

[ https://issues.apache.org/jira/browse/SPARK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-4280. --- Resolution: Duplicate Unless I misunderstand something, this is the same as SPARK-7955; just

[jira] [Updated] (SPARK-10449) StructType.merge shouldn't merge DecimalTypes with different precisions and/or scales

[ https://issues.apache.org/jira/browse/SPARK-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10449: --- Assignee: holdenk > StructType.merge shouldn't merge DecimalTypes with different precisions >

[jira] [Commented] (SPARK-10709) When loading a json dataset as a data frame, if the input path is wrong, the error message is very confusing

[ https://issues.apache.org/jira/browse/SPARK-10709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876767#comment-14876767 ] Kai Sasaki commented on SPARK-10709: [~yhuai] So do you mean this error message should clarify the

[jira] [Created] (SPARK-10702) Dynamic Allocation in Standalone Breaking Parallelism

Mark Khaitman created SPARK-10702: - Summary: Dynamic Allocation in Standalone Breaking Parallelism Key: SPARK-10702 URL: https://issues.apache.org/jira/browse/SPARK-10702 Project: Spark

[jira] [Comment Edited] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876198#comment-14876198 ] Jerry Lam edited comment on SPARK-8118 at 9/18/15 7:30 PM: --- I'm trying to turn

[jira] [Commented] (SPARK-10382) Make example code in user guide testable

[ https://issues.apache.org/jira/browse/SPARK-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876706#comment-14876706 ] Xiangrui Meng commented on SPARK-10382: --- [~holdenk] If you are interested, could you try a quick

[jira] [Commented] (SPARK-10668) Use WeightedLeastSquares in LinearRegression with L2 regularization if the number of features is small

[ https://issues.apache.org/jira/browse/SPARK-10668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876787#comment-14876787 ] Kai Sasaki commented on SPARK-10668: [~mengxr] Hello, can I work on this JIRA? Please assign it to

[jira] [Commented] (SPARK-10709) When loading a json dataset as a data frame, if the input path is wrong, the error message is very confusing

[ https://issues.apache.org/jira/browse/SPARK-10709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876806#comment-14876806 ] Yin Huai commented on SPARK-10709: -- [~lewuathe] Yeah, if the path does not exist, it will be great to

[jira] [Commented] (SPARK-10656) select(df(*)) fails when a column has special characters

[ https://issues.apache.org/jira/browse/SPARK-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14805123#comment-14805123 ] Apache Spark commented on SPARK-10656: -- User 'zhichao-li' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10656) select(df(*)) fails when a column has special characters

[ https://issues.apache.org/jira/browse/SPARK-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10656: Assignee: Apache Spark > select(df(*)) fails when a column has special characters >

[jira] [Updated] (SPARK-10691) Make LogisticRegressionModel's evaluate method public

[ https://issues.apache.org/jira/browse/SPARK-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-10691: Description: The following method in {{LogisticRegressionModel}} is marked as {{private}}, which prevents

[jira] [Commented] (SPARK-10262) Add @Since annotation to ml.attribute

[ https://issues.apache.org/jira/browse/SPARK-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14805079#comment-14805079 ] Tijo Thomas commented on SPARK-10262: - I am sorry for the delay . There are many file and i am almost

[jira] [Created] (SPARK-10691) Make LogisticRegressionModel's evaluate method public

Hao Ren created SPARK-10691: --- Summary: Make LogisticRegressionModel's evaluate method public Key: SPARK-10691 URL: https://issues.apache.org/jira/browse/SPARK-10691 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-10656) select(df(*)) fails when a column has special characters

[ https://issues.apache.org/jira/browse/SPARK-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10656: Assignee: (was: Apache Spark) > select(df(*)) fails when a column has special

[jira] [Commented] (SPARK-7770) Should GBT validationTol be relative tolerance?

[ https://issues.apache.org/jira/browse/SPARK-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14805343#comment-14805343 ] Yanbo Liang commented on SPARK-7770: [~josephkb] After investigate of convergenceTol in

[jira] [Resolved] (SPARK-10684) StructType.interpretedOrdering need not to be serialized

[ https://issues.apache.org/jira/browse/SPARK-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10684. - Resolution: Fixed Assignee: Navis Fix Version/s: 1.6.0 >

[jira] [Commented] (SPARK-9808) Remove hash shuffle file consolidation

[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14805144#comment-14805144 ] Apache Spark commented on SPARK-9808: - User 'rxin' has created a pull request for this issue:

[jira] [Updated] (SPARK-10691) Make LogisticRegressionModel's evaluate method public

[ https://issues.apache.org/jira/browse/SPARK-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-10691: Description: The following method in {{LogisticRegressionModel}} is marked as {{private}}, which prevents

[jira] [Commented] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876066#comment-14876066 ] Dan Brown commented on SPARK-10685: --- Ok. Aside from python UDFs, the zip after repartition behavior is

[jira] [Assigned] (SPARK-10701) Expose SparkContext#stopped flag with @DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10701: Assignee: (was: Apache Spark) > Expose SparkContext#stopped flag with @DeveloperApi >

[jira] [Updated] (SPARK-10684) StructType.interpretedOrdering need not to be serialized

[ https://issues.apache.org/jira/browse/SPARK-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10684: Fix Version/s: 1.5.1 > StructType.interpretedOrdering need not to be serialized >

[jira] [Commented] (SPARK-10704) Consolidate HashShuffleReader and ShuffleReader and refactor ShuffleManager.getReader()

[ https://issues.apache.org/jira/browse/SPARK-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876157#comment-14876157 ] Apache Spark commented on SPARK-10704: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-10706) Add java wrapper for random vector rdd

holdenk created SPARK-10706: --- Summary: Add java wrapper for random vector rdd Key: SPARK-10706 URL: https://issues.apache.org/jira/browse/SPARK-10706 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-10449) StructType.merge shouldn't merge DecimalTypes with different precisions and/or scales

[ https://issues.apache.org/jira/browse/SPARK-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10449. Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull

[jira] [Resolved] (SPARK-9808) Remove hash shuffle file consolidation

[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-9808. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8812

[jira] [Updated] (SPARK-10671) Calling a UDF with insufficient number of input arguments should throw an analysis error

[ https://issues.apache.org/jira/browse/SPARK-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10671: - Priority: Major (was: Blocker) > Calling a UDF with insufficient number of input arguments should throw

[jira] [Updated] (SPARK-10671) Calling a UDF with insufficient number of input arguments should throw an analysis error

[ https://issues.apache.org/jira/browse/SPARK-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10671: - Target Version/s: 1.6.0 (was: 1.5.1) > Calling a UDF with insufficient number of input arguments should

[jira] [Created] (SPARK-10703) Physical filter operators should replace the general AND/OR/equality/etc with a special version that treats null as false

Mingyu Kim created SPARK-10703: -- Summary: Physical filter operators should replace the general AND/OR/equality/etc with a special version that treats null as false Key: SPARK-10703 URL:

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876307#comment-14876307 ] Apache Spark commented on SPARK-10474: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Commented] (SPARK-10672) We should not fail to create a table If we cannot persist metadata of a data source table to metastore in a Hive compatible way

[ https://issues.apache.org/jira/browse/SPARK-10672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876085#comment-14876085 ] Apache Spark commented on SPARK-10672: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10704) Consolidate HashShuffleReader and ShuffleReader and refactor ShuffleManager.getReader()

[ https://issues.apache.org/jira/browse/SPARK-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10704: Assignee: Apache Spark (was: Josh Rosen) > Consolidate HashShuffleReader and

[jira] [Commented] (SPARK-10681) DateTimeUtils needs a method to parse string to SQL's timestamp value

[ https://issues.apache.org/jira/browse/SPARK-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876158#comment-14876158 ] Yin Huai commented on SPARK-10681: -- [~srowen] Yeah, originally that's also my thought. However, for our

[jira] [Resolved] (SPARK-10540) HadoopFsRelationTest's "test all data types" is flaky

[ https://issues.apache.org/jira/browse/SPARK-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10540. -- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876198#comment-14876198 ] Jerry Lam commented on SPARK-8118: -- I'm trying to turn off parquet logging by adding these lines into the

[jira] [Commented] (SPARK-10538) java.lang.NegativeArraySizeException during join

[ https://issues.apache.org/jira/browse/SPARK-10538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876338#comment-14876338 ] Davies Liu commented on SPARK-10538: Do you know which operator (join or other) the exception came

[jira] [Assigned] (SPARK-10538) java.lang.NegativeArraySizeException during join

[ https://issues.apache.org/jira/browse/SPARK-10538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-10538: -- Assignee: Davies Liu > java.lang.NegativeArraySizeException during join >

[jira] [Assigned] (SPARK-10701) Expose SparkContext#stopped flag with @DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10701: Assignee: Apache Spark > Expose SparkContext#stopped flag with @DeveloperApi >

[jira] [Commented] (SPARK-10681) DateTimeUtils needs a method to parse string to SQL's timestamp value

[ https://issues.apache.org/jira/browse/SPARK-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876170#comment-14876170 ] Sean Owen commented on SPARK-10681: --- Oh right, I understand. This is very related and just merged:

[jira] [Created] (SPARK-10705) Stop converting internal rows to external rows in DataFrame.toJSON

Cheng Lian created SPARK-10705: -- Summary: Stop converting internal rows to external rows in DataFrame.toJSON Key: SPARK-10705 URL: https://issues.apache.org/jira/browse/SPARK-10705 Project: Spark

[jira] [Commented] (SPARK-9850) Adaptive execution in Spark

[ https://issues.apache.org/jira/browse/SPARK-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876347#comment-14876347 ] Imran Rashid commented on SPARK-9850: - just to continue brainstorming on what to do with large data --

[jira] [Commented] (SPARK-10539) Intersection Optimization is Wrong

[ https://issues.apache.org/jira/browse/SPARK-10539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876079#comment-14876079 ] Apache Spark commented on SPARK-10539: -- User 'yhuai' has created a pull request for this issue:

[jira] [Commented] (SPARK-10681) DateTimeUtils needs a method to parse string to SQL's timestamp value

[ https://issues.apache.org/jira/browse/SPARK-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876155#comment-14876155 ] Cheng Lian commented on SPARK-10681: {{DateTimeUtils}} is also used to read timestamps, which may

[jira] [Created] (SPARK-10704) Consolidate HashShuffleReader and ShuffleReader and refactor ShuffleManager.getReader()

Josh Rosen created SPARK-10704: -- Summary: Consolidate HashShuffleReader and ShuffleReader and refactor ShuffleManager.getReader() Key: SPARK-10704 URL: https://issues.apache.org/jira/browse/SPARK-10704

[jira] [Commented] (SPARK-10690) SQL select count(distinct ) won't work for a normal load

[ https://issues.apache.org/jira/browse/SPARK-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876215#comment-14876215 ] Yin Huai commented on SPARK-10690: -- This problem should be fixed in 1.5. Can you try it? > SQL select

[jira] [Resolved] (SPARK-10539) Intersection Optimization is Wrong

[ https://issues.apache.org/jira/browse/SPARK-10539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10539. -- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

[jira] [Commented] (SPARK-9808) Remove hash shuffle file consolidation

[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876363#comment-14876363 ] Josh Rosen commented on SPARK-9808: --- I think that we should remove this for now as part of some general

[jira] [Resolved] (SPARK-2793) Correctly lock directory creation in DiskBlockManager.getFile

[ https://issues.apache.org/jira/browse/SPARK-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2793. --- Resolution: Won't Fix Closing this as "Won't Fix" given that we're removing hash shuffle file

[jira] [Updated] (SPARK-9808) Remove hash shuffle file consolidation

[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-9808: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-7271 > Remove hash shuffle file

[jira] [Resolved] (SPARK-10623) NoSuchElementException thrown when ORC predicate push-down is turned on

[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10623. -- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

[jira] [Updated] (SPARK-9935) EqualNullSafe not processed in OrcRelation

[ https://issues.apache.org/jira/browse/SPARK-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-9935: Fix Version/s: 1.5.1 > EqualNullSafe not processed in OrcRelation >

[jira] [Commented] (SPARK-9935) EqualNullSafe not processed in OrcRelation

[ https://issues.apache.org/jira/browse/SPARK-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876817#comment-14876817 ] Yin Huai commented on SPARK-9935: - The 1.5 branch is fixed by the the PR of SPARK-10623,

[jira] [Updated] (SPARK-10641) skewness and kurtosis support

[ https://issues.apache.org/jira/browse/SPARK-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10641: -- Assignee: Seth Hendrickson > skewness and kurtosis support >

[jira] [Updated] (SPARK-10711) Do not assume spark.submit.deployMode is always set

[ https://issues.apache.org/jira/browse/SPARK-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-10711: --- Affects Version/s: (was: 1.6.0) 1.5.0 Fix Version/s: 1.5.1

[jira] [Created] (SPARK-10712) JVM crashes with spark.sql.tungsten.enabled = true

Mauro Pirrone created SPARK-10712: - Summary: JVM crashes with spark.sql.tungsten.enabled = true Key: SPARK-10712 URL: https://issues.apache.org/jira/browse/SPARK-10712 Project: Spark Issue

[jira] [Updated] (SPARK-10712) JVM crashes with spark.sql.tungsten.enabled = true

[ https://issues.apache.org/jira/browse/SPARK-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mauro Pirrone updated SPARK-10712: -- Priority: Critical (was: Blocker) > JVM crashes with spark.sql.tungsten.enabled = true >

[jira] [Commented] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876903#comment-14876903 ] Apache Spark commented on SPARK-10685: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10685: Assignee: (was: Apache Spark) > Misaligned data with RDD.zip and DataFrame.withColumn

[jira] [Commented] (SPARK-8632) Poor Python UDF performance because of RDD caching

[ https://issues.apache.org/jira/browse/SPARK-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876904#comment-14876904 ] Apache Spark commented on SPARK-8632: - User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10685: Assignee: Apache Spark > Misaligned data with RDD.zip and DataFrame.withColumn after

[jira] [Created] (SPARK-10713) SPARK_DIST_CLASSPATH ignored on Mesos executors

Dara Adib created SPARK-10713: - Summary: SPARK_DIST_CLASSPATH ignored on Mesos executors Key: SPARK-10713 URL: https://issues.apache.org/jira/browse/SPARK-10713 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876387#comment-14876387 ] Apache Spark commented on SPARK-10640: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Updated] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-10640: -- Assignee: Andrew Or (was: Thomas Graves) > Spark history server fails to parse

[jira] [Assigned] (SPARK-10710) Remove ability to set spark.shuffle.spill=false

[ https://issues.apache.org/jira/browse/SPARK-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-10710: -- Assignee: Josh Rosen > Remove ability to set spark.shuffle.spill=false >

[jira] [Updated] (SPARK-10155) Memory leak in SQL parsers

[ https://issues.apache.org/jira/browse/SPARK-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10155: - Target Version/s: 1.6.0, 1.5.1 (was: 1.6.0) > Memory leak in SQL parsers > -- >

[jira] [Assigned] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10640: Assignee: Apache Spark (was: Thomas Graves) > Spark history server fails to parse

[jira] [Created] (SPARK-10709) When loading a json dataset as a data frame, if the input path is wrong, the error message is very confusing

Yin Huai created SPARK-10709: Summary: When loading a json dataset as a data frame, if the input path is wrong, the error message is very confusing Key: SPARK-10709 URL:

[jira] [Updated] (SPARK-10710) Remove ability to set spark.shuffle.spill=false and spark.sql.externalSort=false

[ https://issues.apache.org/jira/browse/SPARK-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10710: --- Description: The {{spark.shuffle.spill=false}} configuration doesn't make much sense nowadays: I

[jira] [Commented] (SPARK-9595) Adding API to SparkConf for kryo serializers registration

[ https://issues.apache.org/jira/browse/SPARK-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876624#comment-14876624 ] holdenk commented on SPARK-9595: Cool, I've started doing a little bit on this, I think its probably going

[jira] [Commented] (SPARK-9595) Adding API to SparkConf for kryo serializers registration

[ https://issues.apache.org/jira/browse/SPARK-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876625#comment-14876625 ] holdenk commented on SPARK-9595: Cool, I've started doing a little bit on this, I think its probably going

[jira] [Resolved] (SPARK-2532) Fix issues with consolidated shuffle

[ https://issues.apache.org/jira/browse/SPARK-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2532. --- Resolution: Won't Fix Closing this as "Won't Fix" given that we're removing hash shuffle file

[jira] [Resolved] (SPARK-2795) Improve DiskBlockObjectWriter API

[ https://issues.apache.org/jira/browse/SPARK-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2795. --- Resolution: Won't Fix Closing this as "Won't Fix" given that we're removing hash shuffle file

[jira] [Closed] (SPARK-10559) DataFrame schema ArrayType should accept ResultIterable

[ https://issues.apache.org/jira/browse/SPARK-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-10559. -- Resolution: Won't Fix > DataFrame schema ArrayType should accept ResultIterable >

[jira] [Assigned] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10640: Assignee: Thomas Graves (was: Apache Spark) > Spark history server fails to parse

[jira] [Commented] (SPARK-9681) Support R feature interactions in RFormula

[ https://issues.apache.org/jira/browse/SPARK-9681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876504#comment-14876504 ] Apache Spark commented on SPARK-9681: - User 'ericl' has created a pull request for this issue:

[jira] [Updated] (SPARK-10710) Remove ability to set spark.shuffle.spill=false and spark.sql.externalSort=false

[ https://issues.apache.org/jira/browse/SPARK-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10710: --- Summary: Remove ability to set spark.shuffle.spill=false and spark.sql.externalSort=false (was:

[jira] [Updated] (SPARK-10710) Remove ability to set spark.shuffle.spill=false and spark.sql.planner.externalSort=false

[ https://issues.apache.org/jira/browse/SPARK-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10710: --- Summary: Remove ability to set spark.shuffle.spill=false and spark.sql.planner.externalSort=false

[jira] [Assigned] (SPARK-10707) Set operation output columns may have incorrect nullability

[ https://issues.apache.org/jira/browse/SPARK-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamstra reassigned SPARK-10707: Assignee: Mark Hamstra > Set operation output columns may have incorrect nullability >

[jira] [Created] (SPARK-10711) Do not assume spark.submit.deployMode is always set

Hossein Falaki created SPARK-10711: -- Summary: Do not assume spark.submit.deployMode is always set Key: SPARK-10711 URL: https://issues.apache.org/jira/browse/SPARK-10711 Project: Spark

[jira] [Resolved] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-10611. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8763

[jira] [Updated] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10611: --- Assignee: Mingyu Kim > Configuration object thread safety issue in NewHadoopRDD >

[jira] [Commented] (SPARK-10602) Univariate statistics as UDAFs: single-pass continuous stats

[ https://issues.apache.org/jira/browse/SPARK-10602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876644#comment-14876644 ] Seth Hendrickson commented on SPARK-10602: -- Hey [~josephkb], right now I only have bandwidth for

[jira] [Assigned] (SPARK-10711) Do not assume spark.submit.deployMode is always set

[ https://issues.apache.org/jira/browse/SPARK-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10711: Assignee: Apache Spark > Do not assume spark.submit.deployMode is always set >

[jira] [Assigned] (SPARK-10711) Do not assume spark.submit.deployMode is always set

[ https://issues.apache.org/jira/browse/SPARK-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10711: Assignee: (was: Apache Spark) > Do not assume spark.submit.deployMode is always set >

[jira] [Commented] (SPARK-10711) Do not assume spark.submit.deployMode is always set

[ https://issues.apache.org/jira/browse/SPARK-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876645#comment-14876645 ] Apache Spark commented on SPARK-10711: -- User 'falaki' has created a pull request for this issue:

[jira] [Created] (SPARK-10707) Set operation output columns may have incorrect nullability

Mikhail Bautin created SPARK-10707: -- Summary: Set operation output columns may have incorrect nullability Key: SPARK-10707 URL: https://issues.apache.org/jira/browse/SPARK-10707 Project: Spark

[jira] [Commented] (SPARK-10559) DataFrame schema ArrayType should accept ResultIterable

[ https://issues.apache.org/jira/browse/SPARK-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876377#comment-14876377 ] Davies Liu commented on SPARK-10559: It's easy to turn the ResultIterable into a list, so I'd like to

[jira] [Resolved] (SPARK-2794) Use Java 7 isSymlink when available

[ https://issues.apache.org/jira/browse/SPARK-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2794. --- Resolution: Won't Fix Closing this as "Won't Fix" given that we're removing hash shuffle file

[jira] [Created] (SPARK-10708) Consolidate SortShuffleManager and UnsafeShuffleManager

Josh Rosen created SPARK-10708: -- Summary: Consolidate SortShuffleManager and UnsafeShuffleManager Key: SPARK-10708 URL: https://issues.apache.org/jira/browse/SPARK-10708 Project: Spark Issue

[jira] [Commented] (SPARK-10708) Consolidate SortShuffleManager and UnsafeShuffleManager

[ https://issues.apache.org/jira/browse/SPARK-10708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876406#comment-14876406 ] Apache Spark commented on SPARK-10708: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-10710) Remove ability to set spark.shuffle.spill=false

Josh Rosen created SPARK-10710: -- Summary: Remove ability to set spark.shuffle.spill=false Key: SPARK-10710 URL: https://issues.apache.org/jira/browse/SPARK-10710 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10710) Remove ability to set spark.shuffle.spill=false and spark.sql.planner.externalSort=false

[ https://issues.apache.org/jira/browse/SPARK-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876586#comment-14876586 ] Apache Spark commented on SPARK-10710: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-10670) Link to each language's API in codetabs in ML docs: spark.ml

[ https://issues.apache.org/jira/browse/SPARK-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876590#comment-14876590 ] Joseph K. Bradley commented on SPARK-10670: --- Sure, thank you! > Link to each language's API in

[jira] [Commented] (SPARK-7770) Should GBT validationTol be relative tolerance?

[ https://issues.apache.org/jira/browse/SPARK-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876587#comment-14876587 ] Joseph K. Bradley commented on SPARK-7770: -- I think it's OK to change the behavior, though we'll

[jira] [Assigned] (SPARK-10615) changes assertEquals to assertEqual for existing unit tests

[ https://issues.apache.org/jira/browse/SPARK-10615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10615: Assignee: Apache Spark > changes assertEquals to assertEqual for existing unit tests >

[jira] [Commented] (SPARK-10615) changes assertEquals to assertEqual for existing unit tests

[ https://issues.apache.org/jira/browse/SPARK-10615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14808974#comment-14808974 ] Apache Spark commented on SPARK-10615: -- User 'yanboliang' has created a pull request for this issue:

  1   2   >