[jira] [Comment Edited] (SPARK-8072) Better AnalysisException for writing DataFrame with identically named columns

[ https://issues.apache.org/jira/browse/SPARK-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572714#comment-14572714 ] Animesh Baranawal edited comment on SPARK-8072 at 6/5/15 8:06 AM:

[jira] [Commented] (SPARK-8124) Created more examples on SparkR DataFrames

[ https://issues.apache.org/jira/browse/SPARK-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574162#comment-14574162 ] Apache Spark commented on SPARK-8124: - User 'Emaasit' has created a pull request for

[jira] [Assigned] (SPARK-8124) Created more examples on SparkR DataFrames

[ https://issues.apache.org/jira/browse/SPARK-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8124: --- Assignee: Apache Spark Created more examples on SparkR DataFrames

[jira] [Assigned] (SPARK-8124) Created more examples on SparkR DataFrames

[ https://issues.apache.org/jira/browse/SPARK-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8124: --- Assignee: (was: Apache Spark) Created more examples on SparkR DataFrames

[jira] [Assigned] (SPARK-8121) When using with Hadoop 1.x, spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8121: --- Assignee: Cheng Lian (was: Apache Spark) When using with Hadoop 1.x,

[jira] [Commented] (SPARK-8121) When using with Hadoop 1.x, spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574198#comment-14574198 ] Apache Spark commented on SPARK-8121: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-8121) When using with Hadoop 1.x, spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8121: --- Assignee: Apache Spark (was: Cheng Lian) When using with Hadoop 1.x,

[jira] [Updated] (SPARK-8121) When using with Hadoop 1.x, spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8121: -- Description: When using Spark with Hadoop 1.x (the version I tested is 1.2.0) and

[jira] [Updated] (SPARK-8121) spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8121: -- Description: When {{spark.sql.sources.outputCommitterClass}} is configured,

[jira] [Updated] (SPARK-8121) spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8121: -- Description: When {{spark.sql.sources.outputCommitterClass}} is configured,

[jira] [Created] (SPARK-8123) Bucketizer must implement copy

Xiangrui Meng created SPARK-8123: Summary: Bucketizer must implement copy Key: SPARK-8123 URL: https://issues.apache.org/jira/browse/SPARK-8123 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-8121) When using with Hadoop 1.x, spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8121: -- Summary: When using with Hadoop 1.x, spark.sql.parquet.output.committer.class is overriden by

[jira] [Commented] (SPARK-7596) Let AM's Reporter thread to wake up from sleep if new executors required

[ https://issues.apache.org/jira/browse/SPARK-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574145#comment-14574145 ] Zoltán Zvara commented on SPARK-7596: - This has been fixed in

[jira] [Created] (SPARK-8124) Created more examples on SparkR DataFrames

Daniel Emaasit created SPARK-8124: - Summary: Created more examples on SparkR DataFrames Key: SPARK-8124 URL: https://issues.apache.org/jira/browse/SPARK-8124 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-8121) When using with Hadoop 1.x, spark.sql.parquet.output.committer.class is overriden by spark.sql.sources.outputCommitterClass

[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8121: -- Description: When using Spark with Hadoop 1.x (the version I tested is 1.2.0) and

[jira] [Commented] (SPARK-8114) Remove wildcard import on TestSQLContext._

[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574137#comment-14574137 ] Apache Spark commented on SPARK-8114: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574287#comment-14574287 ] Cheng Lian commented on SPARK-8122: --- Just saw this JIRA ticket after opening [PR

[jira] [Commented] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574291#comment-14574291 ] Cheng Lian commented on SPARK-8122: --- Hm, just noticed that the logger created in the

[jira] [Assigned] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8118: --- Assignee: Apache Spark (was: Cheng Lian) Turn off noisy log output produced by Parquet

[jira] [Resolved] (SPARK-7596) Let AM's Reporter thread to wake up from sleep if new executors required

[ https://issues.apache.org/jira/browse/SPARK-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7596. -- Resolution: Duplicate Let AM's Reporter thread to wake up from sleep if new executors required

[jira] [Commented] (SPARK-4001) Add FP-growth algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574259#comment-14574259 ] Guangwen Liu commented on SPARK-4001: - Hi, Xiangrui. Thanks for your great talk last

[jira] [Comment Edited] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574287#comment-14574287 ] Cheng Lian edited comment on SPARK-8122 at 6/5/15 10:39 AM:

[jira] [Comment Edited] (SPARK-5493) Support proxy users under kerberos

[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574359#comment-14574359 ] Kaveen Raajan edited comment on SPARK-5493 at 6/5/15 12:12 PM:

[jira] [Commented] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574419#comment-14574419 ] Konstantin Shaposhnikov commented on SPARK-8122: Parquet itself surfers

[jira] [Commented] (SPARK-7106) Support model save/load in Python's FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574225#comment-14574225 ] Hrishikesh commented on SPARK-7106: --- Do we have support for save/load in scala?

[jira] [Created] (SPARK-8125) Accelerate ParquetRelation2 metadata discovery

Cheng Lian created SPARK-8125: - Summary: Accelerate ParquetRelation2 metadata discovery Key: SPARK-8125 URL: https://issues.apache.org/jira/browse/SPARK-8125 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-5493) Support proxy users under kerberos

[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574359#comment-14574359 ] Kaveen Raajan edited comment on SPARK-5493 at 6/5/15 11:43 AM:

[jira] [Commented] (SPARK-5493) Support proxy users under kerberos

[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574359#comment-14574359 ] Kaveen Raajan commented on SPARK-5493: -- I'm using *SPARK-1.3.1* on *windows machine*

[jira] [Commented] (SPARK-8016) YARN cluster / client modes have different app names for python

[ https://issues.apache.org/jira/browse/SPARK-8016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574396#comment-14574396 ] Apache Spark commented on SPARK-8016: - User 'ehnalis' has created a pull request for

[jira] [Assigned] (SPARK-8016) YARN cluster / client modes have different app names for python

[ https://issues.apache.org/jira/browse/SPARK-8016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8016: --- Assignee: (was: Apache Spark) YARN cluster / client modes have different app names for

[jira] [Assigned] (SPARK-8016) YARN cluster / client modes have different app names for python

[ https://issues.apache.org/jira/browse/SPARK-8016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8016: --- Assignee: Apache Spark YARN cluster / client modes have different app names for python

[jira] [Commented] (SPARK-1018) take and collect don't work on HadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574472#comment-14574472 ] Igor Berman commented on SPARK-1018: Hi Patrick, We spent some time to understand why

[jira] [Updated] (SPARK-6324) Clean up usage code in command-line scripts

[ https://issues.apache.org/jira/browse/SPARK-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6324: - Assignee: Marcelo Vanzin Clean up usage code in command-line scripts

[jira] [Resolved] (SPARK-6324) Clean up usage code in command-line scripts

[ https://issues.apache.org/jira/browse/SPARK-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6324. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 5841

[jira] [Commented] (SPARK-6107) event log file ends with .inprogress should be able to display on webUI for standalone mode

[ https://issues.apache.org/jira/browse/SPARK-6107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574510#comment-14574510 ] Octavian Ganea commented on SPARK-6107: --- I still see this in 1.3.1 . I have

[jira] [Commented] (SPARK-6950) Spark master UI believes some applications are in progress when they are actually completed

[ https://issues.apache.org/jira/browse/SPARK-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574514#comment-14574514 ] Octavian Ganea commented on SPARK-6950: --- Happens to me on 1.3.1 Spark master UI

[jira] [Commented] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574940#comment-14574940 ] Reynold Xin commented on SPARK-8056: Add it here:

[jira] [Updated] (SPARK-8105) sqlContext.table(databaseName.tableName) broke with SPARK-6908

[ https://issues.apache.org/jira/browse/SPARK-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8105: Target Version/s: 1.5.0 (was: 1.4.1, 1.5.0) sqlContext.table(databaseName.tableName) broke with

[jira] [Commented] (SPARK-8105) sqlContext.table(databaseName.tableName) broke with SPARK-6908

[ https://issues.apache.org/jira/browse/SPARK-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574956#comment-14574956 ] Yin Huai commented on SPARK-8105: - Actually, as I said in the mailing list, it is not

[jira] [Updated] (SPARK-8105) sqlContext.table(databaseName.tableName) broke with SPARK-6908

[ https://issues.apache.org/jira/browse/SPARK-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8105: Issue Type: New Feature (was: Bug) sqlContext.table(databaseName.tableName) broke with SPARK-6908

[jira] [Updated] (SPARK-8105) sqlContext.table(databaseName.tableName) broke with SPARK-6908

[ https://issues.apache.org/jira/browse/SPARK-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8105: Target Version/s: 1.4.1, 1.5.0 (was: 1.4.1) sqlContext.table(databaseName.tableName) broke with

[jira] [Commented] (SPARK-8102) Big performance difference when joining 3 tables in different order

[ https://issues.apache.org/jira/browse/SPARK-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574943#comment-14574943 ] Yin Huai commented on SPARK-8102: - Can you post query plans with these two queries? You

[jira] [Assigned] (SPARK-8077) Optimisation of TreeNode for large number of children

[ https://issues.apache.org/jira/browse/SPARK-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8077: --- Assignee: (was: Apache Spark) Optimisation of TreeNode for large number of children

[jira] [Updated] (SPARK-8099) In yarn-cluster mode, --executor-cores can't be setted into SparkConf

[ https://issues.apache.org/jira/browse/SPARK-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-8099: -- Assignee: meiyoula In yarn-cluster mode, --executor-cores can't be setted into SparkConf

[jira] [Updated] (SPARK-8099) In yarn-cluster mode, --executor-cores can't be setted into SparkConf

[ https://issues.apache.org/jira/browse/SPARK-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-8099: -- Assignee: (was: meiyoula) In yarn-cluster mode, --executor-cores can't be setted into SparkConf

[jira] [Commented] (SPARK-8093) Failure to save empty json object as parquet

[ https://issues.apache.org/jira/browse/SPARK-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575013#comment-14575013 ] Harish Butani commented on SPARK-8093: -- yes Failure to save empty json object as

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574933#comment-14574933 ] Yu Ishikawa commented on SPARK-5992: h2. Initial version of desing doc

[jira] [Commented] (SPARK-8077) Optimisation of TreeNode for large number of children

[ https://issues.apache.org/jira/browse/SPARK-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574941#comment-14574941 ] Apache Spark commented on SPARK-8077: - User 'MickDavies' has created a pull request

[jira] [Commented] (SPARK-8093) Failure to save empty json object as parquet

[ https://issues.apache.org/jira/browse/SPARK-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574952#comment-14574952 ] Yin Huai commented on SPARK-8093: - [~rhbutani] Is your test based on RC4? Failure to

[jira] [Resolved] (SPARK-8099) In yarn-cluster mode, --executor-cores can't be setted into SparkConf

[ https://issues.apache.org/jira/browse/SPARK-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-8099. --- Resolution: Fixed Fix Version/s: 1.5.0 Assignee: meiyoula In yarn-cluster mode,

[jira] [Created] (SPARK-8126) Use temp directory under build dir for unit tests

Marcelo Vanzin created SPARK-8126: - Summary: Use temp directory under build dir for unit tests Key: SPARK-8126 URL: https://issues.apache.org/jira/browse/SPARK-8126 Project: Spark Issue

[jira] [Updated] (SPARK-8099) In yarn-cluster mode, --executor-cores can't be setted into SparkConf

[ https://issues.apache.org/jira/browse/SPARK-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8099: - Affects Version/s: 1.0.0 In yarn-cluster mode, --executor-cores can't be setted into SparkConf

[jira] [Assigned] (SPARK-8077) Optimisation of TreeNode for large number of children

[ https://issues.apache.org/jira/browse/SPARK-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8077: --- Assignee: Apache Spark Optimisation of TreeNode for large number of children

[jira] [Assigned] (SPARK-8126) Use temp directory under build dir for unit tests

[ https://issues.apache.org/jira/browse/SPARK-8126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8126: --- Assignee: (was: Apache Spark) Use temp directory under build dir for unit tests

[jira] [Assigned] (SPARK-8126) Use temp directory under build dir for unit tests

[ https://issues.apache.org/jira/browse/SPARK-8126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8126: --- Assignee: Apache Spark Use temp directory under build dir for unit tests

[jira] [Commented] (SPARK-8093) Failure to save empty json object as parquet

[ https://issues.apache.org/jira/browse/SPARK-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575023#comment-14575023 ] Yin Huai commented on SPARK-8093: - When you get time, can you try it with master? We just

[jira] [Resolved] (SPARK-8107) sqlContext.table() should be able to take a database name as an additional argument.

[ https://issues.apache.org/jira/browse/SPARK-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-8107. - Resolution: Duplicate sqlContext.table() should be able to take a database name as an additional

[jira] [Commented] (SPARK-6987) Node Locality is determined with String Matching instead of Inet Comparison

[ https://issues.apache.org/jira/browse/SPARK-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574983#comment-14574983 ] Russell Alexander Spitzer commented on SPARK-6987: -- Or being able to

[jira] [Commented] (SPARK-8126) Use temp directory under build dir for unit tests

[ https://issues.apache.org/jira/browse/SPARK-8126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575003#comment-14575003 ] Apache Spark commented on SPARK-8126: - User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575002#comment-14575002 ] Karl Higley commented on SPARK-5992: To make it easier to define a common interface,

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574750#comment-14574750 ] Yu Ishikawa commented on SPARK-5992: [~debasish83] Thank you for your comment. I

[jira] [Commented] (SPARK-4072) Storage UI does not reflect memory usage by streaming blocks

[ https://issues.apache.org/jira/browse/SPARK-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574691#comment-14574691 ] Apache Spark commented on SPARK-4072: - User 'zsxwing' has created a pull request for

[jira] [Comment Edited] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574847#comment-14574847 ] Ilya Ganelin edited comment on SPARK-8056 at 6/5/15 5:18 PM: -

[jira] [Resolved] (SPARK-8085) Pass in user-specified schema in read.df

[ https://issues.apache.org/jira/browse/SPARK-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-8085. -- Resolution: Fixed Fix Version/s: 1.4.1 1.5.0 Issue

[jira] [Commented] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574847#comment-14574847 ] Ilya Ganelin commented on SPARK-8056: - [~rxin] Sounds good :). Where would you suggest

[jira] [Comment Edited] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574847#comment-14574847 ] Ilya Ganelin edited comment on SPARK-8056 at 6/5/15 5:17 PM: -

[jira] [Assigned] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8118: --- Assignee: Cheng Lian (was: Apache Spark) Turn off noisy log output produced by Parquet

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574231#comment-14574231 ] Apache Spark commented on SPARK-8118: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-8064) Upgrade Hive to 1.2

[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575070#comment-14575070 ] Steve Loughran commented on SPARK-8064: --- I'm working on this Upgrade Hive to 1.2

[jira] [Updated] (SPARK-8128) Dataframe Fails to Recognize Column in Schema

[ https://issues.apache.org/jira/browse/SPARK-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Willard updated SPARK-8128: Description: I'm loading a folder of parquet files with about 600 parquet files and loading it

[jira] [Assigned] (SPARK-8129) Securely pass auth secret to executors in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8129: --- Assignee: (was: Apache Spark) Securely pass auth secret to executors in standalone

[jira] [Commented] (SPARK-8129) Securely pass auth secret to executors in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575323#comment-14575323 ] Apache Spark commented on SPARK-8129: - User 'kanzhang' has created a pull request for

[jira] [Assigned] (SPARK-8129) Securely pass auth secret to executors in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8129: --- Assignee: Apache Spark Securely pass auth secret to executors in standalone cluster mode

[jira] [Resolved] (SPARK-7991) Python DataFrame: support passing a list into describe

[ https://issues.apache.org/jira/browse/SPARK-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7991. Resolution: Fixed Fix Version/s: 1.4.1 Assignee: Amey Chaugule Python DataFrame:

[jira] [Updated] (SPARK-7991) Python DataFrame: support passing a list into describe

[ https://issues.apache.org/jira/browse/SPARK-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7991: --- Fix Version/s: 1.5.0 Python DataFrame: support passing a list into describe

[jira] [Created] (SPARK-8127) KafkaRDD optimize count() take() isEmpty()

Cody Koeninger created SPARK-8127: - Summary: KafkaRDD optimize count() take() isEmpty() Key: SPARK-8127 URL: https://issues.apache.org/jira/browse/SPARK-8127 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-7747) Document spark.sql.planner.externalSort option

[ https://issues.apache.org/jira/browse/SPARK-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-7747. - Resolution: Fixed It has been fixed by https://github.com/apache/spark/pull/6272. Since it is a doc

[jira] [Updated] (SPARK-7747) Document spark.sql.planner.externalSort option

[ https://issues.apache.org/jira/browse/SPARK-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-7747: Fix Version/s: 1.4.0 Document spark.sql.planner.externalSort option

[jira] [Comment Edited] (SPARK-7747) Document spark.sql.planner.externalSort option

[ https://issues.apache.org/jira/browse/SPARK-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575193#comment-14575193 ] Yin Huai edited comment on SPARK-7747 at 6/5/15 8:47 PM: - It has

[jira] [Updated] (SPARK-8064) Upgrade Hive to 1.2

[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8064: --- Assignee: Steve Loughran Upgrade Hive to 1.2 --- Key: SPARK-8064

[jira] [Resolved] (SPARK-7699) Dynamic allocation: initial executors may be canceled before first job

[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-7699. --- Resolution: Fixed Fix Version/s: 1.5.0 Dynamic allocation: initial executors may be canceled

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575079#comment-14575079 ] Joseph K. Bradley commented on SPARK-5992: -- I'm just noting here that

[jira] [Resolved] (SPARK-8112) Received block event count through the StreamingListener can be negative

[ https://issues.apache.org/jira/browse/SPARK-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-8112. -- Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 Received block event

[jira] [Commented] (SPARK-8107) sqlContext.table() should be able to take a database name as an additional argument.

[ https://issues.apache.org/jira/browse/SPARK-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575189#comment-14575189 ] Apache Spark commented on SPARK-8107: - User 'dougb' has created a pull request for

[jira] [Updated] (SPARK-7041) Avoid writing empty files in BypassMergeSortShuffleWriter

[ https://issues.apache.org/jira/browse/SPARK-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7041: -- Description: In BypassMergeSortShuffleWriter, we may end up opening disk writers files for empty

[jira] [Assigned] (SPARK-8127) KafkaRDD optimize count() take() isEmpty()

[ https://issues.apache.org/jira/browse/SPARK-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8127: --- Assignee: (was: Apache Spark) KafkaRDD optimize count() take() isEmpty()

[jira] [Commented] (SPARK-8127) KafkaRDD optimize count() take() isEmpty()

[ https://issues.apache.org/jira/browse/SPARK-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575184#comment-14575184 ] Apache Spark commented on SPARK-8127: - User 'koeninger' has created a pull request for

[jira] [Assigned] (SPARK-8127) KafkaRDD optimize count() take() isEmpty()

[ https://issues.apache.org/jira/browse/SPARK-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8127: --- Assignee: Apache Spark KafkaRDD optimize count() take() isEmpty()

[jira] [Created] (SPARK-8128) Dataframe Fails to Recognize Column in Schema

Brad Willard created SPARK-8128: --- Summary: Dataframe Fails to Recognize Column in Schema Key: SPARK-8128 URL: https://issues.apache.org/jira/browse/SPARK-8128 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5784) Add StatsDSink to MetricsSystem

[ https://issues.apache.org/jira/browse/SPARK-5784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vidhya Arvind updated SPARK-5784: - Attachment: statsd.patch Attaching patch file. Not sure how to get this jira reopened and patch

[jira] [Updated] (SPARK-7041) Avoid writing empty files in BypassMergeSortShuffleWriter

[ https://issues.apache.org/jira/browse/SPARK-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7041: -- Summary: Avoid writing empty files in BypassMergeSortShuffleWriter (was: Avoid writing empty files in

[jira] [Created] (SPARK-8129) Securely pass auth secret to executors in standalone cluster mode

Kan Zhang created SPARK-8129: Summary: Securely pass auth secret to executors in standalone cluster mode Key: SPARK-8129 URL: https://issues.apache.org/jira/browse/SPARK-8129 Project: Spark

[jira] [Commented] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575083#comment-14575083 ] Joseph K. Bradley commented on SPARK-7334: -- I won't be able to look at the PR

[jira] [Created] (SPARK-8130) spark.files.useFetchCache should be off by default

Josh Rosen created SPARK-8130: - Summary: spark.files.useFetchCache should be off by default Key: SPARK-8130 URL: https://issues.apache.org/jira/browse/SPARK-8130 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-8105) sqlContext.table(databaseName.tableName) broke with SPARK-6908

[ https://issues.apache.org/jira/browse/SPARK-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8105: Issue Type: Sub-task (was: New Feature) Parent: SPARK-8131

[jira] [Updated] (SPARK-7943) saveAsTable in DataFrameWriter can only add table to DataBase “default”

[ https://issues.apache.org/jira/browse/SPARK-7943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-7943: Issue Type: Sub-task (was: Bug) Parent: SPARK-8131 saveAsTable in DataFrameWriter can only add

[jira] [Updated] (SPARK-8129) Securely pass auth secret to executors in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang updated SPARK-8129: - Description: Currently, when authentication is turned on, Worker passes auth secret to executors (also

[jira] [Updated] (SPARK-8129) Securely pass auth secret to executors in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang updated SPARK-8129: - Description: Currently, when authentication is turned on, cluster manager passes auth secrets to

[jira] [Updated] (SPARK-8135) Don't load defaults when reconstituting Hadoop Configurations

[ https://issues.apache.org/jira/browse/SPARK-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-8135: -- Summary: Don't load defaults when reconstituting Hadoop Configurations (was: In SerializableWritable,

[jira] [Commented] (SPARK-8114) Remove wildcard import on TestSQLContext._

[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575385#comment-14575385 ] Apache Spark commented on SPARK-8114: - User 'rxin' has created a pull request for this

  1   2   >