[jira] [Resolved] (SPARK-6182) spark-parent pom needs to be published for both 2.10 and 2.11

[ https://issues.apache.org/jira/browse/SPARK-6182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-6182. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Sean Owen spark-parent

[jira] [Comment Edited] (SPARK-5981) pyspark ML models should support predict/transform on vector within map

[ https://issues.apache.org/jira/browse/SPARK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349347#comment-14349347 ] Manoj Kumar edited comment on SPARK-5981 at 3/5/15 7:38 PM:

[jira] [Updated] (SPARK-6175) Executor log links are using internal addresses in EC2; display `:0` when ephemeral ports are used

[ https://issues.apache.org/jira/browse/SPARK-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6175: --- Priority: Blocker (was: Major) Executor log links are using internal addresses in EC2;

[jira] [Commented] (SPARK-6191) Generalize spark-ec2's ability to download libraries from PyPI

[ https://issues.apache.org/jira/browse/SPARK-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349434#comment-14349434 ] Apache Spark commented on SPARK-6191: - User 'nchammas' has created a pull request for

[jira] [Commented] (SPARK-5981) pyspark ML models should support predict/transform on vector within map

[ https://issues.apache.org/jira/browse/SPARK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349347#comment-14349347 ] Manoj Kumar commented on SPARK-5981: Thanks a lot for your patient explanation, I'm

[jira] [Updated] (SPARK-6191) Generalize spark-ec2's ability to download libraries from PyPI

[ https://issues.apache.org/jira/browse/SPARK-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-6191: Description: Right now we have a method to specifically download boto. Let's generalize it

[jira] [Created] (SPARK-6191) Generalize spark-ec2's ability to download libraries from PyPI

Nicholas Chammas created SPARK-6191: --- Summary: Generalize spark-ec2's ability to download libraries from PyPI Key: SPARK-6191 URL: https://issues.apache.org/jira/browse/SPARK-6191 Project: Spark

[jira] [Updated] (SPARK-6191) Generalize spark-ec2's ability to download libraries from PyPI

[ https://issues.apache.org/jira/browse/SPARK-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-6191: Description: Right now we have a method to specifically download boto. Let's generalize it

[jira] [Commented] (SPARK-6145) ORDER BY fails to resolve nested fields

[ https://issues.apache.org/jira/browse/SPARK-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349421#comment-14349421 ] Apache Spark commented on SPARK-6145: - User 'marmbrus' has created a pull request for

[jira] [Resolved] (SPARK-6090) Add BinaryClassificationMetrics in PySpark/MLlib

[ https://issues.apache.org/jira/browse/SPARK-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6090. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4863

[jira] [Resolved] (SPARK-6175) Executor log links are using internal addresses in EC2; display `:0` when ephemeral ports are used

[ https://issues.apache.org/jira/browse/SPARK-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6175. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4903

[jira] [Commented] (SPARK-4311) ContainerLauncher setting up executor -- invalid Xms settings (-Xms0m -Xmx0m)

[ https://issues.apache.org/jira/browse/SPARK-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349468#comment-14349468 ] Rafael Alfaro Flores commented on SPARK-4311: - [~kadiyalakc] Were you able to

[jira] [Updated] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6192: - Labels: gsoc gsoc2015 mentor (was: gsoc gsoc2015) Enhance MLlib's Python API (GSoC 2015)

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator-Iterable is inconsistent with Scala's Iterator-Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349594#comment-14349594 ] Sean Owen commented on SPARK-3369: -- I sympathize with this. The problem is that changing

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator-Iterable is inconsistent with Scala's Iterator-Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349593#comment-14349593 ] Sean Owen commented on SPARK-3369: -- I sympathize with this. The problem is that changing

[jira] [Created] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

Xiangrui Meng created SPARK-6192: Summary: Enhance MLlib's Python API (GSoC 2015) Key: SPARK-6192 URL: https://issues.apache.org/jira/browse/SPARK-6192 Project: Spark Issue Type: Umbrella

[jira] [Resolved] (SPARK-6145) ORDER BY fails to resolve nested fields

[ https://issues.apache.org/jira/browse/SPARK-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6145. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4918

[jira] [Resolved] (SPARK-6163) jsonFile should be backed by the data source API

[ https://issues.apache.org/jira/browse/SPARK-6163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6163. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4896

[jira] [Comment Edited] (SPARK-3369) Java mapPartitions Iterator-Iterable is inconsistent with Scala's Iterator-Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122611#comment-14122611 ] Lukas Nalezenec edited comment on SPARK-3369 at 3/5/15 10:50 PM:

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator-Iterable is inconsistent with Scala's Iterator-Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349577#comment-14349577 ] Nicholas Chammas commented on SPARK-3369: - {quote} How about breaking backward

[jira] [Commented] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349646#comment-14349646 ] Nishkam Ravi commented on SPARK-6142: - [~zsxwing]Yes, the 10-12% regression was

[jira] [Updated] (SPARK-6095) Support model save/load in Python's linear models

[ https://issues.apache.org/jira/browse/SPARK-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6095: - Assignee: Yanbo Liang Support model save/load in Python's linear models

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349689#comment-14349689 ] Marcelo Vanzin commented on SPARK-5124: --- re: a local Endpoint, if it's really

[jira] [Created] (SPARK-6194) collect() in PySpark will cause memory leak in JVM

Davies Liu created SPARK-6194: - Summary: collect() in PySpark will cause memory leak in JVM Key: SPARK-6194 URL: https://issues.apache.org/jira/browse/SPARK-6194 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6196) Add MAPR 4.0.2 support to the build

[ https://issues.apache.org/jira/browse/SPARK-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350034#comment-14350034 ] Apache Spark commented on SPARK-6196: - User 'trystanleftwich' has created a pull

[jira] [Updated] (SPARK-6095) Support model save/load in Python's linear models

[ https://issues.apache.org/jira/browse/SPARK-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6095: - Target Version/s: 1.4.0 Support model save/load in Python's linear models

[jira] [Created] (SPARK-6193) Speed up how spark-ec2 searches for clusters

Nicholas Chammas created SPARK-6193: --- Summary: Speed up how spark-ec2 searches for clusters Key: SPARK-6193 URL: https://issues.apache.org/jira/browse/SPARK-6193 Project: Spark Issue Type:

[jira] [Created] (SPARK-6195) Specialized in-memory column type for decimal

Cheng Lian created SPARK-6195: - Summary: Specialized in-memory column type for decimal Key: SPARK-6195 URL: https://issues.apache.org/jira/browse/SPARK-6195 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6194) collect() in PySpark will cause memory leak in JVM

[ https://issues.apache.org/jira/browse/SPARK-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349803#comment-14349803 ] Apache Spark commented on SPARK-6194: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-6113) Stabilize DecisionTree and ensembles APIs

[ https://issues.apache.org/jira/browse/SPARK-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349811#comment-14349811 ] Joseph K. Bradley commented on SPARK-6113: -- Pinging [~MechCoder] since you've

[jira] [Created] (SPARK-6196) Add MAPR 4.0.2 support to the build

Trystan Leftwich created SPARK-6196: --- Summary: Add MAPR 4.0.2 support to the build Key: SPARK-6196 URL: https://issues.apache.org/jira/browse/SPARK-6196 Project: Spark Issue Type:

[jira] [Updated] (SPARK-6194) collect() in PySpark will cause memory leak in JVM

[ https://issues.apache.org/jira/browse/SPARK-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6194: -- Description: It could be reproduced by: {code} for i in range(40): sc.parallelize(range(5000),

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349808#comment-14349808 ] Shixiong Zhu commented on SPARK-5124: - 1. For the local Endpoint, let's put it in a

[jira] [Updated] (SPARK-6141) Upgrade Breeze to 0.11 to fix convergence bug

[ https://issues.apache.org/jira/browse/SPARK-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6141: --- Fix Version/s: (was: 1.3.1) 1.3.0 Upgrade Breeze to 0.11 to fix

[jira] [Created] (SPARK-6197) handle json parse exception for eventlog file not finished writing

Zhang, Liye created SPARK-6197: -- Summary: handle json parse exception for eventlog file not finished writing Key: SPARK-6197 URL: https://issues.apache.org/jira/browse/SPARK-6197 Project: Spark

[jira] [Commented] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349052#comment-14349052 ] Sean Owen commented on SPARK-5838: -- I don't know a lot about this, but I don't know if

[jira] [Commented] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349059#comment-14349059 ] Ernesto Alejandro Menéndez Castillo commented on SPARK-4925: I

[jira] [Updated] (SPARK-6187) Report full executor exceptions to the driver

[ https://issues.apache.org/jira/browse/SPARK-6187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6187: - Priority: Minor (was: Major) It already does this: {{sc.parallelize(Array(1,2,3)).map { i = throw new

[jira] [Commented] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349124#comment-14349124 ] Theodore Vasiloudis commented on SPARK-5838: I guess what remains here is the

[jira] [Commented] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349131#comment-14349131 ] Theodore Vasiloudis commented on SPARK-5838: I had misunderstood the purpose

[jira] [Commented] (SPARK-6188) Instance types can be mislabeled when re-starting cluster with default arguments

[ https://issues.apache.org/jira/browse/SPARK-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349115#comment-14349115 ] Apache Spark commented on SPARK-6188: - User 'thvasilo' has created a pull request for

[jira] [Commented] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349130#comment-14349130 ] Sean Owen commented on SPARK-5838: -- I don't expect that processes re-read the environment

[jira] [Commented] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349139#comment-14349139 ] Yin Huai commented on SPARK-5791: - [~jameszhouyi] Thank you for the updated physical plan.

[jira] [Created] (SPARK-6189) Pandas to DataFrame conversion should check field names for periods

Joseph K. Bradley created SPARK-6189: Summary: Pandas to DataFrame conversion should check field names for periods Key: SPARK-6189 URL: https://issues.apache.org/jira/browse/SPARK-6189 Project:

[jira] [Commented] (SPARK-5981) pyspark ML models should support predict/transform on vector within map

[ https://issues.apache.org/jira/browse/SPARK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349179#comment-14349179 ] Manoj Kumar commented on SPARK-5981: Sorry for being slow. But do you mean when

[jira] [Comment Edited] (SPARK-5981) pyspark ML models should support predict/transform on vector within map

[ https://issues.apache.org/jira/browse/SPARK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349179#comment-14349179 ] Manoj Kumar edited comment on SPARK-5981 at 3/5/15 6:04 PM:

[jira] [Commented] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349183#comment-14349183 ] Yin Huai commented on SPARK-5791: - Also, how large is the results of name subquery?

[jira] [Created] (SPARK-6190) create LargeByteBuffer abstraction for eliminating 2GB limit on blocks

Imran Rashid created SPARK-6190: --- Summary: create LargeByteBuffer abstraction for eliminating 2GB limit on blocks Key: SPARK-6190 URL: https://issues.apache.org/jira/browse/SPARK-6190 Project: Spark

[jira] [Commented] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349184#comment-14349184 ] Sean Owen commented on SPARK-5838: -- I would link it to the continuation JIRA maybe, and

[jira] [Updated] (SPARK-6153) intellij import from maven cannot debug sparksqlclidriver

[ https://issues.apache.org/jira/browse/SPARK-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6153: -- Assignee: Adrian Wang intellij import from maven cannot debug sparksqlclidriver

[jira] [Resolved] (SPARK-6153) intellij import from maven cannot debug sparksqlclidriver

[ https://issues.apache.org/jira/browse/SPARK-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6153. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4884

[jira] [Updated] (SPARK-6153) intellij import from maven cannot debug sparksqlclidriver

[ https://issues.apache.org/jira/browse/SPARK-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6153: -- Description: The {{hive-thriftserver}} module depends on Guava indirectly via {{hive}} module. However,

[jira] [Comment Edited] (SPARK-5763) Sort-based Groupby and Join to resolve skewed data

[ https://issues.apache.org/jira/browse/SPARK-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348361#comment-14348361 ] Jianshi Huang edited comment on SPARK-5763 at 3/5/15 8:21 AM: --

[jira] [Commented] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348444#comment-14348444 ] Shixiong Zhu commented on SPARK-6142: - @nishkamravi2 I tested in our environment, and

[jira] [Created] (SPARK-6185) Deltele repeated TOKEN. TOK_CREATEFUNCTION has existed at Line 84;

DoingDone9 created SPARK-6185: - Summary: Deltele repeated TOKEN. TOK_CREATEFUNCTION has existed at Line 84; Key: SPARK-6185 URL: https://issues.apache.org/jira/browse/SPARK-6185 Project: Spark

[jira] [Updated] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xia Hu updated SPARK-6135: -- Attachment: check_hostname.patch this patch check_hostname.patch check for hostname, and use the IP address if

[jira] [Issue Comment Deleted] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6135: - Comment: was deleted (was: Sorry I didn't make it clearly. SPARK_LOCAL_HOSTNAME can solve this problem,

[jira] [Commented] (SPARK-6185) Deltele repeated TOKEN. TOK_CREATEFUNCTION has existed at Line 84;

[ https://issues.apache.org/jira/browse/SPARK-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348460#comment-14348460 ] Apache Spark commented on SPARK-6185: - User 'DoingDone9' has created a pull request

[jira] [Issue Comment Deleted] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6135: - Comment: was deleted (was: Sorry I didn't make it clearly. SPARK_LOCAL_HOSTNAME can solve this problem,

[jira] [Resolved] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6135. -- Resolution: Not a Problem We use pull requests rather than patches to propose a change. That said, this

[jira] [Issue Comment Deleted] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6135: - Comment: was deleted (was: Sorry I didn't make it clearly. SPARK_LOCAL_HOSTNAME can solve this problem,

[jira] [Issue Comment Deleted] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6135: - Comment: was deleted (was: Sorry I didn't make it clearly. SPARK_LOCAL_HOSTNAME can solve this problem,

[jira] [Updated] (SPARK-6183) Skip bad workers when re-launching executors

[ https://issues.apache.org/jira/browse/SPARK-6183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peng Zhen updated SPARK-6183: - Description: In standalone cluster, when an executor launch fails, the master should avoid re-launching

[jira] [Commented] (SPARK-6025) Helper method for GradientBoostedTrees to compute validation error

[ https://issues.apache.org/jira/browse/SPARK-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348438#comment-14348438 ] Apache Spark commented on SPARK-6025: - User 'MechCoder' has created a pull request for

[jira] [Commented] (SPARK-5763) Sort-based Groupby and Join to resolve skewed data

[ https://issues.apache.org/jira/browse/SPARK-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348361#comment-14348361 ] Jianshi Huang commented on SPARK-5763: -- Upvote for this improvement. Jianshi

[jira] [Commented] (SPARK-6184) Relocate logDebug to correct location in ResolveSortReferences

[ https://issues.apache.org/jira/browse/SPARK-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348374#comment-14348374 ] Apache Spark commented on SPARK-6184: - User 'viirya' has created a pull request for

[jira] [Updated] (SPARK-6153) intellij import from maven cannot debug sparksqlclidriver

[ https://issues.apache.org/jira/browse/SPARK-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6153: -- Description: The {{hive-thriftserver}} module depends on Guava indirectly via {{hive]} module. However,

[jira] [Comment Edited] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348444#comment-14348444 ] Shixiong Zhu edited comment on SPARK-6142 at 3/5/15 9:05 AM: -

[jira] [Created] (SPARK-6184) Relocate logDebug to correct location in ResolveSortReferences

Liang-Chi Hsieh created SPARK-6184: -- Summary: Relocate logDebug to correct location in ResolveSortReferences Key: SPARK-6184 URL: https://issues.apache.org/jira/browse/SPARK-6184 Project: Spark

[jira] [Commented] (SPARK-6153) intellij import from maven cannot debug sparksqlclidriver

[ https://issues.apache.org/jira/browse/SPARK-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348407#comment-14348407 ] Cheng Lian commented on SPARK-6153: --- Updated JIRA description to reflect the actual

[jira] [Commented] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348452#comment-14348452 ] Sean Owen commented on SPARK-6142: -- Yeah I'm curious how this change would affect

[jira] [Issue Comment Deleted] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6135: - Comment: was deleted (was: Sorry I didn't make it clearly. SPARK_LOCAL_HOSTNAME can solve this problem,

[jira] [Updated] (SPARK-6185) Deltele repeated TOKEN. TOK_CREATEFUNCTION has existed at Line 84;

[ https://issues.apache.org/jira/browse/SPARK-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6185: -- Description: TOK_CREATEFUNCTION has existed at Line 84; Line 84TOK_CREATEFUNCTION, Line 85

[jira] [Issue Comment Deleted] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

[ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6135: - Comment: was deleted (was: Sorry I didn't make it clearly. SPARK_LOCAL_HOSTNAME can solve this problem,

[jira] [Created] (SPARK-6183) Skip bad workers when re-launching executors

Peng Zhen created SPARK-6183: Summary: Skip bad workers when re-launching executors Key: SPARK-6183 URL: https://issues.apache.org/jira/browse/SPARK-6183 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4833) spark-shell.cmd not starting again after a while in windows 8.1

[ https://issues.apache.org/jira/browse/SPARK-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348348#comment-14348348 ] Masayoshi TSUZUKI commented on SPARK-4833: -- Are you working on the network drive?

[jira] [Closed] (SPARK-6181) Support SHOW COMPACTIONS;

[ https://issues.apache.org/jira/browse/SPARK-6181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 closed SPARK-6181. - Resolution: Invalid sparkSQL does not support transactions Support SHOW COMPACTIONS;

[jira] [Commented] (SPARK-6145) ORDER BY fails to resolve nested fields

[ https://issues.apache.org/jira/browse/SPARK-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348350#comment-14348350 ] Apache Spark commented on SPARK-6145: - User 'cloud-fan' has created a pull request for

[jira] [Commented] (SPARK-4833) spark-shell.cmd not starting again after a while in windows 8.1

[ https://issues.apache.org/jira/browse/SPARK-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348351#comment-14348351 ] Masayoshi TSUZUKI commented on SPARK-4833: -- I forgot to say. I'm developing Spark

[jira] [Commented] (SPARK-6169) Shuffle based join

[ https://issues.apache.org/jira/browse/SPARK-6169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348995#comment-14348995 ] Mridul Muralidharan commented on SPARK-6169: I have not done too much research

[jira] [Updated] (SPARK-6175) Executor log links are using internal addresses in EC2; display `:0` when ephemeral ports are used

[ https://issues.apache.org/jira/browse/SPARK-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6175: -- Summary: Executor log links are using internal addresses in EC2; display `:0` when ephemeral ports are

[jira] [Commented] (SPARK-4560) Lambda deserialization error

[ https://issues.apache.org/jira/browse/SPARK-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349095#comment-14349095 ] Sean Owen commented on SPARK-4560: -- Some poking around on the internet suggests that it

[jira] [Assigned] (SPARK-6175) Executor log links are using internal addresses in EC2

[ https://issues.apache.org/jira/browse/SPARK-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-6175: - Assignee: Josh Rosen Executor log links are using internal addresses in EC2

[jira] [Created] (SPARK-6188) Instance types can be mislabeled when re-starting cluster with default arguments

Theodore Vasiloudis created SPARK-6188: -- Summary: Instance types can be mislabeled when re-starting cluster with default arguments Key: SPARK-6188 URL: https://issues.apache.org/jira/browse/SPARK-6188

[jira] [Resolved] (SPARK-1867) Spark Documentation Error causes java.lang.IllegalStateException: unread block data

[ https://issues.apache.org/jira/browse/SPARK-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1867. -- Resolution: Not a Problem OK, I think there are two problems with the same manifestation here, but I

[jira] [Commented] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349099#comment-14349099 ] Sean Owen commented on SPARK-5838: -- Is there anything here then that's not covered by

[jira] [Comment Edited] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348798#comment-14348798 ] Theodore Vasiloudis edited comment on SPARK-5838 at 3/5/15 3:41 PM:

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348987#comment-14348987 ] Mridul Muralidharan commented on SPARK-4879: [~joshrosen] The former - the

[jira] [Commented] (SPARK-6182) spark-parent pom needs to be published for both 2.10 and 2.11

[ https://issues.apache.org/jira/browse/SPARK-6182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348752#comment-14348752 ] Sean Owen commented on SPARK-6182: -- I think there are two ways to resolve this: 1)

[jira] [Updated] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-4879: --- Affects Version/s: 1.3.0 Missing output partitions after job completes with

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348522#comment-14348522 ] Mridul Muralidharan commented on SPARK-4879: With 1.3 RC, we are still seeing

[jira] [Created] (SPARK-6187) Report full executor exceptions to the driver

Piotr Kołaczkowski created SPARK-6187: - Summary: Report full executor exceptions to the driver Key: SPARK-6187 URL: https://issues.apache.org/jira/browse/SPARK-6187 Project: Spark Issue

[jira] [Commented] (SPARK-5838) Changing SPARK_LOCAL_DIRS option in spark-env.sh does not take effect without daemon restart

[ https://issues.apache.org/jira/browse/SPARK-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349004#comment-14349004 ] Theodore Vasiloudis commented on SPARK-5838: So this seems to be the case:

[jira] [Updated] (SPARK-6187) Report full executor exceptions to the driver

[ https://issues.apache.org/jira/browse/SPARK-6187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated SPARK-6187: -- Description: If the task fails for some reason, the driver seems to report only the

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349188#comment-14349188 ] Matt Cheah commented on SPARK-4879: --- Can you perhaps jstack or profile the driver and

[jira] [Commented] (SPARK-2311) Added additional GLMs (Poisson and Gamma) into MLlib

[ https://issues.apache.org/jira/browse/SPARK-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349192#comment-14349192 ] Sean Owen commented on SPARK-2311: -- I think this one may be obsolete as a suggested

[jira] [Commented] (SPARK-5981) pyspark ML models should support predict/transform on vector within map

[ https://issues.apache.org/jira/browse/SPARK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349217#comment-14349217 ] Joseph K. Bradley commented on SPARK-5981: -- I'd recommend exploring the code path

[jira] [Updated] (SPARK-6190) create LargeByteBuffer abstraction for eliminating 2GB limit on blocks

[ https://issues.apache.org/jira/browse/SPARK-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-6190: Attachment: LargeByteBuffer.pdf design doc create LargeByteBuffer abstraction for eliminating 2GB

[jira] [Commented] (SPARK-6190) create LargeByteBuffer abstraction for eliminating 2GB limit on blocks

[ https://issues.apache.org/jira/browse/SPARK-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349191#comment-14349191 ] Imran Rashid commented on SPARK-6190: - Hi [~rxin], I've attached a design doc here. I

[jira] [Commented] (SPARK-5396) Syntax error in spark scripts on windows.

[ https://issues.apache.org/jira/browse/SPARK-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349203#comment-14349203 ] Vladimir Protsenko commented on SPARK-5396: --- It says Syntax error in command..

  1   2   >