[jira] [Created] (SPARK-6474) Replace image.run with connection.run_instances

Andrew Drozdov created SPARK-6474: - Summary: Replace image.run with connection.run_instances Key: SPARK-6474 URL: https://issues.apache.org/jira/browse/SPARK-6474 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

[ https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376569#comment-14376569 ] Apache Spark commented on SPARK-6473: - User 'vanzin' has created a pull request for th

[jira] [Updated] (SPARK-6255) Python MLlib API missing items: Classification

[ https://issues.apache.org/jira/browse/SPARK-6255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6255: - Assignee: Yanbo Liang > Python MLlib API missing items: Classification > -

[jira] [Created] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

Marcelo Vanzin created SPARK-6473: - Summary: Launcher lib shouldn't try to figure out Scala version when not in dev mode Key: SPARK-6473 URL: https://issues.apache.org/jira/browse/SPARK-6473 Project:

[jira] [Commented] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376501#comment-14376501 ] vijay commented on SPARK-6435: -- I came up with square brackets after 2 minutes of googling/st

[jira] [Commented] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376495#comment-14376495 ] Patrick Wendell commented on SPARK-2331: By the way - [~rxin] recently pointed out

[jira] [Commented] (SPARK-5945) Spark should not retry a stage infinitely on a FetchFailedException

[ https://issues.apache.org/jira/browse/SPARK-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376454#comment-14376454 ] Imran Rashid commented on SPARK-5945: - Hi [~ilganeli], sorry for taking a while to re

[jira] [Updated] (SPARK-5945) Spark should not retry a stage infinitely on a FetchFailedException

[ https://issues.apache.org/jira/browse/SPARK-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-5945: Assignee: Ilya Ganelin > Spark should not retry a stage infinitely on a FetchFailedException > -

[jira] [Resolved] (SPARK-6200) Support dialect in SQL

[ https://issues.apache.org/jira/browse/SPARK-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6200. - Resolution: Duplicate > Support dialect in SQL > -- > >

[jira] [Commented] (SPARK-6200) Support dialect in SQL

[ https://issues.apache.org/jira/browse/SPARK-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376450#comment-14376450 ] Michael Armbrust commented on SPARK-6200: - Making comments there would be a good i

[jira] [Updated] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2331: --- Description: The return type for SparkContext.emptyRDD is EmptyRDD[T]. It should be RDD[T]. That mea

[jira] [Commented] (SPARK-6320) Adding new query plan strategy to SQLContext

[ https://issues.apache.org/jira/browse/SPARK-6320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376434#comment-14376434 ] Michael Armbrust commented on SPARK-6320: - If that can be done in a minimally inva

[jira] [Updated] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6435: - Component/s: Windows Great debugging! [~tsudukim] do you have thoughts on this? I think this bit was part

[jira] [Commented] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376430#comment-14376430 ] Imran Rashid commented on SPARK-5928: - sorry to hear that [~douglaz]. To help underst

[jira] [Resolved] (SPARK-6472) Elements of an array of structs cannot be accessed.

[ https://issues.apache.org/jira/browse/SPARK-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6472. - Resolution: Not a Problem It is not a problem. For select, we support column name string. I need to use s

[jira] [Updated] (SPARK-6472) Elements of an array of structs cannot be accessed.

[ https://issues.apache.org/jira/browse/SPARK-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6472: Component/s: SQL > Elements of an array of structs cannot be accessed. > ---

[jira] [Created] (SPARK-6472) Elements of an array of structs cannot be accessed.

Yin Huai created SPARK-6472: --- Summary: Elements of an array of structs cannot be accessed. Key: SPARK-6472 URL: https://issues.apache.org/jira/browse/SPARK-6472 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376409#comment-14376409 ] Sean Owen commented on SPARK-6469: -- [~preaudc] would you like to open a small PR to add a

[jira] [Commented] (SPARK-6213) sql.catalyst.expressions.Expression is not friendly to java

[ https://issues.apache.org/jira/browse/SPARK-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376394#comment-14376394 ] Michael Armbrust commented on SPARK-6213: - {{PrunedFilteredScan}} is exactly that,

[jira] [Reopened] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-2331: -- Reopening to catch this for version 2.x > SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] >

[jira] [Updated] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2331: - Priority: Minor (was: Major) Target Version/s: 2+ > SparkContext.emptyRDD should return RDD[T

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376361#comment-14376361 ] Zhan Zhang commented on SPARK-3720: --- [~iward] Since this jiar is duplicated to Spark-288

[jira] [Commented] (SPARK-4086) Fold-style aggregation for VertexRDD

[ https://issues.apache.org/jira/browse/SPARK-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376341#comment-14376341 ] Apache Spark commented on SPARK-4086: - User 'brennonyork' has created a pull request f

[jira] [Commented] (SPARK-4830) Spark Streaming Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376335#comment-14376335 ] sam commented on SPARK-4830: I've been seeing a similar problem but just for a regular Spark j

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376329#comment-14376329 ] Manoj Kumar commented on SPARK-6192: [~mengxr] Sorry for spamming, but do you have any

[jira] [Issue Comment Deleted] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-6471: -- Comment: was deleted (was: https://github.com/apache/spark/pull/5141) > Metastore schema should only be

[jira] [Commented] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376311#comment-14376311 ] Apache Spark commented on SPARK-6471: - User 'saucam' has created a pull request for th

[jira] [Commented] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376286#comment-14376286 ] Yash Datta commented on SPARK-6471: --- https://github.com/apache/spark/pull/5141 > Metast

[jira] [Created] (SPARK-6471) Metastoreschema should only be a subset of parquetSchema to support dropping of columns using replace columns

Yash Datta created SPARK-6471: - Summary: Metastoreschema should only be a subset of parquetSchema to support dropping of columns using replace columns Key: SPARK-6471 URL: https://issues.apache.org/jira/browse/SPARK-6

[jira] [Updated] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-6471: -- Summary: Metastore schema should only be a subset of parquet schema to support dropping of columns using

[jira] [Commented] (SPARK-4848) On a stand-alone cluster, several worker-specific variables are read only on the master

[ https://issues.apache.org/jira/browse/SPARK-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376239#comment-14376239 ] Apache Spark commented on SPARK-4848: - User 'nkronenfeld' has created a pull request f

[jira] [Assigned] (SPARK-6470) Allow Spark apps to put YARN node labels in their requests

[ https://issues.apache.org/jira/browse/SPARK-6470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-6470: - Assignee: Sandy Ryza > Allow Spark apps to put YARN node labels in their requests > -

[jira] [Commented] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376234#comment-14376234 ] Apache Spark commented on SPARK-6369: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-5954) Add topByKey to pair RDDs

[ https://issues.apache.org/jira/browse/SPARK-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376215#comment-14376215 ] Xiangrui Meng commented on SPARK-5954: -- Note: We added topByKey in mllib.rdd.MLPairRD

[jira] [Updated] (SPARK-6308) VectorUDT is displayed as `vecto` in dtypes

[ https://issues.apache.org/jira/browse/SPARK-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6308: - Assignee: Manoj Kumar > VectorUDT is displayed as `vecto` in dtypes >

[jira] [Closed] (SPARK-4746) integration tests should be separated from faster unit tests

[ https://issues.apache.org/jira/browse/SPARK-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid closed SPARK-4746. --- Resolution: Won't Fix looks like there isn't interest in this, closing to clean up jira > integration

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376147#comment-14376147 ] Marcelo Vanzin commented on SPARK-6229: --- [~adav] the problem of exposing the pipelin

[jira] [Created] (SPARK-6470) Allow Spark apps to put YARN node labels in their requests

Sandy Ryza created SPARK-6470: - Summary: Allow Spark apps to put YARN node labels in their requests Key: SPARK-6470 URL: https://issues.apache.org/jira/browse/SPARK-6470 Project: Spark Issue Type

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376119#comment-14376119 ] Thomas Graves commented on SPARK-6469: -- looks good, thanks. > The YARN driver in yar

[jira] [Updated] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-6469: - Component/s: (was: Spark Core) YARN > The YARN driver in yarn-client mode wil

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376111#comment-14376111 ] Christophe PRÉAUD commented on SPARK-6469: -- I've changed the JIRA title, type and

[jira] [Updated] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Description: According to the [Spark YARN doc page|http://spark.apache.org/docs/latest/ru

[jira] [Updated] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Summary: The YARN driver in yarn-client mode will not use the local directories configured

[jira] [Updated] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Description: According to the [Spark YARN doc page|http://spark.apache.org/docs/latest/ru

[jira] [Updated] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Issue Type: Documentation (was: Bug) > Local directories configured for YARN are not used

[jira] [Commented] (SPARK-6451) Support CombineSum in Code Gen

[ https://issues.apache.org/jira/browse/SPARK-6451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376088#comment-14376088 ] Apache Spark commented on SPARK-6451: - User 'gvramana' has created a pull request for

[jira] [Commented] (SPARK-6255) Python MLlib API missing items: Classification

[ https://issues.apache.org/jira/browse/SPARK-6255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376095#comment-14376095 ] Yanbo Liang commented on SPARK-6255: [~josephkb] Can you assign it to me? > Python ML

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376092#comment-14376092 ] Thomas Graves commented on SPARK-6469: -- Yeah so you should be able to set the spark.l

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376084#comment-14376084 ] Christophe PRÉAUD commented on SPARK-6469: -- The problem I have is that spark temp

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376065#comment-14376065 ] Thomas Graves commented on SPARK-6469: -- Note if its purely the documentation confused

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376061#comment-14376061 ] Thomas Graves commented on SPARK-6469: -- Are you saying they are not set on the driver

[jira] [Commented] (SPARK-3735) Sending the factor directly or AtA based on the cost in ALS

[ https://issues.apache.org/jira/browse/SPARK-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376046#comment-14376046 ] Debasish Das commented on SPARK-3735: - We might want to consider doing some of these t

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376039#comment-14376039 ] Christophe PRÉAUD commented on SPARK-6469: -- Not exactly, sorry if this was not cl

[jira] [Commented] (SPARK-4227) Document external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376022#comment-14376022 ] Thomas Graves commented on SPARK-4227: -- Looks like I had build issues. The instructio

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376017#comment-14376017 ] Sean Owen commented on SPARK-6469: -- Ah I get you, I think you have a point. So, you are r

[jira] [Closed] (SPARK-6436) io/netty missing from external shuffle service jars for yarn

[ https://issues.apache.org/jira/browse/SPARK-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-6436. Resolution: Invalid This is working for me. Sorry for the confusion, I had build environment issue

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376009#comment-14376009 ] Christophe PRÉAUD commented on SPARK-6469: -- Sorry if I'm saying something stupid,

[jira] [Commented] (SPARK-6255) Python MLlib API missing items: Classification

[ https://issues.apache.org/jira/browse/SPARK-6255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375994#comment-14375994 ] Apache Spark commented on SPARK-6255: - User 'yanboliang' has created a pull request fo

[jira] [Commented] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376001#comment-14376001 ] Sean Owen commented on SPARK-2167: -- (Thanks [~tgraves] for having a look at some of these

[jira] [Closed] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-2167. Resolution: Invalid I think this was mostly fixed when SPARK-3627 went in. If there are cases some

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375976#comment-14375976 ] Sean Owen commented on SPARK-6469: -- So, if {{YARN_LOCAL_DIRS}} is set, then {{isRunningIn

[jira] [Comment Edited] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375941#comment-14375941 ] Christophe PRÉAUD edited comment on SPARK-6469 at 3/23/15 2:15 PM: -

[jira] [Comment Edited] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375941#comment-14375941 ] Christophe PRÉAUD edited comment on SPARK-6469 at 3/23/15 2:16 PM: -

[jira] [Updated] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Attachment: TestYarnVars.scala Attached a simple application to check the value of the {{C

[jira] [Commented] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375929#comment-14375929 ] Nicholas Chammas commented on SPARK-2394: - Thank you for posting this information

[jira] [Created] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

Christophe PRÉAUD created SPARK-6469: Summary: Local directories configured for YARN are not used in yarn-client mode Key: SPARK-6469 URL: https://issues.apache.org/jira/browse/SPARK-6469 Project:

[jira] [Commented] (SPARK-6449) Driver OOM results in reported application result SUCCESS

[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375905#comment-14375905 ] Thomas Graves commented on SPARK-6449: -- [~rdub] Was there an exception in the log hig

[jira] [Commented] (SPARK-6468) Fix the race condition of subDirs in DiskBlockManager

[ https://issues.apache.org/jira/browse/SPARK-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375873#comment-14375873 ] Apache Spark commented on SPARK-6468: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-6468) Fix the race condition of subDirs in DiskBlockManager

Shixiong Zhu created SPARK-6468: --- Summary: Fix the race condition of subDirs in DiskBlockManager Key: SPARK-6468 URL: https://issues.apache.org/jira/browse/SPARK-6468 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis edited comment on SPARK-2394 at 3/23/15 11:38 AM: --

[jira] [Commented] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis commented on SPARK-2394: Just adding some more info here f

[jira] [Commented] (SPARK-5320) Joins on simple table created using select gives error

[ https://issues.apache.org/jira/browse/SPARK-5320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375748#comment-14375748 ] Yuri Saito commented on SPARK-5320: --- Thank you very much! I'm very grad! > Joins on sim

[jira] [Comment Edited] (SPARK-1702) Mesos executor won't start because of a ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375599#comment-14375599 ] Littlestar edited comment on SPARK-1702 at 3/23/15 11:00 AM: -

[jira] [Commented] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375707#comment-14375707 ] vijay commented on SPARK-6435: -- I tested this on Linux with the 1.3.0 release, works fine. A

[jira] [Resolved] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6397. --- Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by pull request

[jira] [Commented] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375680#comment-14375680 ] Cheng Lian commented on SPARK-6397: --- Hey [~smolav], after some discussion with [~waterma

[jira] [Updated] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Description: Virtual columns like GROUPING__ID should never be considered as missing input, and thus sho

[jira] [Created] (SPARK-6467) Override QueryPlan.missingInput when necessary and rely on it CheckAnalysis

Cheng Lian created SPARK-6467: - Summary: Override QueryPlan.missingInput when necessary and rely on it CheckAnalysis Key: SPARK-6467 URL: https://issues.apache.org/jira/browse/SPARK-6467 Project: Spark

[jira] [Updated] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Summary: Exclude virtual columns from QueryPlan.missingInput (was: Override QueryPlan.missingInput when

[jira] [Updated] (SPARK-6397) Override QueryPlan.missingInput when necessary and rely on CheckAnalysis

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Affects Version/s: 1.3.0 > Override QueryPlan.missingInput when necessary and rely on CheckAnalysis > --

[jira] [Updated] (SPARK-6397) Override QueryPlan.missingInput when necessary and rely on CheckAnalysis

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Assignee: Yadong Qi > Override QueryPlan.missingInput when necessary and rely on CheckAnalysis > ---

[jira] [Commented] (SPARK-6466) Remove unnecessary attributes when resolving GroupingSets

[ https://issues.apache.org/jira/browse/SPARK-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375657#comment-14375657 ] Apache Spark commented on SPARK-6466: - User 'viirya' has created a pull request for th

[jira] [Created] (SPARK-6466) Remove unnecessary attributes when resolving GroupingSets

Liang-Chi Hsieh created SPARK-6466: -- Summary: Remove unnecessary attributes when resolving GroupingSets Key: SPARK-6466 URL: https://issues.apache.org/jira/browse/SPARK-6466 Project: Spark I

[jira] [Issue Comment Deleted] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

Env.PATH in spark-defaults.conf is not pass to mesos > spark.executorEnv.PATH > spark.executorEnv.HADOOP_HOME > spark.executorEnv.JAVA_HOME > E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-b

[jira] [Comment Edited] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

Env.PATH in spark-defaults.conf is not pass to mesos > spark.executorEnv.PATH > spark.executorEnv.HADOOP_HOME > spark.executorEnv.JAVA_HOME > E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/s

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Attachment: screenshot-1.png > Add a new transformation of rdd named processCoalesce which was part

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the number

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the number

[jira] [Commented] (SPARK-6213) sql.catalyst.expressions.Expression is not friendly to java

[ https://issues.apache.org/jira/browse/SPARK-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375612#comment-14375612 ] Littlestar commented on SPARK-6213: --- may be change protected[sql] def selectFilters(filt

[jira] [Created] (SPARK-6465) GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor):

Earthson Lu created SPARK-6465: -- Summary: GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor): Key: SPARK-6465 URL: https://issues.apache.org/jira/browse/SPARK-6465

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the number

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375590#comment-14375590 ] iward edited comment on SPARK-3720 at 3/23/15 9:10 AM: --- hi,[~zhzhan]

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375590#comment-14375590 ] iward edited comment on SPARK-3720 at 3/23/15 9:05 AM: --- hi,[~zhzhan]

[jira] [Comment Edited] (SPARK-1702) Mesos executor won't start because of a ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375599#comment-14375599 ] Littlestar edited comment on SPARK-1702 at 3/23/15 9:05 AM: I

[jira] [Commented] (SPARK-1702) Mesos executor won't start because of a ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375599#comment-14375599 ] Littlestar commented on SPARK-1702: --- I met this on spak 1.3.0 + mesos 0.21.1 > Mesos ex

[jira] [Comment Edited] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

0635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks/20150323-133400-1214949568-5050-15440-0007/execu

[jira] [Created] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

SaintBacchus created SPARK-6464: --- Summary: Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd Key: SPARK-6464 URL: https://issues.apache.org/jira/brow

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375590#comment-14375590 ] iward commented on SPARK-3720: -- hi,Zhan Zhang , I have the same problem.And I just contact or

[jira] [Comment Edited] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

> fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks/20150323-133400-1214949568-5050-15440-0007/executors/20150323-100710-1214949568-5050-3453-S3/runs/915b40d

[jira] [Commented] (SPARK-1403) Spark on Mesos does not set Thread's context class loader

[ https://issues.apache.org/jira/browse/SPARK-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375583#comment-14375583 ] Littlestar commented on SPARK-1403: --- I want to reopen this bug, because I can reproduce

<    1   2   3   >