[jira] [Commented] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

TH in spark-defaults.conf is not pass to mesos > spark.executorEnv.PATH > spark.executorEnv.HADOOP_HOME > spark.executorEnv.JAVA_HOME > E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.

[jira] [Commented] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks/20150323-133400-1214949568-5050-15440-0007/executors/20150323-100710-1214949568-5050-3453-S3/runs/915b40d8-f7c4-428a-9df8-ac9804c6cd21/spark-1.3.0-bin-2.4.0.t

[jira] [Commented] (SPARK-6397) Override QueryPlan.missingInput when necessary and rely on CheckAnalysis

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375477#comment-14375477 ] Apache Spark commented on SPARK-6397: - User 'watermen' has created a pull request for

[jira] [Updated] (SPARK-6456) Spark Sql throwing exception on large partitioned data

[ https://issues.apache.org/jira/browse/SPARK-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6456: -- Description: Spark connects with Hive Metastore. I am able to run simple queries like show table and se

[jira] [Commented] (SPARK-6456) Spark Sql throwing exception on large partitioned data

[ https://issues.apache.org/jira/browse/SPARK-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375481#comment-14375481 ] Cheng Lian commented on SPARK-6456: --- How many partitions are there? > Spark Sql throwin

[jira] [Comment Edited] (SPARK-6456) Spark Sql throwing exception on large partitioned data

[ https://issues.apache.org/jira/browse/SPARK-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375481#comment-14375481 ] Cheng Lian edited comment on SPARK-6456 at 3/23/15 7:21 AM: Ho

[jira] [Updated] (SPARK-6443) Could not submit app in standalone cluster mode when HA is enabled

[ https://issues.apache.org/jira/browse/SPARK-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Wang updated SPARK-6443: Description: After digging some codes, I found user could not submit app in standalone cluster mode when HA

[jira] [Created] (SPARK-6462) UpdateStateByKey should allow inner join of new with old keys

Andre Schumacher created SPARK-6462: --- Summary: UpdateStateByKey should allow inner join of new with old keys Key: SPARK-6462 URL: https://issues.apache.org/jira/browse/SPARK-6462 Project: Spark

[jira] [Updated] (SPARK-6443) Could not submit app in standalone cluster mode when HA is enabled

[ https://issues.apache.org/jira/browse/SPARK-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Wang updated SPARK-6443: Description: After digging some codes, I found user could not submit app in standalone cluster mode when HA

[jira] [Commented] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks/20150323-133400-1214949568-5050-15440-0007/executors/20150323-10071

[jira] [Created] (SPARK-6463) [SPARK][SQL] AttributeSet.equal should compare size

June created SPARK-6463: --- Summary: [SPARK][SQL] AttributeSet.equal should compare size Key: SPARK-6463 URL: https://issues.apache.org/jira/browse/SPARK-6463 Project: Spark Issue Type: Bug Com

[jira] [Commented] (SPARK-1480) Choose classloader consistently inside of Spark codebase

21.1 100%.. I0323 16:32:18.933440 14504 fetcher.cpp:64] Extracted resource '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S4/frameworks/20150323-152848-1214949568-5050-21134-0009/executors/20150323-100710-1214949568-5050-3453-S4/runs/3d8f22f5-7fed-44ed-b5f9-98a219133911/sp

[jira] [Updated] (SPARK-6463) AttributeSet.equal should compare size

[ https://issues.apache.org/jira/browse/SPARK-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] June updated SPARK-6463: Summary: AttributeSet.equal should compare size (was: [SPARK][SQL] AttributeSet.equal should compare size) > Attri

[jira] [Commented] (SPARK-1480) Choose classloader consistently inside of Spark codebase

[ https://issues.apache.org/jira/browse/SPARK-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375576#comment-14375576 ] Littlestar commented on SPARK-1480: --- same as https://issues.apache.org/jira/browse/SPARK

[jira] [Comment Edited] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

.executorEnv.PATH > spark.executorEnv.HADOOP_HOME > spark.executorEnv.JAVA_HOME > E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/

[jira] [Commented] (SPARK-6463) AttributeSet.equal should compare size

[ https://issues.apache.org/jira/browse/SPARK-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375581#comment-14375581 ] Apache Spark commented on SPARK-6463: - User 'sisihj' has created a pull request for th

[jira] [Commented] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

ecutorEnv.HADOOP_HOME > spark.executorEnv.JAVA_HOME > E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/work_dir/slaves/20150323-10

[jira] [Commented] (SPARK-1403) Spark on Mesos does not set Thread's context class loader

[ https://issues.apache.org/jira/browse/SPARK-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375583#comment-14375583 ] Littlestar commented on SPARK-1403: --- I want to reopen this bug, because I can reproduce

[jira] [Comment Edited] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

> fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks/20150323-133400-1214949568-5050-15440-0007/executors/20150323-100710-1214949568-5050-3453-S3/runs/915b40d

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375590#comment-14375590 ] iward commented on SPARK-3720: -- hi,Zhan Zhang , I have the same problem.And I just contact or

[jira] [Created] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

SaintBacchus created SPARK-6464: --- Summary: Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd Key: SPARK-6464 URL: https://issues.apache.org/jira/brow

[jira] [Comment Edited] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

0635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' > '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks/20150323-133400-1214949568-5050-15440-0007/execu

[jira] [Commented] (SPARK-1702) Mesos executor won't start because of a ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375599#comment-14375599 ] Littlestar commented on SPARK-1702: --- I met this on spak 1.3.0 + mesos 0.21.1 > Mesos ex

[jira] [Comment Edited] (SPARK-1702) Mesos executor won't start because of a ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375599#comment-14375599 ] Littlestar edited comment on SPARK-1702 at 3/23/15 9:05 AM: I

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375590#comment-14375590 ] iward edited comment on SPARK-3720 at 3/23/15 9:05 AM: --- hi,[~zhzhan]

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375590#comment-14375590 ] iward edited comment on SPARK-3720 at 3/23/15 9:10 AM: --- hi,[~zhzhan]

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the number

[jira] [Created] (SPARK-6465) GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor):

Earthson Lu created SPARK-6465: -- Summary: GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor): Key: SPARK-6465 URL: https://issues.apache.org/jira/browse/SPARK-6465

[jira] [Commented] (SPARK-6213) sql.catalyst.expressions.Expression is not friendly to java

[ https://issues.apache.org/jira/browse/SPARK-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375612#comment-14375612 ] Littlestar commented on SPARK-6213: --- may be change protected[sql] def selectFilters(filt

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the number

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the number

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Attachment: screenshot-1.png > Add a new transformation of rdd named processCoalesce which was part

[jira] [Comment Edited] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

Env.PATH in spark-defaults.conf is not pass to mesos > spark.executorEnv.PATH > spark.executorEnv.HADOOP_HOME > spark.executorEnv.JAVA_HOME > E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/s

[jira] [Issue Comment Deleted] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

Env.PATH in spark-defaults.conf is not pass to mesos > spark.executorEnv.PATH > spark.executorEnv.HADOOP_HOME > spark.executorEnv.JAVA_HOME > E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop > fs -copyToLocal > 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-b

[jira] [Created] (SPARK-6466) Remove unnecessary attributes when resolving GroupingSets

Liang-Chi Hsieh created SPARK-6466: -- Summary: Remove unnecessary attributes when resolving GroupingSets Key: SPARK-6466 URL: https://issues.apache.org/jira/browse/SPARK-6466 Project: Spark I

[jira] [Commented] (SPARK-6466) Remove unnecessary attributes when resolving GroupingSets

[ https://issues.apache.org/jira/browse/SPARK-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375657#comment-14375657 ] Apache Spark commented on SPARK-6466: - User 'viirya' has created a pull request for th

[jira] [Updated] (SPARK-6397) Override QueryPlan.missingInput when necessary and rely on CheckAnalysis

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Assignee: Yadong Qi > Override QueryPlan.missingInput when necessary and rely on CheckAnalysis > ---

[jira] [Updated] (SPARK-6397) Override QueryPlan.missingInput when necessary and rely on CheckAnalysis

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Affects Version/s: 1.3.0 > Override QueryPlan.missingInput when necessary and rely on CheckAnalysis > --

[jira] [Updated] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Summary: Exclude virtual columns from QueryPlan.missingInput (was: Override QueryPlan.missingInput when

[jira] [Created] (SPARK-6467) Override QueryPlan.missingInput when necessary and rely on it CheckAnalysis

Cheng Lian created SPARK-6467: - Summary: Override QueryPlan.missingInput when necessary and rely on it CheckAnalysis Key: SPARK-6467 URL: https://issues.apache.org/jira/browse/SPARK-6467 Project: Spark

[jira] [Updated] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6397: -- Description: Virtual columns like GROUPING__ID should never be considered as missing input, and thus sho

[jira] [Commented] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375680#comment-14375680 ] Cheng Lian commented on SPARK-6397: --- Hey [~smolav], after some discussion with [~waterma

[jira] [Resolved] (SPARK-6397) Exclude virtual columns from QueryPlan.missingInput

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6397. --- Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by pull request

[jira] [Commented] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375707#comment-14375707 ] vijay commented on SPARK-6435: -- I tested this on Linux with the 1.3.0 release, works fine. A

[jira] [Comment Edited] (SPARK-1702) Mesos executor won't start because of a ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375599#comment-14375599 ] Littlestar edited comment on SPARK-1702 at 3/23/15 11:00 AM: -

[jira] [Commented] (SPARK-5320) Joins on simple table created using select gives error

[ https://issues.apache.org/jira/browse/SPARK-5320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375748#comment-14375748 ] Yuri Saito commented on SPARK-5320: --- Thank you very much! I'm very grad! > Joins on sim

[jira] [Commented] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis commented on SPARK-2394: Just adding some more info here f

[jira] [Comment Edited] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis edited comment on SPARK-2394 at 3/23/15 11:38 AM: --

[jira] [Created] (SPARK-6468) Fix the race condition of subDirs in DiskBlockManager

Shixiong Zhu created SPARK-6468: --- Summary: Fix the race condition of subDirs in DiskBlockManager Key: SPARK-6468 URL: https://issues.apache.org/jira/browse/SPARK-6468 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6468) Fix the race condition of subDirs in DiskBlockManager

[ https://issues.apache.org/jira/browse/SPARK-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375873#comment-14375873 ] Apache Spark commented on SPARK-6468: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-6449) Driver OOM results in reported application result SUCCESS

[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375905#comment-14375905 ] Thomas Graves commented on SPARK-6449: -- [~rdub] Was there an exception in the log hig

[jira] [Created] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

Christophe PRÉAUD created SPARK-6469: Summary: Local directories configured for YARN are not used in yarn-client mode Key: SPARK-6469 URL: https://issues.apache.org/jira/browse/SPARK-6469 Project:

[jira] [Commented] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375929#comment-14375929 ] Nicholas Chammas commented on SPARK-2394: - Thank you for posting this information

[jira] [Updated] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Attachment: TestYarnVars.scala Attached a simple application to check the value of the {{C

[jira] [Comment Edited] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375941#comment-14375941 ] Christophe PRÉAUD edited comment on SPARK-6469 at 3/23/15 2:16 PM: -

[jira] [Comment Edited] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375941#comment-14375941 ] Christophe PRÉAUD edited comment on SPARK-6469 at 3/23/15 2:15 PM: -

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375976#comment-14375976 ] Sean Owen commented on SPARK-6469: -- So, if {{YARN_LOCAL_DIRS}} is set, then {{isRunningIn

[jira] [Closed] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-2167. Resolution: Invalid I think this was mostly fixed when SPARK-3627 went in. If there are cases some

[jira] [Commented] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376001#comment-14376001 ] Sean Owen commented on SPARK-2167: -- (Thanks [~tgraves] for having a look at some of these

[jira] [Commented] (SPARK-6255) Python MLlib API missing items: Classification

[ https://issues.apache.org/jira/browse/SPARK-6255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375994#comment-14375994 ] Apache Spark commented on SPARK-6255: - User 'yanboliang' has created a pull request fo

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376009#comment-14376009 ] Christophe PRÉAUD commented on SPARK-6469: -- Sorry if I'm saying something stupid,

[jira] [Closed] (SPARK-6436) io/netty missing from external shuffle service jars for yarn

[ https://issues.apache.org/jira/browse/SPARK-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-6436. Resolution: Invalid This is working for me. Sorry for the confusion, I had build environment issue

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376017#comment-14376017 ] Sean Owen commented on SPARK-6469: -- Ah I get you, I think you have a point. So, you are r

[jira] [Commented] (SPARK-4227) Document external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376022#comment-14376022 ] Thomas Graves commented on SPARK-4227: -- Looks like I had build issues. The instructio

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376039#comment-14376039 ] Christophe PRÉAUD commented on SPARK-6469: -- Not exactly, sorry if this was not cl

[jira] [Commented] (SPARK-3735) Sending the factor directly or AtA based on the cost in ALS

[ https://issues.apache.org/jira/browse/SPARK-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376046#comment-14376046 ] Debasish Das commented on SPARK-3735: - We might want to consider doing some of these t

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376061#comment-14376061 ] Thomas Graves commented on SPARK-6469: -- Are you saying they are not set on the driver

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376065#comment-14376065 ] Thomas Graves commented on SPARK-6469: -- Note if its purely the documentation confused

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376084#comment-14376084 ] Christophe PRÉAUD commented on SPARK-6469: -- The problem I have is that spark temp

[jira] [Commented] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376092#comment-14376092 ] Thomas Graves commented on SPARK-6469: -- Yeah so you should be able to set the spark.l

[jira] [Commented] (SPARK-6255) Python MLlib API missing items: Classification

[ https://issues.apache.org/jira/browse/SPARK-6255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376095#comment-14376095 ] Yanbo Liang commented on SPARK-6255: [~josephkb] Can you assign it to me? > Python ML

[jira] [Commented] (SPARK-6451) Support CombineSum in Code Gen

[ https://issues.apache.org/jira/browse/SPARK-6451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376088#comment-14376088 ] Apache Spark commented on SPARK-6451: - User 'gvramana' has created a pull request for

[jira] [Updated] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Issue Type: Documentation (was: Bug) > Local directories configured for YARN are not used

[jira] [Updated] (SPARK-6469) Local directories configured for YARN are not used in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Description: According to the [Spark YARN doc page|http://spark.apache.org/docs/latest/ru

[jira] [Updated] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Summary: The YARN driver in yarn-client mode will not use the local directories configured

[jira] [Updated] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-6469: - Description: According to the [Spark YARN doc page|http://spark.apache.org/docs/latest/ru

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376111#comment-14376111 ] Christophe PRÉAUD commented on SPARK-6469: -- I've changed the JIRA title, type and

[jira] [Updated] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-6469: - Component/s: (was: Spark Core) YARN > The YARN driver in yarn-client mode wil

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376119#comment-14376119 ] Thomas Graves commented on SPARK-6469: -- looks good, thanks. > The YARN driver in yar

[jira] [Created] (SPARK-6470) Allow Spark apps to put YARN node labels in their requests

Sandy Ryza created SPARK-6470: - Summary: Allow Spark apps to put YARN node labels in their requests Key: SPARK-6470 URL: https://issues.apache.org/jira/browse/SPARK-6470 Project: Spark Issue Type

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376147#comment-14376147 ] Marcelo Vanzin commented on SPARK-6229: --- [~adav] the problem of exposing the pipelin

[jira] [Closed] (SPARK-4746) integration tests should be separated from faster unit tests

[ https://issues.apache.org/jira/browse/SPARK-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid closed SPARK-4746. --- Resolution: Won't Fix looks like there isn't interest in this, closing to clean up jira > integration

[jira] [Updated] (SPARK-6308) VectorUDT is displayed as `vecto` in dtypes

[ https://issues.apache.org/jira/browse/SPARK-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6308: - Assignee: Manoj Kumar > VectorUDT is displayed as `vecto` in dtypes >

[jira] [Commented] (SPARK-5954) Add topByKey to pair RDDs

[ https://issues.apache.org/jira/browse/SPARK-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376215#comment-14376215 ] Xiangrui Meng commented on SPARK-5954: -- Note: We added topByKey in mllib.rdd.MLPairRD

[jira] [Commented] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376234#comment-14376234 ] Apache Spark commented on SPARK-6369: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-6470) Allow Spark apps to put YARN node labels in their requests

[ https://issues.apache.org/jira/browse/SPARK-6470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-6470: - Assignee: Sandy Ryza > Allow Spark apps to put YARN node labels in their requests > -

[jira] [Commented] (SPARK-4848) On a stand-alone cluster, several worker-specific variables are read only on the master

[ https://issues.apache.org/jira/browse/SPARK-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376239#comment-14376239 ] Apache Spark commented on SPARK-4848: - User 'nkronenfeld' has created a pull request f

[jira] [Created] (SPARK-6471) Metastoreschema should only be a subset of parquetSchema to support dropping of columns using replace columns

Yash Datta created SPARK-6471: - Summary: Metastoreschema should only be a subset of parquetSchema to support dropping of columns using replace columns Key: SPARK-6471 URL: https://issues.apache.org/jira/browse/SPARK-6

[jira] [Updated] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-6471: -- Summary: Metastore schema should only be a subset of parquet schema to support dropping of columns using

[jira] [Commented] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376286#comment-14376286 ] Yash Datta commented on SPARK-6471: --- https://github.com/apache/spark/pull/5141 > Metast

[jira] [Commented] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376311#comment-14376311 ] Apache Spark commented on SPARK-6471: - User 'saucam' has created a pull request for th

[jira] [Issue Comment Deleted] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-6471: -- Comment: was deleted (was: https://github.com/apache/spark/pull/5141) > Metastore schema should only be

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376329#comment-14376329 ] Manoj Kumar commented on SPARK-6192: [~mengxr] Sorry for spamming, but do you have any

[jira] [Commented] (SPARK-4830) Spark Streaming Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376335#comment-14376335 ] sam commented on SPARK-4830: I've been seeing a similar problem but just for a regular Spark j

[jira] [Commented] (SPARK-4086) Fold-style aggregation for VertexRDD

[ https://issues.apache.org/jira/browse/SPARK-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376341#comment-14376341 ] Apache Spark commented on SPARK-4086: - User 'brennonyork' has created a pull request f

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376361#comment-14376361 ] Zhan Zhang commented on SPARK-3720: --- [~iward] Since this jiar is duplicated to Spark-288

[jira] [Updated] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2331: - Priority: Minor (was: Major) Target Version/s: 2+ > SparkContext.emptyRDD should return RDD[T

[jira] [Reopened] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-2331: -- Reopening to catch this for version 2.x > SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] >

[jira] [Commented] (SPARK-6213) sql.catalyst.expressions.Expression is not friendly to java

[ https://issues.apache.org/jira/browse/SPARK-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376394#comment-14376394 ] Michael Armbrust commented on SPARK-6213: - {{PrunedFilteredScan}} is exactly that,

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376409#comment-14376409 ] Sean Owen commented on SPARK-6469: -- [~preaudc] would you like to open a small PR to add a

  1   2   3   >