[jira] [Commented] (SPARK-3619) Upgrade to Mesos 0.21 to work around MESOS-1688

2015-01-09 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270730#comment-14270730 ] Jongyoul Lee commented on SPARK-3619: - [~tnachen] [~matei] I am finished some tests un

[jira] [Created] (SPARK-5170) fetch the correct max attempts

2015-01-09 Thread WangTaoTheTonic (JIRA)
WangTaoTheTonic created SPARK-5170: -- Summary: fetch the correct max attempts Key: SPARK-5170 URL: https://issues.apache.org/jira/browse/SPARK-5170 Project: Spark Issue Type: Bug Co

[jira] [Created] (SPARK-5169) fetch the correct max attempts

2015-01-09 Thread WangTaoTheTonic (JIRA)
WangTaoTheTonic created SPARK-5169: -- Summary: fetch the correct max attempts Key: SPARK-5169 URL: https://issues.apache.org/jira/browse/SPARK-5169 Project: Spark Issue Type: Bug Co

[jira] [Closed] (SPARK-5170) fetch the correct max attempts

2015-01-09 Thread WangTaoTheTonic (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic closed SPARK-5170. -- Resolution: Duplicate Duplicated submit as the server block issue. So close it. > fetch the cor

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-3019: Attachment: Pluggable RPC - draft 1.pdf > Pluggable block transfer (data plane communication) interf

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-3019: Attachment: (was: Pluggable RPC - draft 1.pdf) > Pluggable block transfer (data plane communicat

[jira] [Updated] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5124: Attachment: Pluggable RPC - draft 1.pdf > Standardize internal RPC interface > -

[jira] [Commented] (SPARK-5169) fetch the correct max attempts

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270743#comment-14270743 ] Apache Spark commented on SPARK-5169: - User 'WangTaoTheTonic' has created a pull reque

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270748#comment-14270748 ] Apache Spark commented on SPARK-5124: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-5171) Standalone cluster: masters scheduling independently

2015-01-09 Thread Roberto Vaquerizo Rodriguez (JIRA)
Roberto Vaquerizo Rodriguez created SPARK-5171: -- Summary: Standalone cluster: masters scheduling independently Key: SPARK-5171 URL: https://issues.apache.org/jira/browse/SPARK-5171 Project

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270771#comment-14270771 ] Reynold Xin commented on SPARK-5124: Thanks for doing this. I haven't looked through a

[jira] [Comment Edited] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270771#comment-14270771 ] Reynold Xin edited comment on SPARK-5124 at 1/9/15 8:49 AM: Th

[jira] [Updated] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5124: Attachment: (was: Pluggable RPC - draft 1.pdf) > Standardize internal RPC interface > --

[jira] [Updated] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5124: Attachment: Pluggable RPC - draft 1.pdf > Standardize internal RPC interface > -

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270809#comment-14270809 ] Shixiong Zhu commented on SPARK-5124: - {quote} 1. For DAGScheduler, we are probably OK

[jira] [Comment Edited] (SPARK-5022) Change VectorUDT to object

2015-01-09 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270817#comment-14270817 ] Manoj Kumar edited comment on SPARK-5022 at 1/9/15 9:50 AM: [~

[jira] [Commented] (SPARK-5022) Change VectorUDT to object

2015-01-09 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270817#comment-14270817 ] Manoj Kumar commented on SPARK-5022: @josephkb I want to have a go at this one. Should

[jira] [Comment Edited] (SPARK-5119) java.lang.ArrayIndexOutOfBoundsException on trying to train decision tree model

2015-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270825#comment-14270825 ] Sean Owen edited comment on SPARK-5119 at 1/9/15 10:00 AM: --- Yes,

[jira] [Commented] (SPARK-5119) java.lang.ArrayIndexOutOfBoundsException on trying to train decision tree model

2015-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270825#comment-14270825 ] Sean Owen commented on SPARK-5119: -- Yes, the input must contain categories that are posit

[jira] [Created] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-5172: --- Summary: spark-examples-***.jar shades a wrong Hadoop distribution Key: SPARK-5172 URL: https://issues.apache.org/jira/browse/SPARK-5172 Project: Spark Issue T

[jira] [Updated] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5172: Component/s: Build Priority: Minor (was: Major) > spark-examples-***.jar shades a wrong Hado

[jira] [Updated] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5172: Description: Steps to check it: 1. Download "spark-1.2.0-bin-hadoop2.4.tgz" from http://www.apach

[jira] [Updated] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5172: Description: Steps to check it: 1. Download "spark-1.2.0-bin-hadoop2.4.tgz" from http://www.apach

[jira] [Commented] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270853#comment-14270853 ] Shixiong Zhu commented on SPARK-5172: - If someone puts the spark-examples.jar in the c

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-01-09 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270869#comment-14270869 ] yuhao yang commented on SPARK-1405: --- Great design doc and solid proposal. I noticed th

[jira] [Created] (SPARK-5173) support python running on yarn cluster mode

2015-01-09 Thread Lianhui Wang (JIRA)
Lianhui Wang created SPARK-5173: --- Summary: support python running on yarn cluster mode Key: SPARK-5173 URL: https://issues.apache.org/jira/browse/SPARK-5173 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-5173) support python running on yarn cluster mode

2015-01-09 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-5173: Description: now when we run python application on yarn cluster mode through spark-submit, spark-sub

[jira] [Updated] (SPARK-5173) support python application running on yarn cluster mode

2015-01-09 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-5173: Summary: support python application running on yarn cluster mode (was: support python running on ya

[jira] [Commented] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270946#comment-14270946 ] Sean Owen commented on SPARK-5172: -- It's not quite that -- it's that the HBase examples a

[jira] [Created] (SPARK-5174) Missing Document for starting multiple workers/supervisors in actor-based receiver

2015-01-09 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-5174: -- Summary: Missing Document for starting multiple workers/supervisors in actor-based receiver Key: SPARK-5174 URL: https://issues.apache.org/jira/browse/SPARK-5174 Project: Spark

[jira] [Commented] (SPARK-5174) Missing Document for starting multiple workers/supervisors in actor-based receiver

2015-01-09 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270955#comment-14270955 ] Nan Zhu commented on SPARK-5174: can anyone assign this to me, I'd submit a PR to fix this

[jira] [Commented] (SPARK-5175) bug in updating counters when starting multiple workers/supervisors in actor-based receiver

2015-01-09 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270957#comment-14270957 ] Nan Zhu commented on SPARK-5175: can anyone assign this to me, I'd like to submit a PR on

[jira] [Created] (SPARK-5175) bug in updating counters when starting multiple workers/supervisors in actor-based receiver

2015-01-09 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-5175: -- Summary: bug in updating counters when starting multiple workers/supervisors in actor-based receiver Key: SPARK-5175 URL: https://issues.apache.org/jira/browse/SPARK-5175 Project

[jira] [Closed] (SPARK-4673) Optimizing limit using coalesce

2015-01-09 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei closed SPARK-4673. -- Resolution: Fixed since coalesce (1) will lead to run with a single thread, not always speed up limit so close

[jira] [Commented] (SPARK-5119) java.lang.ArrayIndexOutOfBoundsException on trying to train decision tree model

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270959#comment-14270959 ] Apache Spark commented on SPARK-5119: - User 'Lewuathe' has created a pull request for

[jira] [Commented] (SPARK-4572) [SQL] spark-sql exits while encountered an error

2015-01-09 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270960#comment-14270960 ] wangfei commented on SPARK-4572: which version you get this error? it should be fixed alre

[jira] [Commented] (SPARK-5153) flaky test of "Reliable Kafka input stream with multiple topics"

2015-01-09 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270964#comment-14270964 ] Nan Zhu commented on SPARK-5153: [~saisai_shao], yeah, I got up early to check whether the

[jira] [Issue Comment Deleted] (SPARK-4865) Include temporary tables in SHOW TABLES

2015-01-09 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated SPARK-4865: --- Comment: was deleted (was: Since I've worked on the related task SPARK-3299, if no one is currently w

[jira] [Commented] (SPARK-4574) Adding support for defining schema in foreign DDL commands.

2015-01-09 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270967#comment-14270967 ] wangfei commented on SPARK-4574: [~pwendell] get it, thanks. > Adding support for definin

[jira] [Commented] (SPARK-4865) Include temporary tables in SHOW TABLES

2015-01-09 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270969#comment-14270969 ] Bill Bejeck commented on SPARK-4865: I'm done with SPARK-3299. If no one is currently

[jira] [Commented] (SPARK-1442) Add Window function support

2015-01-09 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270973#comment-14270973 ] wangfei commented on SPARK-1442: why the two PR both closed? > Add Window function suppor

[jira] [Commented] (SPARK-5173) support python application running on yarn cluster mode

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270986#comment-14270986 ] Apache Spark commented on SPARK-5173: - User 'lianhuiwang' has created a pull request f

[jira] [Commented] (SPARK-5138) pyspark unable to infer schema of namedtuple

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271042#comment-14271042 ] Apache Spark commented on SPARK-5138: - User 'mulby' has created a pull request for thi

[jira] [Resolved] (SPARK-5169) fetch the correct max attempts

2015-01-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-5169. -- Resolution: Fixed Fix Version/s: 1.3.0 Assignee: WangTaoTheTonic > fetch the cor

[jira] [Commented] (SPARK-5153) flaky test of "Reliable Kafka input stream with multiple topics"

2015-01-09 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271047#comment-14271047 ] Saisai Shao commented on SPARK-5153: Yes, I agree with you that increasing the timeout

[jira] [Created] (SPARK-5176) Thrift server fails with confusing error message when deploy-mode is cluster

2015-01-09 Thread Tom Panning (JIRA)
Tom Panning created SPARK-5176: -- Summary: Thrift server fails with confusing error message when deploy-mode is cluster Key: SPARK-5176 URL: https://issues.apache.org/jira/browse/SPARK-5176 Project: Spark

[jira] [Created] (SPARK-5177) Add environment variables in dev/run-tests to enable Hive 0.12 and Scala 2.11 Jenkins builder

2015-01-09 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-5177: - Summary: Add environment variables in dev/run-tests to enable Hive 0.12 and Scala 2.11 Jenkins builder Key: SPARK-5177 URL: https://issues.apache.org/jira/browse/SPARK-5177

[jira] [Commented] (SPARK-5177) Add environment variables in dev/run-tests to enable Hive 0.12 and Scala 2.11 Jenkins builder

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271490#comment-14271490 ] Apache Spark commented on SPARK-5177: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

2015-01-09 Thread Gerard Maas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271508#comment-14271508 ] Gerard Maas commented on SPARK-5095: If I understand correctly, setting `spark.mesos.c

[jira] [Commented] (SPARK-5022) Change VectorUDT to object

2015-01-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271573#comment-14271573 ] Joseph K. Bradley commented on SPARK-5022: -- A major refactor of SQL data types is

[jira] [Resolved] (SPARK-5136) Improve documentation around setting up Spark IntelliJ project

2015-01-09 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5136. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 > Improve documentati

[jira] [Updated] (SPARK-5136) Improve documentation around setting up Spark IntelliJ project

2015-01-09 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5136: --- Assignee: Sean Owen > Improve documentation around setting up Spark IntelliJ project > ---

[jira] [Updated] (SPARK-5073) "spark.storage.memoryMapThreshold" has two default value

2015-01-09 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5073: --- Summary: "spark.storage.memoryMapThreshold" has two default value (was: "spark.storage.memory

[jira] [Resolved] (SPARK-1143) ClusterSchedulerSuite (soon to be TaskSchedulerImplSuite) does not actually test the ClusterScheduler/TaskSchedulerImpl

2015-01-09 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1143. Resolution: Fixed Assignee: Kay Ousterhout (was: Nan Zhu) > ClusterSchedulerSuite (so

[jira] [Updated] (SPARK-2101) Python unit tests fail on Python 2.6 because of lack of unittest.skipIf()

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2101: - Fix Version/s: 0.9.3 > Python unit tests fail on Python 2.6 because of lack of unittest.skipIf() > ---

[jira] [Updated] (SPARK-2954) PySpark MLlib serialization tests fail on Python 2.6

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2954: - Fix Version/s: 0.9.3 > PySpark MLlib serialization tests fail on Python 2.6 >

[jira] [Updated] (SPARK-2948) PySpark doesn't work on Python 2.6

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2948: - Fix Version/s: 0.9.3 > PySpark doesn't work on Python 2.6 > -- > >

[jira] [Resolved] (SPARK-5163) Load properties from configuration file for example spark-defaults.conf when creating SparkConf object

2015-01-09 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5163. Resolution: Won't Fix I'd prefer not to accept this patch for now - the spark-defaults.conf

[jira] [Updated] (SPARK-2910) Test with Python 2.6 on Jenkins

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2910: - Fix Version/s: 0.9.2 > Test with Python 2.6 on Jenkins > --- > >

[jira] [Updated] (SPARK-5073) "spark.storage.memoryMapThreshold" has two default values

2015-01-09 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5073: --- Summary: "spark.storage.memoryMapThreshold" has two default values (was: "spark.storage.memor

[jira] [Commented] (SPARK-5022) Change VectorUDT to object

2015-01-09 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271608#comment-14271608 ] Manoj Kumar commented on SPARK-5022: Hi, Thanks for the reply. I have worked on implem

[jira] [Commented] (SPARK-5152) Let metrics.properties file take an hdfs:// path

2015-01-09 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271625#comment-14271625 ] Ryan Williams commented on SPARK-5152: -- so I've been fumbling my way around the "Metr

[jira] [Updated] (SPARK-3490) Alleviate port collisions during tests

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3490: - Fix Version/s: 0.9.3 > Alleviate port collisions during tests > -- > >

[jira] [Updated] (SPARK-5145) Add BLAS.dsyr and use it in GaussianMixtureEM

2015-01-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5145: - Assignee: Liang-Chi Hsieh > Add BLAS.dsyr and use it in GaussianMixtureEM > --

[jira] [Resolved] (SPARK-5145) Add BLAS.dsyr and use it in GaussianMixtureEM

2015-01-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5145. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3949 [https://githu

[jira] [Commented] (SPARK-5015) GaussianMixtureEM should take random seed parameter

2015-01-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271656#comment-14271656 ] Joseph K. Bradley commented on SPARK-5015: -- I'm upping this to critical since it

[jira] [Assigned] (SPARK-5015) GaussianMixtureEM should take random seed parameter

2015-01-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-5015: Assignee: Joseph K. Bradley > GaussianMixtureEM should take random seed parameter >

[jira] [Updated] (SPARK-5015) GaussianMixtureEM should take random seed parameter

2015-01-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5015: - Priority: Critical (was: Minor) > GaussianMixtureEM should take random seed parameter > -

[jira] [Commented] (SPARK-5015) GaussianMixtureEM should take random seed parameter

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271682#comment-14271682 ] Apache Spark commented on SPARK-5015: - User 'jkbradley' has created a pull request for

[jira] [Resolved] (SPARK-3619) Upgrade to Mesos 0.21 to work around MESOS-1688

2015-01-09 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3619. -- Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Jongyoul Lee (was: Timothy Chen

[jira] [Commented] (SPARK-5022) Change VectorUDT to object

2015-01-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271696#comment-14271696 ] Joseph K. Bradley commented on SPARK-5022: -- There are lots of items out there, bu

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-01-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271700#comment-14271700 ] Joseph K. Bradley commented on SPARK-1405: -- That's great to hear that online vari

[jira] [Commented] (SPARK-5075) Memory Leak when repartitioning SchemaRDD or running queries in general

2015-01-09 Thread Brad Willard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271706#comment-14271706 ] Brad Willard commented on SPARK-5075: - I wanted to add that this is greatly exacerbate

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-01-09 Thread Pedro Rodriguez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271717#comment-14271717 ] Pedro Rodriguez commented on SPARK-1405: Second on nice design doc and proposal. I

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-01-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271725#comment-14271725 ] Joseph K. Bradley commented on SPARK-1405: -- Pedro, that sounds great. I wouldn't

[jira] [Updated] (SPARK-5041) hive-exec jar should be generated with JDK 6

2015-01-09 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-5041: -- Labels: maven (was: ) > hive-exec jar should be generated with JDK 6 >

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-01-09 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271788#comment-14271788 ] Josh Rosen commented on SPARK-4879: --- Hey Zach, >From my last round of attempts (maybe a

[jira] [Commented] (SPARK-5053) Test maintenance branches on Jenkins using SBT

2015-01-09 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271793#comment-14271793 ] Josh Rosen commented on SPARK-5053: --- It doesn't look like the "disable web UI in tests"

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4006: - Target Version/s: 1.2.0, 1.1.1, 0.9.3, 1.0.3 (was: 1.1.1, 1.2.0, 1.0.3) > Spark Driver crashes whenever a

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4006: - Fix Version/s: 0.9.3 > Spark Driver crashes whenever an Executor is registered twice > ---

[jira] [Commented] (SPARK-5053) Test maintenance branches on Jenkins using SBT

2015-01-09 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271805#comment-14271805 ] Josh Rosen commented on SPARK-5053: --- Ah, looks like Andrew has a PR to backport the webU

[jira] [Created] (SPARK-5178) Integrate Python unit tests into Jenkins

2015-01-09 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-5178: --- Summary: Integrate Python unit tests into Jenkins Key: SPARK-5178 URL: https://issues.apache.org/jira/browse/SPARK-5178 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-5162) Python yarn-cluster mode

2015-01-09 Thread Dana Klassen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271822#comment-14271822 ] Dana Klassen commented on SPARK-5162: - this is duplicate of https://issues.apache.org/

[jira] [Commented] (SPARK-5178) Integrate Python unit tests into Jenkins

2015-01-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271858#comment-14271858 ] Nicholas Chammas commented on SPARK-5178: - Relevant links: * http://stackoverflow.

[jira] [Resolved] (SPARK-5015) GaussianMixtureEM should take random seed parameter

2015-01-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5015. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3981 [https://githu

[jira] [Commented] (SPARK-5174) Missing Document for starting multiple workers/supervisors in actor-based receiver

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271868#comment-14271868 ] Apache Spark commented on SPARK-5174: - User 'CodingCat' has created a pull request for

[jira] [Commented] (SPARK-5175) bug in updating counters when starting multiple workers/supervisors in actor-based receiver

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271869#comment-14271869 ] Apache Spark commented on SPARK-5175: - User 'CodingCat' has created a pull request for

[jira] [Created] (SPARK-5179) Spark UI history job duration is wrong

2015-01-09 Thread Olivier Toupin (JIRA)
Olivier Toupin created SPARK-5179: - Summary: Spark UI history job duration is wrong Key: SPARK-5179 URL: https://issues.apache.org/jira/browse/SPARK-5179 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-5180) Data source API improvement

2015-01-09 Thread Yin Huai (JIRA)
Yin Huai created SPARK-5180: --- Summary: Data source API improvement Key: SPARK-5180 URL: https://issues.apache.org/jira/browse/SPARK-5180 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-4574) Adding support for defining schema in foreign DDL commands.

2015-01-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4574: Issue Type: Sub-task (was: New Feature) Parent: SPARK-5180 > Adding support for defining schema in

[jira] [Updated] (SPARK-4912) Persistent data source tables

2015-01-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4912: Issue Type: Sub-task (was: Improvement) Parent: SPARK-5180 > Persistent data source tables > --

[jira] [Updated] (SPARK-4574) Adding support for defining schema in foreign DDL commands.

2015-01-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4574: Priority: Blocker (was: Major) > Adding support for defining schema in foreign DDL commands. >

[jira] [Created] (SPARK-5181) inaccurate log when WAL is disabled

2015-01-09 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-5181: -- Summary: inaccurate log when WAL is disabled Key: SPARK-5181 URL: https://issues.apache.org/jira/browse/SPARK-5181 Project: Spark Issue Type: Bug Components: S

[jira] [Created] (SPARK-5182) Partitioning support for tables created by the data source API

2015-01-09 Thread Yin Huai (JIRA)
Yin Huai created SPARK-5182: --- Summary: Partitioning support for tables created by the data source API Key: SPARK-5182 URL: https://issues.apache.org/jira/browse/SPARK-5182 Project: Spark Issue Typ

[jira] [Created] (SPARK-5183) Document data source API

2015-01-09 Thread Yin Huai (JIRA)
Yin Huai created SPARK-5183: --- Summary: Document data source API Key: SPARK-5183 URL: https://issues.apache.org/jira/browse/SPARK-5183 Project: Spark Issue Type: Sub-task Components: Docum

[jira] [Commented] (SPARK-5181) inaccurate log when WAL is disabled

2015-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271887#comment-14271887 ] Apache Spark commented on SPARK-5181: - User 'CodingCat' has created a pull request for

[jira] [Closed] (SPARK-1953) yarn client mode Application Master memory size is same as driver memory size

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1953. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: WangTaoTheTonic Target Version/

[jira] [Created] (SPARK-5184) Improve the performance of metadata operations

2015-01-09 Thread Yin Huai (JIRA)
Yin Huai created SPARK-5184: --- Summary: Improve the performance of metadata operations Key: SPARK-5184 URL: https://issues.apache.org/jira/browse/SPARK-5184 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-5185) pyspark --jars does not add classes to driver class path

2015-01-09 Thread Uri Laserson (JIRA)
Uri Laserson created SPARK-5185: --- Summary: pyspark --jars does not add classes to driver class path Key: SPARK-5185 URL: https://issues.apache.org/jira/browse/SPARK-5185 Project: Spark Issue Ty

[jira] [Closed] (SPARK-4737) Prevent serialization errors from ever crashing the DAG scheduler

2015-01-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4737. Resolution: Fixed Fix Version/s: 1.3.0 > Prevent serialization errors from ever crashing the DAG sche

  1   2   >