[jira] [Commented] (SPARK-15390) Memory management issue in complex DataFrame join and filter

2016-10-05 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547955#comment-15547955 ] Iulian Dragos commented on SPARK-15390: --- Thanks [~davies]! > Memory management issue in complex

[jira] [Reopened] (SPARK-15390) Memory management issue in complex DataFrame join and filter

2016-09-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos reopened SPARK-15390: --- > Memory management issue in complex DataFrame join and filter >

[jira] [Commented] (SPARK-15390) Memory management issue in complex DataFrame join and filter

2016-09-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513206#comment-15513206 ] Iulian Dragos commented on SPARK-15390: --- I'm still seeing a similar stack trace with the 2.0

[jira] [Commented] (SPARK-17109) When we serialize UserDefinedGenerator to json, scala reflection throws an error

2016-08-17 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425082#comment-15425082 ] Iulian Dragos commented on SPARK-17109: --- So the problem is somewhere along the caller list :) There

[jira] [Commented] (SPARK-17109) When we serialize UserDefinedGenerator to json, scala reflection throws an error

2016-08-17 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424979#comment-15424979 ] Iulian Dragos commented on SPARK-17109: --- No, it seems strange. I suggest you print out the `tpe`

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-23 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159111#comment-15159111 ] Iulian Dragos commented on SPARK-13431: --- I think the only way is to revert the offending changes,

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157213#comment-15157213 ] Iulian Dragos commented on SPARK-13431: --- Sbt assembly is using a different library, I think it's

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157119#comment-15157119 ] Iulian Dragos commented on SPARK-13431: --- Probably the easiest fix is to break some grammar elements

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157044#comment-15157044 ] Iulian Dragos commented on SPARK-13431: --- Well, the class file format isn't embedding any names

[jira] [Comment Edited] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156902#comment-15156902 ] Iulian Dragos edited comment on SPARK-13431 at 2/22/16 1:29 PM: In my

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156902#comment-15156902 ] Iulian Dragos commented on SPARK-13431: --- In my experience the 'shade' plugin does a lot more than

[jira] [Updated] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-13431: -- Description: Cannot build the project when run the normal build commands: eg. {code} build/mvn

[jira] [Updated] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-13431: -- Description: Cannot build the project when run the normal build commands: eg. {code} build/mvn

[jira] [Commented] (SPARK-12430) Temporary folders do not get deleted after Task completes causing problems with disk space.

2016-02-05 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133887#comment-15133887 ] Iulian Dragos commented on SPARK-12430: --- The PR I linked to was just merged. Normally this should

[jira] [Commented] (SPARK-12430) Temporary folders do not get deleted after Task completes causing problems with disk space.

2016-02-04 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132341#comment-15132341 ] Iulian Dragos commented on SPARK-12430: --- There is a long discussion about this in the dynamic

[jira] [Commented] (SPARK-12430) Temporary folders do not get deleted after Task completes causing problems with disk space.

2016-02-04 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132640#comment-15132640 ] Iulian Dragos commented on SPARK-12430: --- Are you using 1.6? In that case, the blockmgr directory

[jira] [Closed] (SPARK-13159) External shuffle service broken w/ Mesos

2016-02-03 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos closed SPARK-13159. - Resolution: Duplicate > External shuffle service broken w/ Mesos >

[jira] [Created] (SPARK-13159) External shuffle service broken w/ Mesos

2016-02-03 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-13159: - Summary: External shuffle service broken w/ Mesos Key: SPARK-13159 URL: https://issues.apache.org/jira/browse/SPARK-13159 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12430) Temporary folders do not get deleted after Task completes causing problems with disk space.

2016-02-03 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130554#comment-15130554 ] Iulian Dragos commented on SPARK-12430: --- I guess the thinking was that Mesos would clean up those

[jira] [Updated] (SPARK-12423) Mesos executor home should not be resolved on the driver's file system

2016-02-02 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-12423: -- Fix Version/s: 2.0.0 > Mesos executor home should not be resolved on the driver's file system

[jira] [Resolved] (SPARK-12423) Mesos executor home should not be resolved on the driver's file system

2016-02-02 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos resolved SPARK-12423. --- Resolution: Fixed > Mesos executor home should not be resolved on the driver's file system >

[jira] [Commented] (SPARK-12423) Mesos executor home should not be resolved on the driver's file system

2016-02-02 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128036#comment-15128036 ] Iulian Dragos commented on SPARK-12423: --- Yes, closing this. > Mesos executor home should not be

[jira] [Created] (SPARK-13086) Spark Shell for 2.11 does not allow loading files via '-i'

2016-01-29 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-13086: - Summary: Spark Shell for 2.11 does not allow loading files via '-i' Key: SPARK-13086 URL: https://issues.apache.org/jira/browse/SPARK-13086 Project: Spark

[jira] [Created] (SPARK-12979) Paths are resolved relative to the local file system

2016-01-25 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-12979: - Summary: Paths are resolved relative to the local file system Key: SPARK-12979 URL: https://issues.apache.org/jira/browse/SPARK-12979 Project: Spark Issue

[jira] [Commented] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

2016-01-14 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098355#comment-15098355 ] Iulian Dragos commented on SPARK-11857: --- Sure, but that seems to be done already... by you:

[jira] [Comment Edited] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

2016-01-13 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15095975#comment-15095975 ] Iulian Dragos edited comment on SPARK-11857 at 1/13/16 10:35 AM: - There's

[jira] [Commented] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

2016-01-13 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15095975#comment-15095975 ] Iulian Dragos commented on SPARK-11857: --- There's one pending issue in the email thread, regarding a

[jira] [Created] (SPARK-12423) Mesos executor home should not be resolved on the driver's file system

2015-12-18 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-12423: - Summary: Mesos executor home should not be resolved on the driver's file system Key: SPARK-12423 URL: https://issues.apache.org/jira/browse/SPARK-12423 Project:

[jira] [Commented] (SPARK-12345) Mesos cluster mode is broken

2015-12-16 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060236#comment-15060236 ] Iulian Dragos commented on SPARK-12345: --- I'd prefer filtering it at at the submit side, if

[jira] [Commented] (SPARK-12345) Mesos cluster mode is broken

2015-12-16 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059717#comment-15059717 ] Iulian Dragos commented on SPARK-12345: --- There's isn't any {{SPARK_HOME}} set on any of the Mesos

[jira] [Commented] (SPARK-12345) Mesos cluster mode is broken

2015-12-15 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15058873#comment-15058873 ] Iulian Dragos commented on SPARK-12345: --- [~skonto] pointed out this commit:

[jira] [Updated] (SPARK-12345) Mesos cluster mode is broken

2015-12-15 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-12345: -- Description: The same setup worked in 1.5.2 but is now failing for 1.6.0-RC2. The driver is

[jira] [Created] (SPARK-12265) Spark calls System.exit inside driver instead of throwing exception

2015-12-10 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-12265: - Summary: Spark calls System.exit inside driver instead of throwing exception Key: SPARK-12265 URL: https://issues.apache.org/jira/browse/SPARK-12265 Project: Spark

[jira] [Created] (SPARK-12202) Pass additional Scala REPL options to the underlying REPL

2015-12-08 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-12202: - Summary: Pass additional Scala REPL options to the underlying REPL Key: SPARK-12202 URL: https://issues.apache.org/jira/browse/SPARK-12202 Project: Spark

[jira] [Updated] (SPARK-12202) Pass additional Scala REPL options to the underlying REPL (2.11)

2015-12-08 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-12202: -- Description: Sometimes it is useful to be able to pass Scala options to the underlying Spark

[jira] [Commented] (SPARK-7286) Precedence of operator not behaving properly

2015-11-20 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015418#comment-15015418 ] Iulian Dragos commented on SPARK-7286: -- I agree with Jakob's analysis. For reference, Slick uses

[jira] [Commented] (SPARK-11327) spark-dispatcher doesn't pass along some spark properties

2015-11-18 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011339#comment-15011339 ] Iulian Dragos commented on SPARK-11327: --- [~abraithwaite] it's a valid question, but it's

[jira] [Commented] (SPARK-11327) spark-dispatcher doesn't pass along some spark properties

2015-11-04 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989280#comment-14989280 ] Iulian Dragos commented on SPARK-11327: --- I think this is related to (possibly a duplicate of)

[jira] [Commented] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-26 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974480#comment-14974480 ] Iulian Dragos commented on SPARK-10986: --- Digging a bit deeper. The problem is that the context

[jira] [Comment Edited] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-26 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974480#comment-14974480 ] Iulian Dragos edited comment on SPARK-10986 at 10/26/15 4:17 PM: - Digging

[jira] [Commented] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-26 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974673#comment-14974673 ] Iulian Dragos commented on SPARK-10986: --- [~kaysoky] can you try my PR and see if it solves it for

[jira] [Commented] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-26 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974777#comment-14974777 ] Iulian Dragos commented on SPARK-10986: --- Great! Please leave a note on the PR as well.. It's been

[jira] [Updated] (SPARK-11280) Mesos cluster deployment using only one node

2015-10-23 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-11280: -- Attachment: Screen Shot 2015-10-23 at 11.37.43.png > Mesos cluster deployment using only one

[jira] [Created] (SPARK-11280) Mesos cluster deployment using only one node

2015-10-23 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-11280: - Summary: Mesos cluster deployment using only one node Key: SPARK-11280 URL: https://issues.apache.org/jira/browse/SPARK-11280 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10975) Shuffle files left behind on Mesos without dynamic allocation

2015-10-23 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971268#comment-14971268 ] Iulian Dragos commented on SPARK-10975: --- No, it's not a duplicate, but fixed by the same PR :) >

[jira] [Updated] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-09 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-10986: -- Priority: Blocker (was: Major) > ClassNotFoundException when running on Client mode, with a

[jira] [Commented] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-09 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950507#comment-14950507 ] Iulian Dragos commented on SPARK-10986: --- I can confirm this issue. It only happens in *fine-grained

[jira] [Updated] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-09 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-10986: -- Description: When running an example task on a Mesos cluster (local master, local agent), any

[jira] [Created] (SPARK-10975) Shuffle files left behind on Mesos without dynamic allocation

2015-10-07 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-10975: - Summary: Shuffle files left behind on Mesos without dynamic allocation Key: SPARK-10975 URL: https://issues.apache.org/jira/browse/SPARK-10975 Project: Spark

[jira] [Commented] (SPARK-7398) Add back-pressure to Spark Streaming (umbrella JIRA)

2015-10-01 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940184#comment-14940184 ] Iulian Dragos commented on SPARK-7398: -- Hey, except the last point, everything is available in 1.5.

[jira] [Commented] (SPARK-7841) Spark build should not use lib_managed for dependencies

2015-09-17 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791674#comment-14791674 ] Iulian Dragos commented on SPARK-7841: -- Yes, there are a few build scripts (including

[jira] [Commented] (SPARK-10444) Remove duplication in Mesos schedulers

2015-09-09 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736434#comment-14736434 ] Iulian Dragos commented on SPARK-10444: --- Another example of duplicated logic:

[jira] [Comment Edited] (SPARK-6350) Make mesosExecutorCores configurable in mesos "fine-grained" mode

2015-09-08 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734575#comment-14734575 ] Iulian Dragos edited comment on SPARK-6350 at 9/8/15 10:26 AM: --- I'm

[jira] [Reopened] (SPARK-6350) Make mesosExecutorCores configurable in mesos "fine-grained" mode

2015-09-08 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos reopened SPARK-6350: -- I'm re-opening this, since in the meantime this regressed. See changes in d86bbb, which regressed

[jira] [Commented] (SPARK-9708) Spark should create local temporary directories in Mesos sandbox when launched with Mesos

2015-09-08 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735011#comment-14735011 ] Iulian Dragos commented on SPARK-9708: -- This won't work when the external shuffle service is enabled,

[jira] [Commented] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-09-04 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730401#comment-14730401 ] Iulian Dragos commented on SPARK-4940: -- [~doctapp] I guess there is no perfect scheduling for every

[jira] [Created] (SPARK-10444) Remove duplication in Mesos schedulers

2015-09-04 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-10444: - Summary: Remove duplication in Mesos schedulers Key: SPARK-10444 URL: https://issues.apache.org/jira/browse/SPARK-10444 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-7874) Add a global setting for the fine-grained mesos scheduler that limits the number of concurrent tasks of a job

2015-09-03 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14729314#comment-14729314 ] Iulian Dragos commented on SPARK-7874: -- [~tnachen] I'm going to implement `spark.cores.max` as the

[jira] [Commented] (SPARK-7874) Add a global setting for the fine-grained mesos scheduler that limits the number of concurrent tasks of a job

2015-09-01 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726775#comment-14726775 ] Iulian Dragos commented on SPARK-7874: -- [~tomdz] do you mean respecting `spark.cores.max`, as it is

[jira] [Commented] (SPARK-3871) compute-classpath.sh does not escape :

2015-09-01 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726785#comment-14726785 ] Iulian Dragos commented on SPARK-3871: -- There's no more compute-classpath.sh. Ok to close this? >

[jira] [Commented] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-09-01 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726784#comment-14726784 ] Iulian Dragos commented on SPARK-4940: -- Would it make sense to allocate resources in a round-robin

[jira] [Commented] (SPARK-9708) Spark should create local temporary directories in Mesos sandbox when launched with Mesos

2015-09-01 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725657#comment-14725657 ] Iulian Dragos commented on SPARK-9708: -- I'm not sure if this is the entire story. Remember that

[jira] [Created] (SPARK-8977) Define the RateEstimator interface, and implement the ReceiverRateController

2015-07-10 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-8977: Summary: Define the RateEstimator interface, and implement the ReceiverRateController Key: SPARK-8977 URL: https://issues.apache.org/jira/browse/SPARK-8977 Project:

[jira] [Commented] (SPARK-5281) Registering table on RDD is giving MissingRequirementError

2015-07-10 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622510#comment-14622510 ] Iulian Dragos commented on SPARK-5281: -- Thanks for pointing them out. Glad it wasn't

[jira] [Created] (SPARK-8975) Implement a mechanism to send a new rate from the driver to the block generator

2015-07-10 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-8975: Summary: Implement a mechanism to send a new rate from the driver to the block generator Key: SPARK-8975 URL: https://issues.apache.org/jira/browse/SPARK-8975

[jira] [Created] (SPARK-8978) Implement the DirectKafkaController

2015-07-10 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-8978: Summary: Implement the DirectKafkaController Key: SPARK-8978 URL: https://issues.apache.org/jira/browse/SPARK-8978 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-8979) Implement a PIDRateEstimator

2015-07-10 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-8979: Summary: Implement a PIDRateEstimator Key: SPARK-8979 URL: https://issues.apache.org/jira/browse/SPARK-8979 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-5281) Registering table on RDD is giving MissingRequirementError

2015-07-09 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620148#comment-14620148 ] Iulian Dragos commented on SPARK-5281: -- [~marmbrus] what were those regressions, out

[jira] [Commented] (SPARK-7398) Add back-pressure to Spark Streaming

2015-07-03 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613240#comment-14613240 ] Iulian Dragos commented on SPARK-7398: -- Document updated. Add back-pressure to

[jira] [Updated] (SPARK-7398) Add back-pressure to Spark Streaming

2015-06-29 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-7398: - Description: Spark Streaming has trouble dealing with situations where batch processing time

[jira] [Commented] (SPARK-7398) Add back-pressure to Spark Streaming

2015-06-29 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605338#comment-14605338 ] Iulian Dragos commented on SPARK-7398: -- [~tdas] here it is:

[jira] [Commented] (SPARK-7944) Spark-Shell 2.11 1.4.0-RC-03 does not add jars to class path

2015-06-24 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599689#comment-14599689 ] Iulian Dragos commented on SPARK-7944: -- Unfortunately I don't think there's can be

[jira] [Commented] (SPARK-7398) Add back-pressure to Spark Streaming

2015-06-22 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595569#comment-14595569 ] Iulian Dragos commented on SPARK-7398: -- [~tdas] thanks for looking into this. I agree

[jira] [Commented] (SPARK-7944) Spark-Shell 2.11 1.4.0-RC-03 does not add jars to class path

2015-06-18 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592101#comment-14592101 ] Iulian Dragos commented on SPARK-7944: -- I can confirm this is working if the Spark

[jira] [Commented] (SPARK-7944) Spark-Shell 2.11 1.4.0-RC-03 does not add jars to class path

2015-06-16 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588740#comment-14588740 ] Iulian Dragos commented on SPARK-7944: -- I'll have a look tomorrow, I vaguely remember

[jira] [Commented] (SPARK-7890) Document that Spark 2.11 now supports Kafka

2015-06-01 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567167#comment-14567167 ] Iulian Dragos commented on SPARK-7890: -- [~srowen] thanks for fixing it, and sorry for

[jira] [Commented] (SPARK-8013) Get JDBC server working with Scala 2.11

2015-06-01 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567561#comment-14567561 ] Iulian Dragos commented on SPARK-8013: -- There's a Scala 2.11.7 milestone due in July,

[jira] [Created] (SPARK-7841) Spark build should not use lib_managed for dependencies

2015-05-23 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-7841: Summary: Spark build should not use lib_managed for dependencies Key: SPARK-7841 URL: https://issues.apache.org/jira/browse/SPARK-7841 Project: Spark Issue

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550142#comment-14550142 ] Iulian Dragos commented on SPARK-7726: -- This fix in 2.11.3 looks like the culprit

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550170#comment-14550170 ] Iulian Dragos commented on SPARK-7726: -- The problem is different visibility rules in

[jira] [Comment Edited] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550170#comment-14550170 ] Iulian Dragos edited comment on SPARK-7726 at 5/19/15 10:11 AM:

[jira] [Commented] (SPARK-5488) SPARK_LOCAL_IP not read by mesos scheduler

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550434#comment-14550434 ] Iulian Dragos commented on SPARK-5488: -- I've noticed something fishy as well, however

[jira] [Resolved] (SPARK-2555) Support configuration spark.scheduler.minRegisteredExecutorsRatio in Mesos mode.

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos resolved SPARK-2555. -- Resolution: Invalid As [commented

[jira] [Issue Comment Deleted] (SPARK-4899) Support Mesos features: roles and checkpoints

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-4899: - Comment: was deleted (was: Seems so, at least.) Support Mesos features: roles and checkpoints

[jira] [Commented] (SPARK-872) Should revive offer after tasks finish in Mesos fine-grained mode

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550473#comment-14550473 ] Iulian Dragos commented on SPARK-872: - I think we should close this due to inactivity.

[jira] [Updated] (SPARK-6679) java.lang.ClassNotFoundException on Mesos fine grained mode and input replication

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-6679: - Description: Spark Streaming 1.3.0, Mesos 0.21.1 - Only when using fine grained mode and

[jira] [Commented] (SPARK-3334) Spark causes mesos-master memory leak

2015-05-18 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547888#comment-14547888 ] Iulian Dragos commented on SPARK-3334: -- This was fixed in Mesos 0.21.0. Should we

[jira] [Commented] (SPARK-4899) Support Mesos features: roles and checkpoints

2015-05-18 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547906#comment-14547906 ] Iulian Dragos commented on SPARK-4899: -- Seems so, at least. Support Mesos features:

[jira] [Commented] (SPARK-5281) Registering table on RDD is giving MissingRequirementError

2015-05-06 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530224#comment-14530224 ] Iulian Dragos commented on SPARK-5281: -- Here's my workaround from [this stack

[jira] [Created] (SPARK-7399) Master fails on 2.11 with compilation error

2015-05-06 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-7399: Summary: Master fails on 2.11 with compilation error Key: SPARK-7399 URL: https://issues.apache.org/jira/browse/SPARK-7399 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6285) Duplicated code leads to errors

2015-03-13 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360937#comment-14360937 ] Iulian Dragos commented on SPARK-6285: -- Thanks, [~lian cheng] Duplicated code leads

[jira] [Commented] (SPARK-6286) Handle TASK_ERROR in TaskState

2015-03-12 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357310#comment-14357310 ] Iulian Dragos commented on SPARK-6286: -- Good point. It's been [introduced in

[jira] [Commented] (SPARK-6286) Handle TASK_ERROR in TaskState

2015-03-12 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358905#comment-14358905 ] Iulian Dragos commented on SPARK-6286: -- Sure, I'll issue a PR for handling

[jira] [Created] (SPARK-6286) Handle TASK_ERROR in TaskState

2015-03-11 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-6286: Summary: Handle TASK_ERROR in TaskState Key: SPARK-6286 URL: https://issues.apache.org/jira/browse/SPARK-6286 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6286) Handle TASK_ERROR in TaskState

2015-03-11 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-6286: - Labels: mesos (was: ) Handle TASK_ERROR in TaskState --

[jira] [Created] (SPARK-6285) Duplicated code leads to errors

2015-03-11 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-6285: Summary: Duplicated code leads to errors Key: SPARK-6285 URL: https://issues.apache.org/jira/browse/SPARK-6285 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6285) Duplicated code leads to errors

2015-03-11 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357297#comment-14357297 ] Iulian Dragos commented on SPARK-6285: -- According to the git commit message that

[jira] [Created] (SPARK-6287) Add support for dynamic allocation in the Mesos coarse-grained scheduler

2015-03-11 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-6287: Summary: Add support for dynamic allocation in the Mesos coarse-grained scheduler Key: SPARK-6287 URL: https://issues.apache.org/jira/browse/SPARK-6287 Project:

[jira] [Commented] (SPARK-6005) Flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.offset recovery

2015-02-25 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336542#comment-14336542 ] Iulian Dragos commented on SPARK-6005: -- Looks similar, but unless I miss something,

[jira] [Created] (SPARK-6005) Flaky test: o.a.s.s.k.DirectKafkaStreamSuite.offset recovery

2015-02-25 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-6005: Summary: Flaky test: o.a.s.s.k.DirectKafkaStreamSuite.offset recovery Key: SPARK-6005 URL: https://issues.apache.org/jira/browse/SPARK-6005 Project: Spark

  1   2   >