[jira] [Commented] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622311#comment-17622311 ] Daniel Darabos commented on SPARK-40873: This works on the R side for dropping t

[jira] [Commented] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622251#comment-17622251 ] Daniel Darabos commented on SPARK-40873: Oh, I think I got it! With debug loggin

[jira] [Created] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
Daniel Darabos created SPARK-40873: -- Summary: Spark doesn't see some Parquet columns written from r-arrow Key: SPARK-40873 URL: https://issues.apache.org/jira/browse/SPARK-40873 Project: Spark

[jira] [Updated] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-40873: --- Attachment: part-0.parquet > Spark doesn't see some Parquet columns written from r-arrow > -

[jira] [Commented] (SPARK-37690) Recursive view `df` detected (cycle: `df` -> `df`)

2022-08-12 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17578901#comment-17578901 ] Daniel Darabos commented on SPARK-37690: It's fixed in Spark 3.3.0. (https://gi

[jira] [Commented] (SPARK-37690) Recursive view `df` detected (cycle: `df` -> `df`)

2022-01-17 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477141#comment-17477141 ] Daniel Darabos commented on SPARK-37690: We've hit this too with Spark 3.2.0. Co

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-11-23 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16697319#comment-16697319 ] Daniel Darabos commented on SPARK-20144: So where do we go from here? Should I t

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650817#comment-16650817 ] Daniel Darabos commented on SPARK-20144: Thanks, those are good questions. # Th

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650722#comment-16650722 ] Daniel Darabos commented on SPARK-20144: Yeah, I'm not too happy about the alpha

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650492#comment-16650492 ] Daniel Darabos commented on SPARK-20144: Thanks Victor! I've expanded the test w

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-08 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642401#comment-16642401 ] Daniel Darabos commented on SPARK-20144: Sorry, I had an idea for a quick fix fo

[jira] [Commented] (SPARK-23207) Shuffle+Repartition on an DataFrame could lead to incorrect answers

2018-08-23 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590435#comment-16590435 ] Daniel Darabos commented on SPARK-23207: Sorry, could you clarify the fix versio

[jira] [Commented] (SPARK-25146) avg() returns null on some decimals

2018-08-17 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584015#comment-16584015 ] Daniel Darabos commented on SPARK-25146: Wonderful, thanks! Sorry I missed the f

[jira] [Updated] (SPARK-25146) avg() returns null on some decimals

2018-08-17 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-25146: --- Description: We compute some 0-10 numbers in a pipeline using Spark SQL. Then we average th

[jira] [Created] (SPARK-25146) avg() returns null on some decimals

2018-08-17 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-25146: -- Summary: avg() returns null on some decimals Key: SPARK-25146 URL: https://issues.apache.org/jira/browse/SPARK-25146 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23666) Undeterministic column name with UDFs

2018-03-13 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396832#comment-16396832 ] Daniel Darabos commented on SPARK-23666: I've looked at the code and both {{Scala

[jira] [Created] (SPARK-23666) Undeterministic column name with UDFs

2018-03-13 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-23666: -- Summary: Undeterministic column name with UDFs Key: SPARK-23666 URL: https://issues.apache.org/jira/browse/SPARK-23666 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21418) NoSuchElementException: None.get in DataSourceScanExec with sun.io.serialization.extendedDebugInfo=true

2017-09-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165946#comment-16165946 ] Daniel Darabos commented on SPARK-21418: Sean's fix should cover you no matter wh

[jira] [Commented] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

2017-09-04 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16152742#comment-16152742 ] Daniel Darabos commented on SPARK-21418: Sorry for the delay. I can confirm that

[jira] [Comment Edited] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

2017-09-04 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088952#comment-16088952 ] Daniel Darabos edited comment on SPARK-21418 at 9/4/17 3:49 PM: ---

[jira] [Commented] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

2017-07-16 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088952#comment-16088952 ] Daniel Darabos commented on SPARK-21418: I'm on holiday without a computer throug

[jira] [Commented] (SPARK-20070) Redact datasource explain output

2017-07-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087378#comment-16087378 ] Daniel Darabos commented on SPARK-20070: I think this change has broken our appli

[jira] [Created] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

2017-07-14 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-21418: -- Summary: NoSuchElementException: None.get on DataFrame.rdd Key: SPARK-21418 URL: https://issues.apache.org/jira/browse/SPARK-21418 Project: Spark Issue T

[jira] [Created] (SPARK-21136) Misleading error message for typo in SQL

2017-06-19 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-21136: -- Summary: Misleading error message for typo in SQL Key: SPARK-21136 URL: https://issues.apache.org/jira/browse/SPARK-21136 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-19209) "No suitable driver" on first try

2017-01-16 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824224#comment-15824224 ] Daniel Darabos edited comment on SPARK-19209 at 1/16/17 4:26 PM: --

[jira] [Commented] (SPARK-19209) "No suitable driver" on first try

2017-01-16 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824224#comment-15824224 ] Daniel Darabos commented on SPARK-19209: Sorry, Jira had some issues when I was t

[jira] [Updated] (SPARK-19209) "No suitable driver" on first try

2017-01-13 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-19209: --- Description: This is a regression from Spark 2.0.2. Observe! {code} $ ~/spark-2.0.2/bin/spar

[jira] [Commented] (SPARK-19209) "No suitable driver" on first try

2017-01-13 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821578#comment-15821578 ] Daniel Darabos commented on SPARK-19209: Puzzlingly this only happens in the appl

[jira] [Created] (SPARK-19209) "No suitable driver" on first try

2017-01-13 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-19209: -- Summary: "No suitable driver" on first try Key: SPARK-19209 URL: https://issues.apache.org/jira/browse/SPARK-19209 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-16625) Oracle JDBC table creation fails with ORA-00902: invalid datatype

2016-07-19 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-16625: -- Summary: Oracle JDBC table creation fails with ORA-00902: invalid datatype Key: SPARK-16625 URL: https://issues.apache.org/jira/browse/SPARK-16625 Project: Spark

[jira] [Updated] (SPARK-12964) SparkContext.localProperties leaked

2016-07-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-12964: --- Description: I have a non-deterministic but quite reliable reproduction for a case where {{s

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-07 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318523#comment-15318523 ] Daniel Darabos commented on SPARK-15796: > The only argument against it was that

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-07 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318486#comment-15318486 ] Daniel Darabos commented on SPARK-15796: The example program takes less than a mi

[jira] [Updated] (SPARK-11293) Spillable collections leak shuffle memory

2016-03-23 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-11293: --- Affects Version/s: 1.6.1 > Spillable collections leak shuffle memory > --

[jira] [Updated] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-13620: --- Fix Version/s: 2.0.0 > Avoid reverse DNS lookup for 0.0.0.0 on startup >

[jira] [Resolved] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos resolved SPARK-13620. Resolution: Fixed > Avoid reverse DNS lookup for 0.0.0.0 on startup > -

[jira] [Commented] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177784#comment-15177784 ] Daniel Darabos commented on SPARK-13620: I just tested with the latest 2.0 nightl

[jira] [Commented] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177655#comment-15177655 ] Daniel Darabos commented on SPARK-13620: For the record I figured out a workaroun

[jira] [Commented] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-02 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175974#comment-15175974 ] Daniel Darabos commented on SPARK-13620: Looks like the whole {{InetSocketAddress

[jira] [Updated] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-02 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-13620: --- Description: I noticed we spend 5+ seconds during application startup with the following sta

[jira] [Updated] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-02 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-13620: --- Description: I noticed we spend 5+ seconds during application startup with the following sta

[jira] [Created] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-02 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-13620: -- Summary: Avoid reverse DNS lookup for 0.0.0.0 on startup Key: SPARK-13620 URL: https://issues.apache.org/jira/browse/SPARK-13620 Project: Spark Issue Typ

[jira] [Reopened] (SPARK-10057) Faill to load class org.slf4j.impl.StaticLoggerBinder

2016-03-01 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos reopened SPARK-10057: Thanks for the repro, Stephen! It also works in Spark 1.6.0. This was mentioned in https://git

[jira] [Updated] (SPARK-10057) Faill to load class org.slf4j.impl.StaticLoggerBinder

2016-03-01 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-10057: --- Affects Version/s: 1.6.0 > Faill to load class org.slf4j.impl.StaticLoggerBinder > --

[jira] [Created] (SPARK-13212) Provide a way to unregister data sources from a SQLContext

2016-02-05 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-13212: -- Summary: Provide a way to unregister data sources from a SQLContext Key: SPARK-13212 URL: https://issues.apache.org/jira/browse/SPARK-13212 Project: Spark

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2016-02-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15130351#comment-15130351 ] Daniel Darabos commented on SPARK-1239: --- I've read an interesting article about the

[jira] [Updated] (SPARK-12964) SparkContext.localProperties leaked

2016-01-22 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-12964: --- Description: I have a non-deterministic but quite reliable reproduction for a case where {{s

[jira] [Created] (SPARK-12964) SparkContext.localProperties leaked

2016-01-22 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-12964: -- Summary: SparkContext.localProperties leaked Key: SPARK-12964 URL: https://issues.apache.org/jira/browse/SPARK-12964 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2309) Generalize the binary logistic regression into multinomial logistic regression

2016-01-21 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110628#comment-15110628 ] Daniel Darabos commented on SPARK-2309: --- https://github.com/apache/spark/blob/v1.6.0

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098011#comment-15098011 ] Daniel Darabos commented on SPARK-11293: > so add 1.6.0 as affected version... D

[jira] [Updated] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-11293: --- Affects Version/s: 1.6.0 > Spillable collections leak shuffle memory > --

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097987#comment-15097987 ] Daniel Darabos commented on SPARK-11293: > so should be reopened or not? is there

[jira] [Reopened] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos reopened SPARK-11293: > Spillable collections leak shuffle memory > - > >

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083326#comment-15083326 ] Daniel Darabos commented on SPARK-11293: Sorry, my example was overly complicated

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083311#comment-15083311 ] Daniel Darabos commented on SPARK-11293: I have a somewhat contrived example that

[jira] [Commented] (SPARK-11652) Remote code execution with InvokerTransformer

2015-11-13 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003982#comment-15003982 ] Daniel Darabos commented on SPARK-11652: > I may be missing some point, but Spark

[jira] [Created] (SPARK-11652) Remote code execution with InvokerTransformer

2015-11-11 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-11652: -- Summary: Remote code execution with InvokerTransformer Key: SPARK-11652 URL: https://issues.apache.org/jira/browse/SPARK-11652 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-11-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987046#comment-14987046 ] Daniel Darabos commented on SPARK-1239: --- I can also add some data. I have a ShuffleM

[jira] [Created] (SPARK-11403) Log something when dying from OnOutOfMemoryError

2015-10-29 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-11403: -- Summary: Log something when dying from OnOutOfMemoryError Key: SPARK-11403 URL: https://issues.apache.org/jira/browse/SPARK-11403 Project: Spark Issue Ty

[jira] [Closed] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-06 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos closed SPARK-10792. -- Resolution: Duplicate > Spark + YARN – executor is not re-created > ---

[jira] [Commented] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-06 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945023#comment-14945023 ] Daniel Darabos commented on SPARK-10792: After some digging it appears this is a

[jira] [Commented] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943407#comment-14943407 ] Daniel Darabos commented on SPARK-10792: I've edited the priority, the title, and

[jira] [Comment Edited] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943405#comment-14943405 ] Daniel Darabos edited comment on SPARK-10792 at 10/5/15 2:17 PM: --

[jira] [Updated] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-10792: --- Priority: Critical (was: Minor) Component/s: (was: Streaming) Summary: Spa

[jira] [Commented] (SPARK-10792) Spark streaming + YARN – executor is not re-created on machine restart

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943405#comment-14943405 ] Daniel Darabos commented on SPARK-10792: I work with Andras and can now substanti

[jira] [Commented] (SPARK-5077) Map output statuses can still exceed spark.akka.frameSize

2015-09-11 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14740816#comment-14740816 ] Daniel Darabos commented on SPARK-5077: --- Hi Josh, The MapOutputTracker errors are a

[jira] [Commented] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-08-28 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718559#comment-14718559 ] Daniel Darabos commented on SPARK-6497: --- I've checked 1.5.0-RC2 and the repro still

[jira] [Updated] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-08-28 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-6497: -- Priority: Major (was: Minor) > Class is not registered: scala.reflect.ManifestFactory$$anon$9 >

[jira] [Updated] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-08-28 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-6497: -- Affects Version/s: 1.5.0 1.4.0 > Class is not registered: scala.reflect.M

[jira] [Commented] (SPARK-9382) Tachyon version mismatch

2015-07-27 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642908#comment-14642908 ] Daniel Darabos commented on SPARK-9382: --- Ah, looks like this was just an oversight a

[jira] [Closed] (SPARK-9382) Tachyon version mismatch

2015-07-27 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos closed SPARK-9382. - Resolution: Fixed Fix Version/s: 1.4.1 > Tachyon version mismatch > ---

[jira] [Created] (SPARK-9382) Tachyon version mismatch

2015-07-27 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-9382: - Summary: Tachyon version mismatch Key: SPARK-9382 URL: https://issues.apache.org/jira/browse/SPARK-9382 Project: Spark Issue Type: Bug Components

[jira] [Commented] (SPARK-8898) Jets3t hangs with more than 1 core

2015-07-17 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631153#comment-14631153 ] Daniel Darabos commented on SPARK-8898: --- Unfortunately the {{spark-ec2}} script does

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628380#comment-14628380 ] Daniel Darabos commented on SPARK-4879: --- Thanks, Josh! I wonder when the post-save-a

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628086#comment-14628086 ] Daniel Darabos commented on SPARK-4879: --- I've managed to reproduce on Spark 1.3.1 to

[jira] [Commented] (SPARK-8836) Sorted join

2015-07-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628030#comment-14628030 ] Daniel Darabos commented on SPARK-8836: --- I just noticed this has actually been imple

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-12 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14624000#comment-14624000 ] Daniel Darabos commented on SPARK-4879: --- Good idea! I'll try with 1.3.1 next week.

[jira] [Comment Edited] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-10 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1466#comment-1466 ] Daniel Darabos edited comment on SPARK-4879 at 7/10/15 12:25 PM: ---

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-10 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1466#comment-1466 ] Daniel Darabos commented on SPARK-4879: --- I wonder if this issue is serious enough to

[jira] [Commented] (SPARK-8960) Style cleanup of spark_ec2.py

2015-07-09 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621397#comment-14621397 ] Daniel Darabos commented on SPARK-8960: --- > So, can I sort of hijack this to continue

[jira] [Created] (SPARK-8960) Style cleanup of spark_ec2.py

2015-07-09 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-8960: - Summary: Style cleanup of spark_ec2.py Key: SPARK-8960 URL: https://issues.apache.org/jira/browse/SPARK-8960 Project: Spark Issue Type: Task Comp

[jira] [Comment Edited] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-09 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620570#comment-14620570 ] Daniel Darabos edited comment on SPARK-4879 at 7/9/15 6:15 PM: -

[jira] [Comment Edited] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-09 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620570#comment-14620570 ] Daniel Darabos edited comment on SPARK-4879 at 7/9/15 6:15 PM: -

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-07-09 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620570#comment-14620570 ] Daniel Darabos commented on SPARK-4879: --- I have a fairly reliable reproduction with

[jira] [Updated] (SPARK-8902) Hostname missing in spark-ec2 error message

2015-07-08 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-8902: -- Description: When SSH to a machine fails, the following message is printed: {noformat} Failed t

[jira] [Created] (SPARK-8902) Hostname missing in spark-ec2 error message

2015-07-08 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-8902: - Summary: Hostname missing in spark-ec2 error message Key: SPARK-8902 URL: https://issues.apache.org/jira/browse/SPARK-8902 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-8898) Jets3t hangs with more than 1 core

2015-07-08 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-8898: - Summary: Jets3t hangs with more than 1 core Key: SPARK-8898 URL: https://issues.apache.org/jira/browse/SPARK-8898 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-8893) Require positive partition counts in RDD.repartition

2015-07-08 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-8893: -- Description: What does {{sc.parallelize(1 to 3).repartition(p).collect}} return? I would expect

[jira] [Created] (SPARK-8893) Require positive partition counts in RDD.repartition

2015-07-08 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-8893: - Summary: Require positive partition counts in RDD.repartition Key: SPARK-8893 URL: https://issues.apache.org/jira/browse/SPARK-8893 Project: Spark Issue Ty

[jira] [Created] (SPARK-8836) Sorted join

2015-07-06 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-8836: - Summary: Sorted join Key: SPARK-8836 URL: https://issues.apache.org/jira/browse/SPARK-8836 Project: Spark Issue Type: Improvement Components: Spa

[jira] [Commented] (SPARK-5945) Spark should not retry a stage infinitely on a FetchFailedException

2015-07-02 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611960#comment-14611960 ] Daniel Darabos commented on SPARK-5945: --- At the moment we have a ton of these infini

[jira] [Commented] (SPARK-4357) Modify release publishing to work with Scala 2.11

2015-06-30 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608150#comment-14608150 ] Daniel Darabos commented on SPARK-4357: --- Hi, what is the obstacle to providing pre-b

[jira] [Commented] (SPARK-8310) Spark EC2 branch in 1.4 is wrong

2015-06-29 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605411#comment-14605411 ] Daniel Darabos commented on SPARK-8310: --- It's an easy mistake to make, and one of th

[jira] [Commented] (SPARK-6944) Mechanism to associate generic operator scope with RDD's

2015-05-28 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563729#comment-14563729 ] Daniel Darabos commented on SPARK-6944: --- Yes, I looked at RDDOperationScope a bit. I

[jira] [Commented] (SPARK-6944) Mechanism to associate generic operator scope with RDD's

2015-05-28 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562823#comment-14562823 ] Daniel Darabos commented on SPARK-6944: --- Thanks for this fantastic feature! Please t

[jira] [Commented] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-03-24 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377749#comment-14377749 ] Daniel Darabos commented on SPARK-6497: --- By the way, have you considered running the

[jira] [Created] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-03-24 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-6497: - Summary: Class is not registered: scala.reflect.ManifestFactory$$anon$9 Key: SPARK-6497 URL: https://issues.apache.org/jira/browse/SPARK-6497 Project: Spark

[jira] [Created] (SPARK-5102) CompressedMapStatus needs to be registered with Kryo

2015-01-06 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-5102: - Summary: CompressedMapStatus needs to be registered with Kryo Key: SPARK-5102 URL: https://issues.apache.org/jira/browse/SPARK-5102 Project: Spark Issue Ty

[jira] [Commented] (SPARK-4831) Current directory always on classpath with spark-submit

2014-12-11 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243074#comment-14243074 ] Daniel Darabos commented on SPARK-4831: --- bq. Is it perhaps finding and exploded dire

[jira] [Created] (SPARK-4831) Current directory always on classpath with spark-submit

2014-12-11 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-4831: - Summary: Current directory always on classpath with spark-submit Key: SPARK-4831 URL: https://issues.apache.org/jira/browse/SPARK-4831 Project: Spark Issue

  1   2   >