[jira] [Commented] (SPARK-1932) Race conditions in BlockManager.cachedPeers and ConnectionManager.onReceiveCallback

[ https://issues.apache.org/jira/browse/SPARK-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009330#comment-14009330 ] Shixiong Zhu commented on SPARK-1932: - PR: https://github.com/apache/spark/pull/887

[jira] [Resolved] (SPARK-1932) Race conditions in BlockManager.cachedPeers and ConnectionManager.onReceiveCallback

[ https://issues.apache.org/jira/browse/SPARK-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1932. - Resolution: Fixed Fix Version/s: 1.0.0 Race conditions in BlockManager.cachedPeers and

[jira] [Created] (SPARK-1937) Tasks can be submitted before executors are registered

Rui Li created SPARK-1937: - Summary: Tasks can be submitted before executors are registered Key: SPARK-1937 URL: https://issues.apache.org/jira/browse/SPARK-1937 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1936) Add apache header and remove author tags

[ https://issues.apache.org/jira/browse/SPARK-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009352#comment-14009352 ] Devaraj K commented on SPARK-1936: -- Pull Request added :

[jira] [Commented] (SPARK-1935) Explicitly add commons-codec 1.4 as a dependency

[ https://issues.apache.org/jira/browse/SPARK-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009472#comment-14009472 ] Sean Owen commented on SPARK-1935: -- Yeah, I think this is a matter of Maven's

[jira] [Updated] (SPARK-1935) Explicitly add commons-codec 1.4 as a dependency

[ https://issues.apache.org/jira/browse/SPARK-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1935: --- Assignee: Yin Huai Explicitly add commons-codec 1.4 as a dependency

[jira] [Commented] (SPARK-1928) DAGScheduler suspended by local task OOM

[ https://issues.apache.org/jira/browse/SPARK-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009477#comment-14009477 ] Zhen Peng commented on SPARK-1928: -- https://github.com/apache/spark/pull/883

[jira] [Updated] (SPARK-1937) Tasks can be submitted before executors are registered

[ https://issues.apache.org/jira/browse/SPARK-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated SPARK-1937: -- Attachment: Before-patch.png Tasks can be submitted before executors are registered

[jira] [Closed] (SPARK-1929) DAGScheduler suspended by local task OOM

[ https://issues.apache.org/jira/browse/SPARK-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1929. -- Resolution: Duplicate DAGScheduler suspended by local task OOM

[jira] [Updated] (SPARK-1937) Tasks can be submitted before executors are registered

[ https://issues.apache.org/jira/browse/SPARK-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated SPARK-1937: -- Attachment: RSBTest.scala The program that triggers the problem. With the patch, the whole execution time of

[jira] [Commented] (SPARK-1928) DAGScheduler suspended by local task OOM

[ https://issues.apache.org/jira/browse/SPARK-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009532#comment-14009532 ] Zhen Peng commented on SPARK-1928: -- [~gq] I met this case in our local mode spark

[jira] [Created] (SPARK-1938) ApproxCountDistinctMergeFunction should return Int value.

Takuya Ueshin created SPARK-1938: Summary: ApproxCountDistinctMergeFunction should return Int value. Key: SPARK-1938 URL: https://issues.apache.org/jira/browse/SPARK-1938 Project: Spark

[jira] [Commented] (SPARK-1938) ApproxCountDistinctMergeFunction should return Int value.

[ https://issues.apache.org/jira/browse/SPARK-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009555#comment-14009555 ] Takuya Ueshin commented on SPARK-1938: -- PRed:

[jira] [Commented] (SPARK-1935) Explicitly add commons-codec 1.4 as a dependency

[ https://issues.apache.org/jira/browse/SPARK-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009856#comment-14009856 ] Yin Huai commented on SPARK-1935: - [~srowen] yes, it is the reason. I did a quick search

[jira] [Assigned] (SPARK-1021) sortByKey() launches a cluster job when it shouldn't

[ https://issues.apache.org/jira/browse/SPARK-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamstra reassigned SPARK-1021: --- Assignee: Mark Hamstra sortByKey() launches a cluster job when it shouldn't

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009875#comment-14009875 ] Mark Hamstra commented on SPARK-983: I'm hoping these can be kept orthogonal, but I

[jira] [Comment Edited] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009875#comment-14009875 ] Mark Hamstra edited comment on SPARK-983 at 5/27/14 4:40 PM: -

[jira] [Updated] (SPARK-1930) Container is running beyond physical memory limits

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Fix Version/s: 1.0.1 Container is running beyond physical memory limits

[jira] [Updated] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Summary: The Container is running beyond physical memory limits, so as to be killed. (was:

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010068#comment-14010068 ] Colin Patrick McCabe commented on SPARK-1518: - bq. Hey Colin Patrick McCabe,

[jira] [Commented] (SPARK-1103) Garbage collect RDD information inside of Spark

[ https://issues.apache.org/jira/browse/SPARK-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010099#comment-14010099 ] Andrew Ash commented on SPARK-1103: --- https://github.com/apache/spark/pull/126 Garbage

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010117#comment-14010117 ] Patrick Wendell commented on SPARK-1518: Ah okay. I'm not sure what the oldest

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010182#comment-14010182 ] Madhu Siddalingaiah commented on SPARK-983: --- Understood. Here are my thoughts: I

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010190#comment-14010190 ] Aaron Davidson commented on SPARK-983: -- Does sound reasonable. For some reason it does

[jira] [Updated] (SPARK-1931) Graph.partitionBy does not reconstruct routing tables

[ https://issues.apache.org/jira/browse/SPARK-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-1931: -- Description: Commit 905173df57b90f90ebafb22e43f55164445330e6 introduced a bug in partitionBy where,

[jira] [Updated] (SPARK-1931) Graph.partitionBy does not reconstruct routing tables

[ https://issues.apache.org/jira/browse/SPARK-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-1931: -- Description: Commit 905173df57b90f90ebafb22e43f55164445330e6 introduced a bug in partitionBy where,

[jira] [Updated] (SPARK-1931) Graph.partitionBy does not reconstruct routing tables

[ https://issues.apache.org/jira/browse/SPARK-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-1931: -- Description: Commit 905173df57b90f90ebafb22e43f55164445330e6 introduced a bug in partitionBy where,

[jira] [Commented] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010258#comment-14010258 ] Patrick Wendell commented on SPARK-1930: PySpark might also be an issue with this,

[jira] [Updated] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1518: --- Priority: Critical (was: Major) Spark master doesn't compile against hadoop-common trunk

[jira] [Updated] (SPARK-1931) Graph.partitionBy does not reconstruct routing tables

[ https://issues.apache.org/jira/browse/SPARK-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1931: --- Fix Version/s: (was: 1.0.0) 1.0.1 Graph.partitionBy does not

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010284#comment-14010284 ] Colin Patrick McCabe commented on SPARK-1518: - I think it's very, very, very

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010294#comment-14010294 ] Sean Owen commented on SPARK-1518: -- 0.20.x stopped in early 2010. It is ancient. Spark

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010305#comment-14010305 ] Marcelo Vanzin commented on SPARK-1518: --- Hmm, may I suggest a different approach?

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010322#comment-14010322 ] Sean Owen commented on SPARK-1518: -- RE: Hadoop versions, in my reckoning of the twisted

[jira] [Commented] (SPARK-1931) Graph.partitionBy does not reconstruct routing tables

[ https://issues.apache.org/jira/browse/SPARK-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010340#comment-14010340 ] Ankur Dave commented on SPARK-1931: --- Since the fix didn't make it into Spark 1.0.0, a

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010339#comment-14010339 ] Patrick Wendell commented on SPARK-1518: Re: versioning - I was just asking

[jira] [Comment Edited] (SPARK-1931) Graph.partitionBy does not reconstruct routing tables

[ https://issues.apache.org/jira/browse/SPARK-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010340#comment-14010340 ] Ankur Dave edited comment on SPARK-1931 at 5/27/14 9:47 PM:

[jira] [Updated] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-983: - Assignee: Madhu Siddalingaiah Support external sorting for RDD#sortByKey()

[jira] [Comment Edited] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010190#comment-14010190 ] Aaron Davidson edited comment on SPARK-983 at 5/27/14 9:54 PM:

[jira] [Resolved] (SPARK-1926) Nullability of Max/Min/First should be true.

[ https://issues.apache.org/jira/browse/SPARK-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-1926. Resolution: Fixed Fix Version/s: 1.0.1 1.1.0 Assignee: Takuya

[jira] [Resolved] (SPARK-1915) AverageFunction should not count if the evaluated value is null.

[ https://issues.apache.org/jira/browse/SPARK-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-1915. Resolution: Fixed Fix Version/s: 1.0.1 1.1.0 Assignee: Takuya

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010364#comment-14010364 ] Colin Patrick McCabe commented on SPARK-1518: - bq. It would be good to list

[jira] [Created] (SPARK-1939) Improve takeSample method in RDD

Doris Xin created SPARK-1939: Summary: Improve takeSample method in RDD Key: SPARK-1939 URL: https://issues.apache.org/jira/browse/SPARK-1939 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-1914) Simplify CountFunction not to traverse to evaluate all child expressions.

[ https://issues.apache.org/jira/browse/SPARK-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1914: --- Fix Version/s: (was: 1.0.0) 1.0.1 Simplify CountFunction not to traverse to

[jira] [Created] (SPARK-1940) Enable rolling of executor logs (stdout / stderr)

Tathagata Das created SPARK-1940: Summary: Enable rolling of executor logs (stdout / stderr) Key: SPARK-1940 URL: https://issues.apache.org/jira/browse/SPARK-1940 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-1940) Enable rolling of executor logs (stdout / stderr)

[ https://issues.apache.org/jira/browse/SPARK-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-1940: Assignee: Tathagata Das Enable rolling of executor logs (stdout / stderr)

[jira] [Commented] (SPARK-1940) Enable rolling of executor logs (stdout / stderr)

[ https://issues.apache.org/jira/browse/SPARK-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010413#comment-14010413 ] Tathagata Das commented on SPARK-1940: -- This is solved in

[jira] [Updated] (SPARK-1939) Improve takeSample method in RDD

[ https://issues.apache.org/jira/browse/SPARK-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1939: - Assignee: Doris Xin Improve takeSample method in RDD

[jira] [Updated] (SPARK-1922) hql query throws RuntimeException: Unsupported dataType if struct field of a table has a column with underscore in name

[ https://issues.apache.org/jira/browse/SPARK-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1922: --- Fix Version/s: 1.0.1 1.1.0 hql query throws RuntimeException: Unsupported

[jira] [Updated] (SPARK-1922) hql query throws RuntimeException: Unsupported dataType if struct field of a table has a column with underscore in name

[ https://issues.apache.org/jira/browse/SPARK-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1922: --- Assignee: Reynold Xin hql query throws RuntimeException: Unsupported dataType if struct field of

[jira] [Updated] (SPARK-1922) hql query throws RuntimeException: Unsupported dataType if struct field of a table has a column with underscore in name

[ https://issues.apache.org/jira/browse/SPARK-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1922: --- Assignee: llai (was: Reynold Xin) hql query throws RuntimeException: Unsupported dataType if

[jira] [Resolved] (SPARK-1922) hql query throws RuntimeException: Unsupported dataType if struct field of a table has a column with underscore in name

[ https://issues.apache.org/jira/browse/SPARK-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-1922. Resolution: Fixed hql query throws RuntimeException: Unsupported dataType if struct field of a

[jira] [Commented] (SPARK-1566) Consolidate the Spark Programming Guide with tabs for all languages

[ https://issues.apache.org/jira/browse/SPARK-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010496#comment-14010496 ] Matei Zaharia commented on SPARK-1566: -- https://github.com/apache/spark/pull/896

[jira] [Updated] (SPARK-1915) AverageFunction should not count if the evaluated value is null.

[ https://issues.apache.org/jira/browse/SPARK-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-1915: -- Description: Average values are difference between the calculation is done partially or not

[jira] [Commented] (SPARK-1915) AverageFunction should not count if the evaluated value is null.

[ https://issues.apache.org/jira/browse/SPARK-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010521#comment-14010521 ] Cheng Lian commented on SPARK-1915: --- Added bug reproduction steps. AverageFunction

[jira] [Created] (SPARK-1941) Update streamlib to 2.7.0 and use HyperLogLogPlus instead of HyperLogLog

Reynold Xin created SPARK-1941: -- Summary: Update streamlib to 2.7.0 and use HyperLogLogPlus instead of HyperLogLog Key: SPARK-1941 URL: https://issues.apache.org/jira/browse/SPARK-1941 Project: Spark

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010578#comment-14010578 ] Colin Patrick McCabe commented on SPARK-1518: - Sounds good.

[jira] [Updated] (SPARK-1825) Windows Spark fails to work with Linux YARN

[ https://issues.apache.org/jira/browse/SPARK-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1825: - Fix Version/s: (was: 1.0.0) Windows Spark fails to work with Linux YARN

[jira] [Created] (SPARK-1942) Stop clearing spark.driver.port in unit tests

Matei Zaharia created SPARK-1942: Summary: Stop clearing spark.driver.port in unit tests Key: SPARK-1942 URL: https://issues.apache.org/jira/browse/SPARK-1942 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-1943) Testing use of target version field

Patrick Wendell created SPARK-1943: -- Summary: Testing use of target version field Key: SPARK-1943 URL: https://issues.apache.org/jira/browse/SPARK-1943 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1943) Testing use of target version field

[ https://issues.apache.org/jira/browse/SPARK-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1943: --- Target Version/s: (was: 0.9.2, 1.0.1) Testing use of target version field

[jira] [Updated] (SPARK-1944) Document --verbose in spark-shell -h

[ https://issues.apache.org/jira/browse/SPARK-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-1944: -- Summary: Document --verbose in spark-shell -h (was: Document --verbse in spark-shell -h) Document

[jira] [Created] (SPARK-1944) Document --verbse in spark-shell -h

Andrew Ash created SPARK-1944: - Summary: Document --verbse in spark-shell -h Key: SPARK-1944 URL: https://issues.apache.org/jira/browse/SPARK-1944 Project: Spark Issue Type: Documentation