[jira] [Created] (SPARK-45616) Usages of ParVector are unsafe because it does not propagate ThreadLocals or SparkSession

2023-10-20 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-45616: -- Summary: Usages of ParVector are unsafe because it does not propagate ThreadLocals or SparkSession Key: SPARK-45616 URL: https://issues.apache.org/jira/browse/SPARK-45616

[jira] [Assigned] (SPARK-38677) pyspark hangs in local mode running rdd map operation

2022-04-04 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave reassigned SPARK-38677: -- Assignee: Ankur Dave > pyspark hangs in local mode running rdd map operation >

[jira] [Updated] (SPARK-37089) ParquetFileFormat registers task completion listeners lazily, causing Python writer thread to segfault when off-heap vectorized reader is enabled

2021-10-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-37089: --- Description: The task completion listener that closes the vectorized reader is registered lazily

[jira] [Updated] (SPARK-37089) ParquetFileFormat registers task completion listeners lazily, causing Python writer thread to segfault when off-heap vectorized reader is enabled

2021-10-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-37089: --- Summary: ParquetFileFormat registers task completion listeners lazily, causing Python writer thread

[jira] [Updated] (SPARK-37089) ParquetFileFormat/OrcFileFormat register task completion listeners lazily, causing Python writer thread to segfault when off-heap vectorized reader is enabled

2021-10-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-37089: --- Description: The task completion listener that closes the vectorized reader is registered lazily

[jira] [Updated] (SPARK-37089) ParquetFileFormat/OrcFileFormat register task completion listeners lazily, causing Python writer thread to segfault when off-heap vectorized reader is enabled

2021-10-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-37089: --- Summary: ParquetFileFormat/OrcFileFormat register task completion listeners lazily, causing Python

[jira] [Updated] (SPARK-37089) ParquetFileFormat registers task completion listeners lazily, causing Python writer thread to segfault when off-heap vectorized reader is enabled

2021-10-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-37089: --- Description: The task completion listener that closes the vectorized reader is registered lazily

[jira] [Created] (SPARK-37089) ParquetFileFormat registers task completion listeners lazily, causing Python writer thread to segfault when off-heap vectorized reader is enabled

2021-10-21 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-37089: -- Summary: ParquetFileFormat registers task completion listeners lazily, causing Python writer thread to segfault when off-heap vectorized reader is enabled Key: SPARK-37089 URL:

[jira] [Commented] (SPARK-37088) Python UDF after off-heap vectorized reader can cause crash due to use-after-free in writer thread

2021-10-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432540#comment-17432540 ] Ankur Dave commented on SPARK-37088: https://github.com/apache/spark/pull/34245 > Python UDF after

[jira] [Updated] (SPARK-37088) Python UDF after off-heap vectorized reader can cause crash due to use-after-free in writer thread

2021-10-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-37088: --- Description: Python UDFs in Spark SQL are run in a separate Python process. The Python process is

[jira] [Created] (SPARK-37088) Python UDF after off-heap vectorized reader can cause crash due to use-after-free in writer thread

2021-10-21 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-37088: -- Summary: Python UDF after off-heap vectorized reader can cause crash due to use-after-free in writer thread Key: SPARK-37088 URL: https://issues.apache.org/jira/browse/SPARK-37088

[jira] [Updated] (SPARK-35486) MemoryConsumer reservations that trigger a partial self-spill can fail even if memory is available

2021-05-21 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-35486: --- Description: When a memory reservation triggers a self-spill, ExecutionMemoryPool#releaseMemory()

[jira] [Created] (SPARK-35486) MemoryConsumer reservations that trigger a partial self-spill can fail even if memory is available

2021-05-21 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-35486: -- Summary: MemoryConsumer reservations that trigger a partial self-spill can fail even if memory is available Key: SPARK-35486 URL: https://issues.apache.org/jira/browse/SPARK-35486

[jira] [Updated] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-33260: --- Affects Version/s: (was: 2.4.7) 3.0.0 > SortExec produces incorrect

[jira] [Created] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-33260: -- Summary: SortExec produces incorrect results if sortOrder is a Stream Key: SPARK-33260 URL: https://issues.apache.org/jira/browse/SPARK-33260 Project: Spark

[jira] [Comment Edited] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

2020-09-14 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195252#comment-17195252 ] Ankur Dave edited comment on SPARK-32872 at 9/14/20, 7:37 AM: -- Thanks,

[jira] [Updated] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

2020-09-14 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Affects Version/s: (was: 1.5.2) (was: 1.4.1) > BytesToBytesMap at

[jira] [Comment Edited] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

2020-09-14 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195252#comment-17195252 ] Ankur Dave edited comment on SPARK-32872 at 9/14/20, 7:30 AM: -- Thanks,

[jira] [Commented] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

2020-09-14 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195252#comment-17195252 ] Ankur Dave commented on SPARK-32872: Thanks, [~dongjoon]! Based on a quick look at the history, I

[jira] [Updated] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

2020-09-14 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Affects Version/s: 1.4.1 1.5.2 1.6.3 >

[jira] [Updated] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

2020-09-13 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Summary: BytesToBytesMap at MAX_CAPACITY exceeds growth threshold (was: BytesToBytesMap exceeds

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold at MAX_CAPACITY

2020-09-13 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Description: When BytesToBytesMap is at {{MAX_CAPACITY}} and reaches the growth threshold, 

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold at MAX_CAPACITY

2020-09-13 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Summary: BytesToBytesMap exceeds growth threshold at MAX_CAPACITY (was: BytesToBytesMap exceeds

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

2020-09-13 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Description: When BytesToBytesMap is close to {{MAX_CAPACITY}} and needs to grow, {{numKeys >=

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

2020-09-13 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Description: When BytesToBytesMap is close to {{MAX_CAPACITY}} and needs to grow, {{numKeys >=

[jira] [Created] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

2020-09-13 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-32872: -- Summary: BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY Key: SPARK-32872 URL: https://issues.apache.org/jira/browse/SPARK-32872 Project: Spark

[jira] [Updated] (SPARK-18845) PageRank has incorrect initialization value that leads to slow convergence

2016-12-15 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-18845: --- Assignee: Andrew Ray > PageRank has incorrect initialization value that leads to slow convergence >

[jira] [Resolved] (SPARK-18845) PageRank has incorrect initialization value that leads to slow convergence

2016-12-15 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-18845. Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16271

[jira] [Created] (SPARK-12298) Infinite loop in DataFrame.sortWithinPartitions(String, String*)

2015-12-11 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-12298: -- Summary: Infinite loop in DataFrame.sortWithinPartitions(String, String*) Key: SPARK-12298 URL: https://issues.apache.org/jira/browse/SPARK-12298 Project: Spark

[jira] [Commented] (SPARK-2365) Add IndexedRDD, an efficient updatable key-value store

2015-11-10 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999745#comment-14999745 ] Ankur Dave commented on SPARK-2365: --- Here's the jar:

[jira] [Comment Edited] (SPARK-2365) Add IndexedRDD, an efficient updatable key-value store

2015-11-10 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999745#comment-14999745 ] Ankur Dave edited comment on SPARK-2365 at 11/11/15 1:15 AM: - Here's the jar:

[jira] [Commented] (SPARK-2365) Add IndexedRDD, an efficient updatable key-value store

2015-11-09 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997596#comment-14997596 ] Ankur Dave commented on SPARK-2365: --- It should work to use {code} amplab spark-indexedrdd

[jira] [Commented] (SPARK-3789) [GRAPHX] Python bindings for GraphX

2015-11-04 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990450#comment-14990450 ] Ankur Dave commented on SPARK-3789: --- I don't think there's any danger of graph analytics going away. On

[jira] [Created] (SPARK-11077) Join elimination in Catalyst

2015-10-12 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-11077: -- Summary: Join elimination in Catalyst Key: SPARK-11077 URL: https://issues.apache.org/jira/browse/SPARK-11077 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

2015-10-07 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947330#comment-14947330 ] Ankur Dave commented on SPARK-10945: Thanks [~kammar] and [~srowen] for the careful analysis. It does

[jira] [Resolved] (SPARK-3671) Batch multiput updates within partitions

2015-08-19 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-3671. --- Resolution: Fixed Issue resolved by https://github.com/amplab/spark-indexedrdd/pull/4 Batch

[jira] [Updated] (SPARK-3190) Creation of large graph( 2.15 B nodes) seems to be broken:possible overflow somewhere

2015-08-03 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3190: -- Target Version/s: 1.3.2, 1.4.2, 1.5.0 Creation of large graph( 2.15 B nodes) seems to be

[jira] [Reopened] (SPARK-3190) Creation of large graph( 2.15 B nodes) seems to be broken:possible overflow somewhere

2015-08-03 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave reopened SPARK-3190: --- Regression due to

[jira] [Assigned] (SPARK-9436) Simplify Pregel by merging joins

2015-07-29 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave reassigned SPARK-9436: - Assignee: Ankur Dave Simplify Pregel by merging joins

[jira] [Updated] (SPARK-9436) Simplify Pregel by merging joins

2015-07-29 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-9436: -- Assignee: Alexander Ulanov (was: Ankur Dave) Simplify Pregel by merging joins

[jira] [Resolved] (SPARK-9436) Simplify Pregel by merging joins

2015-07-29 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-9436. --- Resolution: Fixed Fix Version/s: (was: 1.4.0) 1.5.0 Issue resolved by

[jira] [Resolved] (SPARK-8718) Improve EdgePartition2D for non perfect square number of partitions

2015-07-14 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-8718. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7104

[jira] [Resolved] (SPARK-6736) [GraphX]Example of Graph#aggregateMessages has error

2015-04-07 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-6736. --- Resolution: Fixed Issue resolved by pull request 5388 [https://github.com/apache/spark/pull/5388]

[jira] [Resolved] (SPARK-6510) Add Graph#minus method to act as Set#difference

2015-03-26 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-6510. --- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5175

[jira] [Updated] (SPARK-6510) Add Graph#minus method to act as Set#difference

2015-03-26 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-6510: -- Assignee: Brennon York Add Graph#minus method to act as Set#difference

[jira] [Resolved] (SPARK-5922) Add diff(other: RDD[VertexId, VD]) in VertexRDD

2015-03-16 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-5922. --- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4733

[jira] [Updated] (SPARK-5922) Add diff(other: RDD[VertexId, VD]) in VertexRDD

2015-03-16 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-5922: -- Assignee: Brennon York Add diff(other: RDD[VertexId, VD]) in VertexRDD

[jira] [Commented] (SPARK-4086) Fold-style aggregation for VertexRDD

2015-03-14 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362203#comment-14362203 ] Ankur Dave commented on SPARK-4086: --- Thanks! Yeah, I was thinking about adding a new

[jira] [Updated] (SPARK-4086) Fold-style aggregation for VertexRDD

2015-03-14 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-4086: -- Assignee: (was: Ankur Dave) Fold-style aggregation for VertexRDD

[jira] [Commented] (SPARK-4600) org.apache.spark.graphx.VertexRDD.diff does not work

2015-03-09 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353627#comment-14353627 ] Ankur Dave commented on SPARK-4600: --- As I wrote in SPARK-6022, this is the documented

[jira] [Commented] (SPARK-6022) GraphX `diff` test incorrectly operating on values (not VertexId's)

2015-03-09 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353617#comment-14353617 ] Ankur Dave commented on SPARK-6022: --- [~maropu] is correct: the original intent of diff

[jira] [Resolved] (SPARK-3664) Graduate GraphX from alpha to stable

2015-02-26 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-3664. --- Resolution: Fixed Fix Version/s: 1.2.0 Graduate GraphX from alpha to stable

[jira] [Resolved] (SPARK-1955) VertexRDD can incorrectly assume index sharing

2015-02-25 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-1955. --- Resolution: Fixed Fix Version/s: 1.2.2 1.3.0 Assignee: Brennon York

[jira] [Updated] (SPARK-3290) No unpersist callls in SVDPlusPlus

2015-02-14 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3290: -- Assignee: Sean Owen No unpersist callls in SVDPlusPlus --

[jira] [Commented] (SPARK-1955) VertexRDD can incorrectly assume index sharing

2015-02-12 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319672#comment-14319672 ] Ankur Dave commented on SPARK-1955: --- [~boyork] Thanks, it would be great if you could

[jira] [Resolved] (SPARK-5343) ShortestPaths traverses backwards

2015-02-10 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-5343. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4478

[jira] [Commented] (SPARK-5461) Graph should have isCheckpointed, getCheckpointFiles methods

2015-01-29 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14297370#comment-14297370 ] Ankur Dave commented on SPARK-5461: --- [~josephkb] I'm not too familiar with how

[jira] [Created] (SPARK-5484) Pregel should checkpoint periodically to avoid StackOverflowError

2015-01-29 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-5484: - Summary: Pregel should checkpoint periodically to avoid StackOverflowError Key: SPARK-5484 URL: https://issues.apache.org/jira/browse/SPARK-5484 Project: Spark

[jira] [Resolved] (SPARK-3673) Move IndexedRDD from a pull request into a separate repository

2015-01-29 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-3673. --- Resolution: Fixed Move IndexedRDD from a pull request into a separate repository

[jira] [Resolved] (SPARK-5351) Can't zip RDDs with unequal numbers of partitions in ReplicatedVertexView.upgrade()

2015-01-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-5351. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-2823) GraphX jobs throw IllegalArgumentException

2015-01-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-2823. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-5064) GraphX rmatGraph hangs

2015-01-21 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-5064. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-4917) Add a function to convert into a graph with canonical edges in GraphOps

2015-01-08 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4917. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3760

[jira] [Resolved] (SPARK-4646) Replace Scala.util.Sorting.quickSort with Sorter(TimSort) in Spark

2014-12-07 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4646. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3507

[jira] [Resolved] (SPARK-4620) Add unpersist in Graph/GraphImpl

2014-12-07 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4620. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3476

[jira] [Updated] (SPARK-3623) Graph should support the checkpoint operation

2014-12-06 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3623: -- Assignee: Guoqiang Li Graph should support the checkpoint operation

[jira] [Commented] (SPARK-4659) Implement K-core decomposition algorithm

2014-12-06 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236695#comment-14236695 ] Ankur Dave commented on SPARK-4659: --- Great! Submit it it as a PR when you get the

[jira] [Created] (SPARK-4763) All-pairs shortest paths algorithm

2014-12-05 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-4763: - Summary: All-pairs shortest paths algorithm Key: SPARK-4763 URL: https://issues.apache.org/jira/browse/SPARK-4763 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-4672) Cut off the super long serialization chain in GraphX to avoid the StackOverflow error

2014-12-03 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232734#comment-14232734 ] Ankur Dave commented on SPARK-4672: --- [~jerrylead] Thanks for investigating this bug and

[jira] [Resolved] (SPARK-4672) Cut off the super long serialization chain in GraphX to avoid the StackOverflow error

2014-12-03 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4672. --- Resolution: Fixed Issue resolved by pull request 3545 [https://github.com/apache/spark/pull/3545]

[jira] [Created] (SPARK-4444) Drop VD type parameter from EdgeRDD

2014-11-16 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-: - Summary: Drop VD type parameter from EdgeRDD Key: SPARK- URL: https://issues.apache.org/jira/browse/SPARK- Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3206) Error in PageRank values

2014-11-12 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207822#comment-14207822 ] Ankur Dave commented on SPARK-3206: --- I just tested this with the standalone version of

[jira] [Resolved] (SPARK-3206) Error in PageRank values

2014-11-12 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-3206. --- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Ankur Dave Error in PageRank values

[jira] [Resolved] (SPARK-3667) Deprecate Graph#unpersistVertices and document how to correctly unpersist graphs

2014-11-12 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-3667. --- Resolution: Won't Fix Target Version/s: (was: 1.2.0) Deprecate Graph#unpersistVertices

[jira] [Updated] (SPARK-3665) Java API for GraphX

2014-11-12 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3665: -- Description: The Java API will wrap the Scala API in a similar manner as JavaRDD. Components will

[jira] [Resolved] (SPARK-4249) A problem of EdgePartitionBuilder in Graphx

2014-11-06 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4249. --- Resolution: Fixed Fix Version/s: 1.0.3 1.1.1 1.2.0 Issue

[jira] [Resolved] (SPARK-4173) EdgePartitionBuilder uses wrong value for first clustered index

2014-11-06 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4173. --- Resolution: Fixed Fix Version/s: 1.0.3 1.1.1 1.2.0 Issue

[jira] [Created] (SPARK-4173) EdgePartitionBuilder uses wrong value for first clustered index

2014-10-31 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-4173: - Summary: EdgePartitionBuilder uses wrong value for first clustered index Key: SPARK-4173 URL: https://issues.apache.org/jira/browse/SPARK-4173 Project: Spark

[jira] [Created] (SPARK-4086) Fold-style aggregation for VertexRDD

2014-10-24 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-4086: - Summary: Fold-style aggregation for VertexRDD Key: SPARK-4086 URL: https://issues.apache.org/jira/browse/SPARK-4086 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-4086) Fold-style aggregation for VertexRDD

2014-10-24 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-4086: -- Description: VertexRDD currently supports creations and joins only through a reduce-style interface

[jira] [Created] (SPARK-3818) Graph coarsening

2014-10-06 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3818: - Summary: Graph coarsening Key: SPARK-3818 URL: https://issues.apache.org/jira/browse/SPARK-3818 Project: Spark Issue Type: New Feature Components:

[jira] [Resolved] (SPARK-3635) Find Strongly Connected Components with Graphx has a small bug

2014-09-29 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-3635. --- Resolution: Fixed Fix Version/s: 1.1.1 1.2.0 Issue resolved by pull request

[jira] [Updated] (SPARK-3665) Java API for GraphX

2014-09-24 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3665: -- Description: The Java API will wrap the Scala API in a similar manner as JavaRDD. Components will

[jira] [Created] (SPARK-3664) Graduate GraphX from alpha to stable

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3664: - Summary: Graduate GraphX from alpha to stable Key: SPARK-3664 URL: https://issues.apache.org/jira/browse/SPARK-3664 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3665) Java API for GraphX

2014-09-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3665: -- Component/s: GraphX Java API for GraphX --- Key: SPARK-3665

[jira] [Created] (SPARK-3665) Java API for GraphX

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3665: - Summary: Java API for GraphX Key: SPARK-3665 URL: https://issues.apache.org/jira/browse/SPARK-3665 Project: Spark Issue Type: Improvement Reporter:

[jira] [Created] (SPARK-3666) Extract interfaces for EdgeRDD and VertexRDD

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3666: - Summary: Extract interfaces for EdgeRDD and VertexRDD Key: SPARK-3666 URL: https://issues.apache.org/jira/browse/SPARK-3666 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3667) Deprecate Graph#unpersistVertices and document how to correctly unpersist graphs

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3667: - Summary: Deprecate Graph#unpersistVertices and document how to correctly unpersist graphs Key: SPARK-3667 URL: https://issues.apache.org/jira/browse/SPARK-3667 Project:

[jira] [Updated] (SPARK-3665) Java API for GraphX

2014-09-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3665: -- Description: The Java API will wrap the Scala API in a similar manner as JavaRDD. Components will

[jira] [Updated] (SPARK-3664) Graduate GraphX from alpha to stable

2014-09-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3664: -- Description: The GraphX API is officially marked as alpha but has been moving toward stability. This

[jira] [Updated] (SPARK-3665) Java API for GraphX

2014-09-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3665: -- Description: The Java API will wrap the Scala API in a similar manner as JavaRDD. Components will

[jira] [Created] (SPARK-3668) Support for arbitrary key types in IndexedRDD

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3668: - Summary: Support for arbitrary key types in IndexedRDD Key: SPARK-3668 URL: https://issues.apache.org/jira/browse/SPARK-3668 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3668) Support for arbitrary key types in IndexedRDD

2014-09-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3668: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-2365 Support for arbitrary key types in

[jira] [Created] (SPARK-3669) Extract IndexedRDD interface

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3669: - Summary: Extract IndexedRDD interface Key: SPARK-3669 URL: https://issues.apache.org/jira/browse/SPARK-3669 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-3670) Add log-structured updates with merge

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3670: - Summary: Add log-structured updates with merge Key: SPARK-3670 URL: https://issues.apache.org/jira/browse/SPARK-3670 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-3671) Batch multiput updates within partitions

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3671: - Summary: Batch multiput updates within partitions Key: SPARK-3671 URL: https://issues.apache.org/jira/browse/SPARK-3671 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-3672) Add a non-updatable implementation for read performance

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3672: - Summary: Add a non-updatable implementation for read performance Key: SPARK-3672 URL: https://issues.apache.org/jira/browse/SPARK-3672 Project: Spark Issue Type:

[jira] [Created] (SPARK-3673) Move IndexedRDD from a pull request into a separate repository

2014-09-23 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-3673: - Summary: Move IndexedRDD from a pull request into a separate repository Key: SPARK-3673 URL: https://issues.apache.org/jira/browse/SPARK-3673 Project: Spark

[jira] [Commented] (SPARK-2365) Add IndexedRDD, an efficient updatable key-value store

2014-09-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145698#comment-14145698 ] Ankur Dave commented on SPARK-2365: --- [~imranr] Thanks for the comments and

[jira] [Updated] (SPARK-3665) Java API for GraphX

2014-09-23 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3665: -- Component/s: Java API Java API for GraphX --- Key: SPARK-3665

  1   2   >