Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
I can't commit on the timelines. I don't want to block things at my end.
-Sameer.
On Thu, Feb 16, 2017 at 1:23 AM, Marcelo Vanzin <notificati...@github.com>
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Hi Marcelo,
Are you referring to following commit.
5645533(
https://github.com/apache/spark/pull/16387/commits/5645533ede8bb37aa316afbc0d003d042bf5d52d
)
-Sameer
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Please share the commit id...i think that will make it clear. I can't share
the code as its propriety to organization,. but i am just reducing a map phase,
reduce by key and reducebykeyandwindow
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Hi liang,
I got pulled into another project. Will need few more days to catch up on
bandwidth.
Just for confirmation, I will testing with changes just to TaskManager
class
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
@mridulm <https://github.com/mridulm> - need one clarification: if I test
it with out ut @viirya <https://github.com/viirya> commit b1ef9ec,wouldnt
the commit fail at the assert
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
ok..I will get back to you in next couple of days.
-Sameer.
On Thu, Jan 26, 2017 at 3:56 AM, Marcelo Vanzin <notificati...@github.com>
wrote:
> Nope, I didn
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Nope, I didn't tested it in isolation.
-Sameer.
On Jan 24, 2017 10:09 PM, "Marcelo Vanzin" <notificati...@github.com> wrote:
> No the question is whether
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Hi,
No, I havent test this with out the ExternalAppendOnlyMap fix if this is
what you are asking. I tested it with changes to both the classes:
TaskMemoryManager
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16603
Combined with 16387.
On Jan 18, 2017 3:54 PM, "Liang-Chi Hsieh" <notificati...@github.com> wrote:
> @samkum <https://github.com/samkum> Just this fi
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16603
This fix also has GC issues. Please find the screenshot attached.
---
If your
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
The fix is better but still produces visible GC issues. After removing the
fix, GC is no longer a problem.
