Re: APEXMALHAR-1701 Deduper in Malhar

Hi Bhupesh, Dedup is different with regard to state accumulation. For other windowed operations, we collect state and then emit a result after a period of time (trigger or watermark). Here, we only need the state to detect the duplicate. Hence, it is inefficient to collect a list of tuples to dete

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71096442 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedPlainStorage.java --- @@ -0,0 +1,135 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71096391 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71096140 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71096090 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71095332 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71095327 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71095316 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71095282 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71095303 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[GitHub] apex-malhar pull request #345: REVIEW ONLY (WindowedOperator): splitting Win...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/345#discussion_r71095257 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/SpillableWindowedKeyedStorage.java --- @@ -0,0 +1,136 @@ +package org.apache.

[jira] [Created] (APEXCORE-493) Purge the checkpoints from the containers

Sandesh created APEXCORE-493: Summary: Purge the checkpoints from the containers Key: APEXCORE-493 URL: https://issues.apache.org/jira/browse/APEXCORE-493 Project: Apache Apex Core Issue Type: Im

[jira] [Resolved] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh resolved APEXCORE-222. -- Resolution: Fixed Fix Version/s: 3.5.0 > Delegate Buffer Server purge to StreamingContainer > -

[GitHub] apex-site pull request #42: Added updating documentation to release process

Github user tweise commented on a diff in the pull request: https://github.com/apache/apex-site/pull/42#discussion_r71090145 --- Diff: src/md/release.md --- @@ -150,6 +150,47 @@ mkdir svn-dist && cp *-source-* svn-dist/ svn import svn-dist https://dist.apache.org/repos/dist/de

Solving JIRA ISUUE : APEXMALHAR-2016.

Dear Community, We are final year students of Computer Engineering and are Interested to work on Apache Apex. As a exercise to understand the contribution process, we are planning to fix this ISSUE. Here is what we have understood from the JIRA:* APEXMALHAR-2016*. *Problem:* HDFS allows "," as p

[GitHub] apex-site issue #42: Added updating documentation to release process

Github user PramodSSImmaneni commented on the issue: https://github.com/apache/apex-site/pull/42 @sashadt @tweise addressed using major.minor version and other comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If