Re: [jira] [Assigned] (APEXMALHAR-2303) S3 Line By Line Module

2016-10-18 Thread AJAY GUPTA
Hi Apex Dev community, Kindly provide with feedback if any for the following approach for implementing S3RecordReader. *S3RecordReader(delimited records)* *Input *: BlockMetaData containing offset and length *Expected Output :* Records in the block *Approach : * Similar to approach currently bein

[jira] [Resolved] (APEXMALHAR-2263) Offsets in AbstractFileInputOperator should be long rather than int

2016-10-18 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved APEXMALHAR-2263. -- Resolution: Fixed Fix Version/s: 3.6.0 > Offsets in AbstractFileInputOperato

[jira] [Commented] (APEXMALHAR-2263) Offsets in AbstractFileInputOperator should be long rather than int

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587803#comment-15587803 ] ASF GitHub Bot commented on APEXMALHAR-2263: Github user asfgit closed th

[GitHub] apex-malhar pull request #456: APEXMALHAR-2263 change offset in AbstractFile...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/456 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [KUDOS] Contributed runner: Apache Apex!

2016-10-18 Thread Chinmay Kolhatkar
Very nice ..:) I would like to contribute as well. Thanks, Chinmay. On Tue, Oct 18, 2016 at 2:19 AM, David Yan wrote: > I would also like to contribute. Thanks! > > David > > On Mon, Oct 17, 2016 at 10:01 AM, Thomas Weise wrote: > > > FYI, the first cut of Apex runner for Beam is in a feature

[jira] [Created] (APEXMALHAR-2308) BlockReader must consider fixed width length in emitted block alignment.

2016-10-18 Thread Deepak Narkhede (JIRA)
Deepak Narkhede created APEXMALHAR-2308: --- Summary: BlockReader must consider fixed width length in emitted block alignment. Key: APEXMALHAR-2308 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2308

[jira] [Commented] (APEXMALHAR-2190) Use reusable buffer to serial spillable data structure

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587668#comment-15587668 ] ASF GitHub Bot commented on APEXMALHAR-2190: GitHub user brightchen reope

[jira] [Commented] (APEXMALHAR-2190) Use reusable buffer to serial spillable data structure

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587667#comment-15587667 ] ASF GitHub Bot commented on APEXMALHAR-2190: Github user brightchen close

[GitHub] apex-malhar pull request #404: APEXMALHAR-2190 #resolve #comment Use reusabl...

2016-10-18 Thread brightchen
Github user brightchen closed the pull request at: https://github.com/apache/apex-malhar/pull/404 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] apex-malhar pull request #404: APEXMALHAR-2190 #resolve #comment Use reusabl...

2016-10-18 Thread brightchen
GitHub user brightchen reopened a pull request: https://github.com/apache/apex-malhar/pull/404 APEXMALHAR-2190 #resolve #comment Use reusable buffer to serial spill… …able data structure You can merge this pull request into a Git repository by running: $ git pull https://gi

[jira] [Assigned] (APEXMALHAR-2303) S3 Line By Line Module

2016-10-18 Thread Ajay Gupta (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Gupta reassigned APEXMALHAR-2303: -- Assignee: Ajay Gupta > S3 Line By Line Module > -- > >

[jira] [Updated] (APEXMALHAR-2306) Tests should allow for additions to OperatorContext interface

2016-10-18 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-2306: - Summary: Tests should allow for additions to OperatorContext interface (was: malhar

[jira] [Commented] (APEXMALHAR-2306) malhar unit test(s) have a dependency on OperatorContext in apex-core

2016-10-18 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587351#comment-15587351 ] Thomas Weise commented on APEXMALHAR-2306: -- Have a look at FSInputModuleTest

[jira] [Updated] (APEXMALHAR-2307) Session windows are not deleted properly after merge or extend in InMemorySessionWindowedStorage

2016-10-18 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan updated APEXMALHAR-2307: -- Description: The code does not remove any windows from the keyToWindows map. It should over

[jira] [Created] (APEXMALHAR-2307) Session windows are not deleted properly after merge or extend in InMemorySessionWindowedStorage

2016-10-18 Thread David Yan (JIRA)
David Yan created APEXMALHAR-2307: - Summary: Session windows are not deleted properly after merge or extend in InMemorySessionWindowedStorage Key: APEXMALHAR-2307 URL: https://issues.apache.org/jira/browse/APEXMAL

[jira] [Commented] (APEXMALHAR-2306) malhar unit test(s) have a dependency on OperatorContext in apex-core

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587143#comment-15587143 ] ASF GitHub Bot commented on APEXMALHAR-2306: Github user sanjaypujare clo

[GitHub] apex-malhar pull request #459: APEXMALHAR-2306 fix OperatorContext implement...

2016-10-18 Thread sanjaypujare
Github user sanjaypujare closed the pull request at: https://github.com/apache/apex-malhar/pull/459 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[jira] [Updated] (APEXMALHAR-2306) malhar unit test(s) have a dependency on OperatorContext in apex-core

2016-10-18 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXMALHAR-2306: --- Description: While trying to build/test malhar against the latest apex-core, the followin

[jira] [Commented] (APEXMALHAR-2306) malhar unit test(s) have a dependency on OperatorContext in apex-core

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587040#comment-15587040 ] ASF GitHub Bot commented on APEXMALHAR-2306: GitHub user sanjaypujare ope

[GitHub] apex-malhar pull request #459: APEXMALHAR-2306 fix OperatorContext implement...

2016-10-18 Thread sanjaypujare
GitHub user sanjaypujare opened a pull request: https://github.com/apache/apex-malhar/pull/459 APEXMALHAR-2306 fix OperatorContext implementations to add the new required method @vrozov these changes were required to run malhar successfully against latest apex-core. This is just

[jira] [Created] (APEXMALHAR-2306) malhar unit test(s) have a dependency on OperatorContext in apex-core

2016-10-18 Thread Sanjay M Pujare (JIRA)
Sanjay M Pujare created APEXMALHAR-2306: --- Summary: malhar unit test(s) have a dependency on OperatorContext in apex-core Key: APEXMALHAR-2306 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2306

[jira] [Commented] (APEXMALHAR-2305) Change implementation of session window to reflect what is described in streaming 102 blog

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586908#comment-15586908 ] ASF GitHub Bot commented on APEXMALHAR-2305: GitHub user davidyan74 opene

[GitHub] apex-malhar pull request #458: APEXMALHAR-2305 #resolve Mirror the proto-ses...

2016-10-18 Thread davidyan74
GitHub user davidyan74 opened a pull request: https://github.com/apache/apex-malhar/pull/458 APEXMALHAR-2305 #resolve Mirror the proto-session window behavior des… …cribed in the streaming 102 blog @siyuanh please review and merge You can merge this pull request into a

[jira] [Created] (APEXMALHAR-2305) Change implementation of session window to reflect what is described in streaming 102 blog

2016-10-18 Thread David Yan (JIRA)
David Yan created APEXMALHAR-2305: - Summary: Change implementation of session window to reflect what is described in streaming 102 blog Key: APEXMALHAR-2305 URL: https://issues.apache.org/jira/browse/APEXMALHAR-23

[jira] [Commented] (APEXCORE-561) Container info needs to be persisted even after application has been killed

2016-10-18 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586503#comment-15586503 ] David Yan commented on APEXCORE-561: This is a good feature to have to diagnostics.

[jira] [Resolved] (APEXMALHAR-2299) TimeBasedDedupOperator throws exception during time bucket assignment in certain edge cases

2016-10-18 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh resolved APEXMALHAR-2299. --- Resolution: Fixed > TimeBasedDedupOperator throws exception during time bucket as

[jira] [Commented] (APEXCORE-510) Enforce DefaultOutputPort.emit() or Sink.put() thread affinity

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586280#comment-15586280 ] ASF GitHub Bot commented on APEXCORE-510: - GitHub user sanjaypujare opened a pul

[GitHub] apex-core pull request #411: APEXCORE-510 Implement check in DefaultOutputPo...

2016-10-18 Thread sanjaypujare
GitHub user sanjaypujare opened a pull request: https://github.com/apache/apex-core/pull/411 APEXCORE-510 Implement check in DefaultOutputPort for thread affinity between setup and emit @vrozov pls review and merge as appropriate You can merge this pull request into a Git reposit

[jira] [Commented] (APEXMALHAR-2299) TimeBasedDedupOperator throws exception during time bucket assignment in certain edge cases

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586277#comment-15586277 ] ASF GitHub Bot commented on APEXMALHAR-2299: Github user asfgit closed th

[GitHub] apex-malhar pull request #454: APEXMALHAR-2299 Modified to move window when ...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/454 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (APEXMALHAR-2304) Apex SQL: Add examples for SQL in Apex in demos folder

2016-10-18 Thread Chinmay Kolhatkar (JIRA)
Chinmay Kolhatkar created APEXMALHAR-2304: - Summary: Apex SQL: Add examples for SQL in Apex in demos folder Key: APEXMALHAR-2304 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2304 Proje

[jira] [Comment Edited] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-18 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585751#comment-15585751 ] Chandni Singh edited comment on APEXMALHAR-2284 at 10/18/16 3:40 PM: --

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-18 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585751#comment-15585751 ] Chandni Singh commented on APEXMALHAR-2284: --- One concept that I think I sho

[GitHub] apex-malhar pull request #439: APEXMALHAR-2272 : Fixed sequentialFileRead on...

2016-10-18 Thread yogidevendra
Github user yogidevendra closed the pull request at: https://github.com/apache/apex-malhar/pull/439 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[jira] [Commented] (APEXMALHAR-2272) sequentialFileRead property on FSInputModule not functioning as expected

2016-10-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585644#comment-15585644 ] ASF GitHub Bot commented on APEXMALHAR-2272: Github user yogidevendra clo

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-18 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585604#comment-15585604 ] Bhupesh Chawda commented on APEXMALHAR-2284: As far as I understand, the

[GitHub] apex-malhar pull request #457: APEXMALHAR 2302 Add blockSize property to FSR...

2016-10-18 Thread deepak-narkhede
GitHub user deepak-narkhede reopened a pull request: https://github.com/apache/apex-malhar/pull/457 APEXMALHAR 2302 Add blockSize property to FSRecordReaderModule This change adds blockSize property from FileSplitter to FSRecordReaderModule. Tested with RecordReader Application.

[GitHub] apex-malhar pull request #457: APEXMALHAR 2302 Add blockSize property to FSR...

2016-10-18 Thread deepak-narkhede
Github user deepak-narkhede closed the pull request at: https://github.com/apache/apex-malhar/pull/457 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[jira] [Created] (APEXMALHAR-2303) S3 Line By Line Module

2016-10-18 Thread Ajay Gupta (JIRA)
Ajay Gupta created APEXMALHAR-2303: -- Summary: S3 Line By Line Module Key: APEXMALHAR-2303 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2303 Project: Apache Apex Malhar Issue Type: B