[GitHub] samza pull request #857: SAMZA-2041: add hdfs and kinesis descriptor

2018-12-11 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/857 SAMZA-2041: add hdfs and kinesis descriptor You can merge this pull request into a Git repository by running: $ git pull https://github.com/lhaiesp/samza master Alternatively you can review

[GitHub] samza pull request #809: SAMZA-2000: update contributor page

2018-11-16 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/809 SAMZA-2000: update contributor page Trivial commit to test committer workflow You can merge this pull request into a Git repository by running: $ git pull https://github.com/lhaiesp/samza

[GitHub] samza pull request #685: samza documentation: hdfs and eventhubs connector

2018-10-02 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/685 samza documentation: hdfs and eventhubs connector You can merge this pull request into a Git repository by running: $ git pull https://github.com/lhaiesp/samza master Alternatively you can

[GitHub] samza pull request #664: SAMZA-1914: fix out of range starting offset in EH ...

2018-09-26 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/664 SAMZA-1914: fix out of range starting offset in EH consumer You can merge this pull request into a Git repository by running: $ git pull https://github.com/lhaiesp/samza master Alternatively

[GitHub] samza pull request #633: SAMZA-1870: hdfs offset comparator to handle end of...

2018-09-11 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/633 SAMZA-1870: hdfs offset comparator to handle end of stream offset This happens particularly when using HDFS as a bootstrap stream: org.apache.samza.SamzaException: Invalid offset for

[GitHub] samza pull request #592: SAMZA-1794: setting application acl in launch conte...

2018-07-31 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/592 SAMZA-1794: setting application acl in launch context Currently we don't set application acl for container launch context. See https://hadoop.apache.org/docs/r2.6.4/api/org/apache/hadoop/yar

[GitHub] samza pull request #587: add retry logic in eventhubs system consumer for no...

2018-07-27 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/587 add retry logic in eventhubs system consumer for non transient error Implement a retry logic in EH system consumer because of lack of nurse job on azure and lack of retry logic in samza standlone

[GitHub] samza pull request #581: SAMZA-1785: ignore timeout exception during receive...

2018-07-26 Thread lhaiesp
Github user lhaiesp closed the pull request at: https://github.com/apache/samza/pull/581 ---

[GitHub] samza pull request #581: SAMZA-1785: ignore timeout exception during receive...

2018-07-25 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/581 SAMZA-1785: ignore timeout exception during receiver renew in eventhubs consumer You can merge this pull request into a Git repository by running: $ git pull https://github.com/lhaiesp/samza

[GitHub] samza pull request #571: SAMZA-1775: add some delay before renew under trans...

2018-07-19 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/571 SAMZA-1775: add some delay before renew under transient EH exception There is no delay at all before we renew the partition. This sometimes lead to spam in the log for the following messages

[GitHub] samza pull request #548: SAMZA-1741: fix issue that EH consumer taking too l...

2018-06-06 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/548 SAMZA-1741: fix issue that EH consumer taking too long to shutdown 1. lower the shutdown timeout from 1 min to 15 seconds 2. make sure EventHubManagers are shutdown in parallel 3. print a

[GitHub] samza pull request #511: SAMZA-1706 lazy initialization for eventhub system ...

2018-05-08 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/511 SAMZA-1706 lazy initialization for eventhub system producer We are seeing slow shutdown issue for eventhub system producers for users who only use eventhub consumer (but then Samza system creates

[GitHub] samza pull request #489: SAMZA-1688 use per partition eventhubs client

2018-04-25 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/489 SAMZA-1688 use per partition eventhubs client You can merge this pull request into a Git repository by running: $ git pull https://github.com/lhaiesp/samza master Alternatively you can review

[GitHub] samza pull request #484: SAMZA-1676 miscellaneous fix and improvement for ev...

2018-04-23 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/484 SAMZA-1676 miscellaneous fix and improvement for eventhubs system Including these changes: - Log the metadata that we are fetching from the event hubs - Rename readLatency to consumptionLagMs

[GitHub] samza pull request #470: minor fix on eventhubs size limit for event body an...

2018-04-12 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/470 minor fix on eventhubs size limit for event body and partition key You can merge this pull request into a Git repository by running: $ git pull https://github.com/lhaiesp/samza master

[GitHub] samza pull request #452: SAMZA-1623: include avro as the file suffix for hdf...

2018-03-21 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/452 SAMZA-1623: include avro as the file suffix for hdfs producer AvroDataFileHdfsWriter should include avro as the file suffix as some pig jobs couldn't read the avro files if they don't com

[GitHub] samza pull request #449: SAMZA-1622: avro writer to support generic record

2018-03-15 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/449 SAMZA-1622: avro writer to support generic record avro writer in HDFS system producer to support generic record You can merge this pull request into a Git repository by running: $ git pull

[GitHub] samza pull request #447: fix HdfsFileSystemAdapter to get files recursively

2018-03-13 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/447 fix HdfsFileSystemAdapter to get files recursively fix HdfsFileSystemAdapter to get files recursively You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] samza pull request #342: SAMZA-1471: SystemConsumers should not poll ssp tha...

2017-10-26 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/342 SAMZA-1471: SystemConsumers should not poll ssp that hit end of stream When SystemConsumers poll from SSPs that have hit end of stream, obviously there will be no data return and the poll will

[GitHub] samza pull request #329: SAMZA-1463 disable some flaky tests on hdfs system

2017-10-16 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/329 SAMZA-1463 disable some flaky tests on hdfs system disable some flaky tests on hdfs system until future investigation You can merge this pull request into a Git repository by running: $ git

[GitHub] samza pull request #151: fix HdfsSystemAdmin when staging directory is empty

2017-05-01 Thread lhaiesp
GitHub user lhaiesp opened a pull request: https://github.com/apache/samza/pull/151 fix HdfsSystemAdmin when staging directory is empty getSystemStreamMetadata has the potential side effect to persist metadata to a staging directory on hdfs. This could fail if staging directory is