[GitHub] jerrypeng commented on issue #1771: setting auto ack to be true for sources and sinks

jerrypeng commented on issue #1771: setting auto ack to be true for sources and sinks URL: https://github.com/apache/incubator-pulsar/pull/1771#issuecomment-388528354 retest this please This is an automated message from the

[GitHub] jerrypeng commented on issue #1770: remove unnecessary log message in sink cmd

jerrypeng commented on issue #1770: remove unnecessary log message in sink cmd URL: https://github.com/apache/incubator-pulsar/pull/1770#issuecomment-388528334 retest this please This is an automated message from the Apache

[GitHub] jerrypeng commented on issue #1769: refactor source and sink classname to be pulsar source and sink when not set

jerrypeng commented on issue #1769: refactor source and sink classname to be pulsar source and sink when not set URL: https://github.com/apache/incubator-pulsar/pull/1769#issuecomment-388528292 retest this please This is an

[GitHub] jerrypeng opened a new pull request #1771: setting auto ack to be true for sources and sinks

jerrypeng opened a new pull request #1771: setting auto ack to be true for sources and sinks URL: https://github.com/apache/incubator-pulsar/pull/1771 This is an automated message from the Apache Git Service. To respond to

[GitHub] merlimat commented on issue #1764: WIP - Pulsar Go client library

merlimat commented on issue #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#issuecomment-388510252 @bruth Addressed most of the comments This is an automated message from the

[GitHub] jerrypeng opened a new pull request #1770: remove unnecessary log message in sink cmd

jerrypeng opened a new pull request #1770: remove unnecessary log message in sink cmd URL: https://github.com/apache/incubator-pulsar/pull/1770 This is an automated message from the Apache Git Service. To respond to the

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187753302 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187753201 ## File path: pulsar-client-go/examples/reader/reader.go ## @@ -0,0 +1,48 @@ +// +// Licensed to the Apache

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187752900 ## File path: pulsar-client-go/examples/consumer/consumer.go ## @@ -0,0 +1,54 @@ +// +// Licensed to the

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187752774 ## File path: pulsar-client-go/examples/consumer/consumer.go ## @@ -0,0 +1,54 @@ +// +// Licensed to the

[GitHub] jerrypeng commented on issue #1769: refactor source and sink classname to be pulsar source and sink when not set

jerrypeng commented on issue #1769: refactor source and sink classname to be pulsar source and sink when not set URL: https://github.com/apache/incubator-pulsar/pull/1769#issuecomment-388508529 @srkukarni @sijie please review

[GitHub] jerrypeng opened a new pull request #1769: refactor source and sink classname to be pulsar source and sink when not set

jerrypeng opened a new pull request #1769: refactor source and sink classname to be pulsar source and sink when not set URL: https://github.com/apache/incubator-pulsar/pull/1769 This is an automated message from the

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187752595 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] jerrypeng commented on a change in pull request #1758: Windowing for Pulsar Functions

jerrypeng commented on a change in pull request #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#discussion_r187732990 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstance.java

[GitHub] srkukarni commented on a change in pull request #1758: Windowing for Pulsar Functions

srkukarni commented on a change in pull request #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#discussion_r187732720 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstance.java

[GitHub] jerrypeng commented on issue #1758: Windowing for Pulsar Functions

jerrypeng commented on issue #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#issuecomment-388469042 @srkukarni thanks for the review. I have addressed your comments This is an

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388435528 I discussed with @srkukarni and we are going to leave the KakfaSource as it is for now

[GitHub] jerrypeng commented on issue #1758: Windowing for Pulsar Functions

jerrypeng commented on issue #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#issuecomment-388434193 @srkukarni i think that is a good approach. Let me refactor the code This is

[GitHub] ivankelly commented on issue #1768: InputStream to read from S3

ivankelly commented on issue #1768: InputStream to read from S3 URL: https://github.com/apache/incubator-pulsar/pull/1768#issuecomment-388395548 retest this please // V1_ReplicatorTest.testCloseReplicatorStartProducer This

[GitHub] ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader

ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader URL: https://github.com/apache/incubator-pulsar/pull/1746#discussion_r187596277 ## File path:

[GitHub] ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader

ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader URL: https://github.com/apache/incubator-pulsar/pull/1746#discussion_r187597507 ## File path:

[GitHub] ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader

ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader URL: https://github.com/apache/incubator-pulsar/pull/1746#discussion_r187597119 ## File path:

[GitHub] ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader

ivankelly commented on a change in pull request #1746: PIP-17: impl offload() for S3ManagedLedgerOffloader URL: https://github.com/apache/incubator-pulsar/pull/1746#discussion_r187597447 ## File path:

[GitHub] ivankelly opened a new pull request #1768: InputStream to read from S3

ivankelly opened a new pull request #1768: InputStream to read from S3 URL: https://github.com/apache/incubator-pulsar/pull/1768 Reads from S3 in configurable chunks. Interface allows caller to seek to an arbitrary position. Master Issue: #1511

[GitHub] ivankelly commented on issue #1134: Docker compose files for pulsar

ivankelly commented on issue #1134: Docker compose files for pulsar URL: https://github.com/apache/incubator-pulsar/pull/1134#issuecomment-388326182 @merlimat @aahmed-se I reworked these when checking 2.0 rc3. I think they're ready to merge now.

[GitHub] bruth commented on a change in pull request #1764: WIP - Pulsar Go client library

bruth commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187572451 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] bruth commented on a change in pull request #1764: WIP - Pulsar Go client library

bruth commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187570833 ## File path: pulsar-client-go/examples/reader/reader.go ## @@ -0,0 +1,48 @@ +// +// Licensed to the Apache

[GitHub] bruth commented on a change in pull request #1764: WIP - Pulsar Go client library

bruth commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187570569 ## File path: pulsar-client-go/examples/reader/reader.go ## @@ -0,0 +1,48 @@ +// +// Licensed to the Apache

[GitHub] bruth commented on a change in pull request #1764: WIP - Pulsar Go client library

bruth commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187570215 ## File path: pulsar-client-go/examples/reader/reader.go ## @@ -0,0 +1,48 @@ +// +// Licensed to the Apache

[GitHub] bruth commented on a change in pull request #1764: WIP - Pulsar Go client library

bruth commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187566247 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] bruth commented on a change in pull request #1764: WIP - Pulsar Go client library

bruth commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187565770 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[incubator-pulsar] branch asf-site updated: Updated site at revision e2b95f2

This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 18e9728 Updated site at revision

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187536043 ## File path: pulsar-client-go/examples/reader/reader.go ## @@ -0,0 +1,48 @@ +// +// Licensed to the Apache

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187535824 ## File path: pulsar-client-go/examples/reader/reader.go ## @@ -0,0 +1,48 @@ +// +// Licensed to the Apache

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187535910 ## File path: pulsar-client-go/examples/reader/reader.go ## @@ -0,0 +1,48 @@ +// +// Licensed to the Apache

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187535665 ## File path: pulsar-client-go/examples/consumer/consumer.go ## @@ -0,0 +1,54 @@ +// +// Licensed to the

[GitHub] srkukarni commented on issue #1766: Implementing PushSource on top of Source

srkukarni commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388279890 Not really. Infact this is closer to Storm's KafkaSpout which is a pull model

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388279505 @srkukarni but isn't that just the same as using the PushSource but instead you a pulling the queuing logic back into the

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187534640 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187534501 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] srkukarni commented on issue #1766: Implementing PushSource on top of Source

srkukarni commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388278263 So the main interface is the Source<> interface which has a read() blocking call. Since kafka is also a pull interface, kafkasource

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187534072 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187533673 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187533434 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] srkukarni commented on issue #1758: Windowing for Pulsar Functions

srkukarni commented on issue #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#issuecomment-388277539 Can we keep the simple Function(Collection<>) interface that we have and still implement WindowExecutor client side? That way in

[GitHub] merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library

merlimat commented on a change in pull request #1764: WIP - Pulsar Go client library URL: https://github.com/apache/incubator-pulsar/pull/1764#discussion_r187533355 ## File path: pulsar-client-go/examples/consumer-listener/consumer-listener.go ## @@ -0,0 +1,55 @@ +// +//

[GitHub] jerrypeng commented on issue #1758: Windowing for Pulsar Functions

jerrypeng commented on issue #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#issuecomment-388275692 @srkukarni well I thought you didn't like that approach since WindowFunction would be an abstract class that users would extend. It would

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388277056 @srkukarni using a push source might be more efficient since not we are doing async reads in a sense. Remember the actual source

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388277056 @srkukarni using a push source might be more efficient since not we are doing async reads in a sense. Remember the actual source

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388277056 @srkukarni using a push source might be more efficient since not we are doing async reads in a sense. Remember the actual source

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388277056 @srkukarni user a push source might be more efficient since not we are doing async reads in a sense. Remember the actual source

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388276631 @srkukarni are you suggesting we poll one tuple at a time? e.g. onsumerRecords = consumer.poll(1); is that

[GitHub] jerrypeng commented on issue #1758: Windowing for Pulsar Functions

jerrypeng commented on issue #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#issuecomment-388275692 @srkukarni well I thought you didn't like that approach since WindowFunction would be an abstract method that users would extend. It would

[GitHub] srkukarni commented on issue #1766: Implementing PushSource on top of Source

srkukarni commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388275286 Since this is a smallish change, would rather do it here itself? It actually simpifies the code of kafka source

[GitHub] srkukarni commented on issue #1758: Windowing for Pulsar Functions

srkukarni commented on issue #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#issuecomment-388275138 One approach would be to make WindowExecutor function client side? This is an

[GitHub] jerrypeng commented on a change in pull request #1758: Windowing for Pulsar Functions

jerrypeng commented on a change in pull request #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#discussion_r187530968 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstance.java

[GitHub] jerrypeng commented on a change in pull request #1758: Windowing for Pulsar Functions

jerrypeng commented on a change in pull request #1758: Windowing for Pulsar Functions URL: https://github.com/apache/incubator-pulsar/pull/1758#discussion_r187530714 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstance.java

[GitHub] jerrypeng commented on a change in pull request #1766: Implementing PushSource on top of Source

jerrypeng commented on a change in pull request #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#discussion_r187530060 ## File path: pulsar-io/kafka/src/main/java/org/apache/pulsar/io/kafka/KafkaSource.java ## @@

[GitHub] srkukarni commented on a change in pull request #1766: Implementing PushSource on top of Source

srkukarni commented on a change in pull request #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#discussion_r187529796 ## File path: pulsar-io/kafka/src/main/java/org/apache/pulsar/io/kafka/KafkaSource.java ## @@

[GitHub] jerrypeng commented on issue #1767: check record returned by source is not null

jerrypeng commented on issue #1767: check record returned by source is not null URL: https://github.com/apache/incubator-pulsar/pull/1767#issuecomment-388272601 @srkukarni @sijie please review This is an automated message

[GitHub] jerrypeng opened a new pull request #1767: check record returned by source is not null

jerrypeng opened a new pull request #1767: check record returned by source is not null URL: https://github.com/apache/incubator-pulsar/pull/1767 This is an automated message from the Apache Git Service. To respond to the

[GitHub] jerrypeng commented on issue #1766: Implementing PushSource on top of Source

jerrypeng commented on issue #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388271445 @sijie @srkukarni please review This is an automated message from

[GitHub] jerrypeng opened a new pull request #1766: Implementing PushSource on top of Source

jerrypeng opened a new pull request #1766: Implementing PushSource on top of Source URL: https://github.com/apache/incubator-pulsar/pull/1766 Changes: 1. Modified the PushSource interface to change to an abstract function 2. Change setConsumer(Function