[GitHub] [pulsar] jiazhai opened a new issue #3777: Documentation: link in standalone page failed
jiazhai opened a new issue #3777: Documentation: link in standalone page failed URL: https://github.com/apache/pulsar/issues/3777 http://pulsar.apache.org/docs/en/standalone/  This is because in 2.3.0 each connector is packaged into a separate nar. We need to change the link from old: `https://archive.apache.org/dist/pulsar/pulsar-2.3.0/apache-pulsar-io-connectors-2.3.0-bin.tar.gz` into `https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download=pulsar/pulsar-2.3.0/connectors/` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni merged pull request #3722: Retry creation of assignment topic a few times before giving up
srkukarni merged pull request #3722: Retry creation of assignment topic a few times before giving up URL: https://github.com/apache/pulsar/pull/3722 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext
srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext URL: https://github.com/apache/pulsar/pull/3717#issuecomment-470417529 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3671: pulsar-admin topics list public/default fails
merlimat commented on issue #3671: pulsar-admin topics list public/default fails URL: https://github.com/apache/pulsar/issues/3671#issuecomment-470415406 Fix is in #3776 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470412358 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] skyrocknroll commented on issue #3766: Support passing schema definition for JSON and AVRO schemas
skyrocknroll commented on issue #3766: Support passing schema definition for JSON and AVRO schemas URL: https://github.com/apache/pulsar/pull/3766#issuecomment-470411918 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] skyrocknroll opened a new issue #3671: pulsar-admin topics list public/default fails
skyrocknroll opened a new issue #3671: pulsar-admin topics list public/default fails URL: https://github.com/apache/pulsar/issues/3671 **Describe the bug** pulsar-admin topics list public/default Os Ubuntu 16 04 Pulsar 2.3.0 Failed with following error in log ``` [2019-02-22T13:59:00.704Z] INFO pulsar-web-29-1 RequestLog:60 10.10.58.5 - - [22/Feb/2019:13:59:00 +] "GET /admin/clusters/stage HTTP/1.1" 200 224 "-" "python-requests/2.19.1" 2 [2019-02-22T13:59:00.746Z] INFO pulsar-web-29-3 RequestLog:60 10.10.58.5 - - [22/Feb/2019:13:59:00 +] "GET /admin/broker-stats/destinations HTTP/1.1" 200 1266 "-" "python-requests/2.19.1" 2 [2019-02-22T13:59:05.706Z] INFO pulsar-web-29-4 RequestLog:60 10.10.58.5 - - [22/Feb/2019:13:59:05 +] "GET /admin/clusters/stage HTTP/1.1" 200 224 "-" "python-requests/2.19.1" 2 [2019-02-22T13:59:05.750Z] INFO pulsar-web-29-3 RequestLog:60 10.10.58.5 - - [22/Feb/2019:13:59:05 +] "GET /admin/broker-stats/destinations HTTP/1.1" 200 1266 "-" "python-requests/2.19.1" 1 [2019-02-22T13:59:06.354Z] INFO pulsar-web-29-1 RequestLog:60 127.0.0.1 - - [22/Feb/2019:13:59:06 +] "GET /admin/v2/persistent/public/default HTTP/1.1" 200 396 "-" "Jersey/2.27 (HttpUrlConnection 1.8.0_201)" 8 [2019-02-22T14:00:06.407Z] INFO pulsar-web-29-4 RequestLog:60 10.10.58.5 - - [22/Feb/2019:14:00:06 +] "GET /admin/brokers/stage HTTP/1.1" 200 91 "-" "python-requests/2.19.1" 3 [2019-02-22T14:01:01.902Z] INFO pulsar-ordered-OrderedExecutor-5-0-EventThread ZooKeeperDataCache:153 [State:CONNECTED Timeout:3 sessionid:0x1049b573fba001a local:/10.10.58.7:39486 remoteserver:zk-node-1.bm.infra/10.10.58.15:2181 lastZxid:4294968305 xid:532 sent:532 recv:538 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/loadbalance/brokers/pulsar-node-1.bm.infra:8080 [2019-02-22T14:01:06.526Z] ERROR pulsar-web-29-3 NonPersistentTopics:198 [null] Failed to get list of topics under namespace public/default java.util.concurrent.ExecutionException: org.apache.pulsar.client.admin.PulsarAdminException: java.net.SocketTimeoutException: Read timed out at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_201] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) ~[?:1.8.0_201] at org.apache.pulsar.broker.admin.v2.NonPersistentTopics.getList(NonPersistentTopics.java:186) ~[org.apache.pulsar-pulsar-broker-2.3.0.jar:2.3.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) ~[org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:243) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
[GitHub] [pulsar] skyrocknroll commented on issue #3671: pulsar-admin topics list public/default fails
skyrocknroll commented on issue #3671: pulsar-admin topics list public/default fails URL: https://github.com/apache/pulsar/issues/3671#issuecomment-470411519 It was a dev cluster. For some reason, we redeployed the whole cluster and the issue went away This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #3677: PIP-30: interface and mutual change authentication
jiazhai commented on issue #3677: PIP-30: interface and mutual change authentication URL: https://github.com/apache/pulsar/pull/3677#issuecomment-470408233 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext
srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext URL: https://github.com/apache/pulsar/pull/3717#issuecomment-470404482 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern
srkukarni commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern URL: https://github.com/apache/pulsar/pull/3770#issuecomment-470404385 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on issue #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090)
MarvinCai commented on issue #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090) URL: https://github.com/apache/pulsar/pull/3753#issuecomment-470402482 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3722: Retry creation of assignment topic a few times before giving up
jerrypeng commented on issue #3722: Retry creation of assignment topic a few times before giving up URL: https://github.com/apache/pulsar/pull/3722#issuecomment-470391276 rerun cpp tests rerun integration tests rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3737: [python client] set protobuf version to be larger than 3.6.0
jerrypeng commented on issue #3737: [python client] set protobuf version to be larger than 3.6.0 URL: https://github.com/apache/pulsar/pull/3737#issuecomment-470391068 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3740: fix document description bugs
jerrypeng commented on issue #3740: fix document description bugs URL: https://github.com/apache/pulsar/pull/3740#issuecomment-470391019 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3739: [Issue #3728] Add several instructions for IO Connectors
jerrypeng commented on issue #3739: [Issue #3728] Add several instructions for IO Connectors URL: https://github.com/apache/pulsar/pull/3739#issuecomment-470390925 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3735: Implementing authentication for Pulsar Functions
jerrypeng commented on issue #3735: Implementing authentication for Pulsar Functions URL: https://github.com/apache/pulsar/pull/3735#issuecomment-470390768 @merlimat @sijie @srkukarni @jiazhai please review. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3747: Closing FunctionAssignmentTailer should handle ConsumerAlreadyClosedException
jerrypeng commented on issue #3747: Closing FunctionAssignmentTailer should handle ConsumerAlreadyClosedException URL: https://github.com/apache/pulsar/pull/3747#issuecomment-470390548 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3776: Use at least 8 threads in Jetty thread pool
jerrypeng commented on issue #3776: Use at least 8 threads in Jetty thread pool URL: https://github.com/apache/pulsar/pull/3776#issuecomment-470390150 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on issue #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090)
MarvinCai commented on issue #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090) URL: https://github.com/apache/pulsar/pull/3753#issuecomment-470387565 rerun java8 tests rerun integration tests rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lxkaka commented on issue #3671: pulsar-admin topics list public/default fails
lxkaka commented on issue #3671: pulsar-admin topics list public/default fails URL: https://github.com/apache/pulsar/issues/3671#issuecomment-470385603 I have the same problem. `pulsar-admin topics list public/default` or `pulsar-admin persistent list public/default` show error `null Reason: javax.ws.rs.ProcessingException: java.net.SocketTimeoutException: Read timed out` Os Ubuntu 16 04 Pulsar 2.3.0 broker log output just same as @skyrocknroll posted ``` 12:57:55.146 [pulsar-web-28-2] ERROR org.apache.pulsar.broker.admin.v2.NonPersistentTopics - [null] Failed to get list of topics under namespace public/default java.util.concurrent.ExecutionException: org.apache.pulsar.client.admin.PulsarAdminException: java.net.SocketTimeoutException: Read timed out at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_191] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) ~[?:1.8.0_191] at org.apache.pulsar.broker.admin.v2.NonPersistentTopics.getList(NonPersistentTopics.java:186) ~[org.apache.pulsar-pulsar-broker-2.3.0.jar:2.3.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_191] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_191] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_191] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) ~[org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:243) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:316) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:298) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:268) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) [org.glassfish.jersey.core-jersey-common-2.27.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) [org.glassfish.jersey.core-jersey-server-2.27.jar:?] at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.27.jar:?] at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.27.jar:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.27.jar:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.27.jar:?] at
[GitHub] [pulsar] merlimat opened a new pull request #3776: Use at least 8 threads in Jetty thread pool
merlimat opened a new pull request #3776: Use at least 8 threads in Jetty thread pool URL: https://github.com/apache/pulsar/pull/3776 ### Motivation Use at least 8 threads to avoid having Jetty go into threads starving and having the possibility of getting into a deadlock where a Jetty thread is waiting for another HTTP call to complete in same thread. This solve the issues of requests timing out when the broker is making REST calls to itself. Such examples are when running in standalone mode and creating a function. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470378274 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #3761: Updated consumer example to use negative acks
merlimat merged pull request #3761: Updated consumer example to use negative acks URL: https://github.com/apache/pulsar/pull/3761 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3703: Added support for "negative acks" in Java client
merlimat commented on issue #3703: Added support for "negative acks" in Java client URL: https://github.com/apache/pulsar/pull/3703#issuecomment-470375735 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Updated consumer example to use negative acks (#3761)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 17b531b Updated consumer example to use negative acks (#3761)
17b531b is described below
commit 17b531bd18b2e00714d0b2e4bc8d6cfdc2dc1647
Author: Matteo Merli
AuthorDate: Wed Mar 6 19:58:47 2019 -0800
Updated consumer example to use negative acks (#3761)
---
site2/docs/client-libraries-go.md | 16 +---
site2/docs/client-libraries-java.md | 19 +--
site2/docs/client-libraries-python.md | 18 ++
site2/docs/reference-terminology.md | 7 ++-
4 files changed, 46 insertions(+), 14 deletions(-)
diff --git a/site2/docs/client-libraries-go.md
b/site2/docs/client-libraries-go.md
index 5d7466b..5e22ac6 100644
--- a/site2/docs/client-libraries-go.md
+++ b/site2/docs/client-libraries-go.md
@@ -262,7 +262,9 @@ Method | Description | Return type
`Receive(context.Context)` | Receives a single message from the topic. This
method blocks until a message is available. | `(Message, error)`
`Ack(Message)` | [Acknowledges](reference-terminology.md#acknowledgment-ack) a
message to the Pulsar [broker](reference-terminology.md#broker) | `error`
`AckID(MessageID)` |
[Acknowledges](reference-terminology.md#acknowledgment-ack) a message to the
Pulsar [broker](reference-terminology.md#broker) by message ID | `error`
-`AckCumulative(Message)` |
[Acknowledges](reference-terminology.md#acknowledgment-ack) *all* the messages
in the stream, up to and including the specified message. The `AckCumulative`
method will block until the ack has been sent to the broker. After that, the
messages will *not* be redelivered to the consumer. Cumulative acking can only
be used with a [shared](concepts-messaging.md#shared) subscription type.
+`AckCumulative(Message)` |
[Acknowledges](reference-terminology.md#acknowledgment-ack) *all* the messages
in the stream, up to and including the specified message. The `AckCumulative`
method will block until the ack has been sent to the broker. After that, the
messages will *not* be redelivered to the consumer. Cumulative acking can only
be used with a [shared](concepts-messaging.md#shared) subscription type. |
`error`
+`Nack(Message)` | Acknowledge the failure to process a single message. |
`error`
+`NackID(MessageID)` | Acknowledge the failure to process a single message. |
`error`
`Close()` | Closes the consumer, disabling its ability to receive messages
from the broker | `error`
`RedeliverUnackedMessages()` | Redelivers *all* unacknowledged messages on the
topic. In [failover](concepts-messaging.md#failover) mode, this request is
ignored if the consumer isn't active on the specified topic; in
[shared](concepts-messaging.md#shared) mode, redelivered messages are
distributed across all consumers connected to the topic. **Note**: this is a
*non-blocking* operation that doesn't throw an error. |
@@ -305,8 +307,15 @@ func main() {
if err != nil { log.Fatal(err) }
// Do something with the message
-
-consumer.Ack(msg)
+err = processMessage(msg)
+
+if err == nil {
+// Message processed successfully
+consumer.Ack(msg)
+} else {
+// Failed to process messages
+consumer.Nack(msg)
+}
}
}
```
@@ -319,6 +328,7 @@ Parameter | Description | Default
`SubscriptionName` | The subscription name for this consumer |
`Name` | The name of the consumer |
`AckTimeout` | | 0
+`NackRedeliveryDelay` | The delay after which to redeliver the messages that
failed to be processed. Default is 1min. (See `Consumer.Nack()`) | 1 minute
`SubscriptionType` | Available options are `Exclusive`, `Shared`, and
`Failover` | `Exclusive`
`MessageChannel` | The Go channel used by the consumer. Messages that arrive
from the Pulsar topic(s) will be passed to this channel. |
`ReceiverQueueSize` | Sets the size of the consumer's receiver queue, i.e. the
number of messages that can be accumulated by the consumer before the
application calls `Receive`. A value higher than the default of 1000 could
increase consumer throughput, though at the expense of more memory utilization.
| 1000
diff --git a/site2/docs/client-libraries-java.md
b/site2/docs/client-libraries-java.md
index b588e0f..b8680d8 100644
--- a/site2/docs/client-libraries-java.md
+++ b/site2/docs/client-libraries-java.md
@@ -186,18 +186,25 @@ Consumer consumer = client.newConsumer()
.subscribe();
```
-The `subscribe` method will automatically subscribe the consumer to the
specified topic and subscription. One way to make the consumer listen on the
topic is to set up a `while` loop. In this example loop, the consumer listens
for messages, prints the contents of any message that's received, and then
[GitHub] [pulsar] david-streamlio opened a new pull request #3775: Pulsar-IO: Added Kinesis Source
david-streamlio opened a new pull request #3775: Pulsar-IO: Added Kinesis Source URL: https://github.com/apache/pulsar/pull/3775 ### Motivation Added an AWS Kinesis Source connector to the existing Kinesis module. ### Modifications - Added an AWS Kinesis Source connector and associated configuration class - Added test class for AWS Kinesis Source connector - Refactored Configuration hierarchy to promote code reused between the Sink and Source configurations. - Added test cases to confirm the above refactoring did NOT break anything in the Sink ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change added tests and can be verified as follows: - Added unit tests ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): NO - The public API: NO - The schema: NO - The default values of configurations: NO - The wire protocol: NO - The rest endpoints: NO - The admin cli options: NO - Anything that affects deployment: NO ### Documentation - Does this pull request introduce a new feature? YES - If yes, how is the feature documented? YES, I updated the docs This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on a change in pull request #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090)
MarvinCai commented on a change in pull request #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090) URL: https://github.com/apache/pulsar/pull/3753#discussion_r263226100 ## File path: pulsar-client-kafka-compat/pulsar-client-kafka/src/main/java/org/apache/kafka/clients/producer/PulsarKafkaProducer.java ## @@ -34,6 +34,7 @@ import java.util.concurrent.TimeoutException; import java.util.stream.Collectors; +import com.sun.javaws.exceptions.InvalidArgumentException; Review comment: Removed unused import. Also added document that merlimat mentioned. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on issue #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090)
MarvinCai commented on issue #3753: Add support for Kafka's ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG (#1090) URL: https://github.com/apache/pulsar/pull/3753#issuecomment-470370713 rerun java8 tests rerun integration tests rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #3764: Fixed Reader.HasNext() in Go client
wolfstudy commented on issue #3764: Fixed Reader.HasNext() in Go client URL: https://github.com/apache/pulsar/pull/3764#issuecomment-470360644 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3535: Deduplication causes a lot of Full GCs
merlimat commented on issue #3535: Deduplication causes a lot of Full GCs URL: https://github.com/apache/pulsar/issues/3535#issuecomment-470360296 Updated #3612 with correct fix This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #3761: Updated consumer example to use negative acks
sijie commented on issue #3761: Updated consumer example to use negative acks URL: https://github.com/apache/pulsar/pull/3761#issuecomment-470359470 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on a change in pull request #3759: Fixes for Go client unit test
wolfstudy commented on a change in pull request #3759: Fixes for Go client unit
test
URL: https://github.com/apache/pulsar/pull/3759#discussion_r263216289
##
File path: pulsar-client-go/pulsar/consumer_test.go
##
@@ -98,7 +100,10 @@ func TestConsumer(t *testing.T) {
assert.NotNil(t, msg)
assert.Equal(t, string(msg.Payload()), fmt.Sprintf("hello-%d",
i))
- assert.Equal(t, string(msg.Topic()),
"persistent://public/default/my-topic")
+ assert.Equal(t, string(msg.Topic()),
"persistent://public/default/" + topic)
Review comment:
```suggestion
assert.Equal(t, msg.Topic(), "persistent://public/default/" +
topic)
```
The type of `msg.Topic` itself is string, we don't need to convert here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] wolfstudy commented on a change in pull request #3759: Fixes for Go client unit test
wolfstudy commented on a change in pull request #3759: Fixes for Go client unit
test
URL: https://github.com/apache/pulsar/pull/3759#discussion_r263216289
##
File path: pulsar-client-go/pulsar/consumer_test.go
##
@@ -98,7 +100,10 @@ func TestConsumer(t *testing.T) {
assert.NotNil(t, msg)
assert.Equal(t, string(msg.Payload()), fmt.Sprintf("hello-%d",
i))
- assert.Equal(t, string(msg.Topic()),
"persistent://public/default/my-topic")
+ assert.Equal(t, string(msg.Topic()),
"persistent://public/default/" + topic)
Review comment:
```suggestion
assert.Equal(t, msg.Topic(), "persistent://public/default/" +
topic)
```
The type of msg.Topic itself is string, we don't need to convert here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] rdhabalia removed a comment on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
rdhabalia removed a comment on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470356454 rerun java8 tests rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470356454 rerun java8 tests rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #3759: Fixes for Go client unit test
wolfstudy commented on issue #3759: Fixes for Go client unit test URL: https://github.com/apache/pulsar/pull/3759#issuecomment-470356154 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3612: Set the dedup cursor as "inactive" after recovery
merlimat commented on issue #3612: Set the dedup cursor as "inactive" after recovery URL: https://github.com/apache/pulsar/pull/3612#issuecomment-470354878 @hrsakai @massakam Did more experimentation and fixed it by making sure the dedup cursor is never considered "active" in order to never trigger caching behavior. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia removed a comment on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
rdhabalia removed a comment on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470334799 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy opened a new issue #3774: [go client] add go related tests and checks in jenkins
wolfstudy opened a new issue #3774: [go client] add go related tests and checks in jenkins URL: https://github.com/apache/pulsar/issues/3774 **Is your feature request related to a problem? Please describe.** At present, Jenkins does not check the unit tests and code specifications related to go. When we modify the unit test of go or add new code, we can only check it locally. For some code formats and specifications, we have no way. Do a good job of specification and unification **Describe the solution you'd like** Regarding the check of the go code format, the more mature tool in the community is gometalinter **Additional context** check.sh ``` #!/bin/bash # The script does automatic checking on a Go package and its sub-packages, including: # 1. gofmt (http://golang.org/cmd/gofmt/) # 2. golint(https://github.com/golang/lint) # 3. go vet(http://golang.org/cmd/vet) # 4. gosimple (https://github.com/dominikh/go-simple) # 5. unconvert (https://github.com/mdempsky/unconvert) # # gometalinter (github.com/alecthomas/gometalinter) is used to run each static # checker. set -ex # Make sure gometalinter is installed and $GOPATH/bin is in your path. # $ go get -v github.com/alecthomas/gometalinter" # $ gometalinter --install" if [ ! -x "$(type -p gometalinter.v2)" ]; then exit 1 fi linter_targets=$(go list ./...) # Automatic checks test -z "$(gometalinter.v2 -j 4 --disable-all \ --enable=gofmt \ --enable=golint \ --enable=vet \ --enable=gosimple \ --enable=unconvert \ --deadline=10m $linter_targets 2>&1 | grep -v 'ALL_CAPS\|OP_' 2>&1 | tee /dev/stderr)" GO111MODULE=on go test $linter_targets ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #3677: PIP-30: interface and mutual change authentication
jiazhai commented on issue #3677: PIP-30: interface and mutual change authentication URL: https://github.com/apache/pulsar/pull/3677#issuecomment-470347984 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #3772: Force before and after methods to always run
aahmed-se commented on issue #3772: Force before and after methods to always run URL: https://github.com/apache/pulsar/pull/3772#issuecomment-470345592 @rdhabalia it's one part of the problem , if one test fails teardown doesn't run so it won't solve the issue but it will the right logs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lovelle commented on issue #3748: [cpp client] Bugfix prevent dup consumer for same topic subscription
lovelle commented on issue #3748: [cpp client] Bugfix prevent dup consumer for same topic subscription URL: https://github.com/apache/pulsar/pull/3748#issuecomment-470340551 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription
lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription URL: https://github.com/apache/pulsar/pull/3746#issuecomment-470340371 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470334799 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3772: Force before and after methods to always run
rdhabalia commented on issue #3772: Force before and after methods to always run URL: https://github.com/apache/pulsar/pull/3772#issuecomment-470333240 how does it help if function-worker actually has the deadlock issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #3772: Force before and after methods to always run
aahmed-se commented on issue #3772: Force before and after methods to always run URL: https://github.com/apache/pulsar/pull/3772#issuecomment-470329785 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #3766: Support passing schema definition for JSON and AVRO schemas
codelipenghui commented on issue #3766: Support passing schema definition for JSON and AVRO schemas URL: https://github.com/apache/pulsar/pull/3766#issuecomment-470329631 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470328994 rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern
srkukarni commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern URL: https://github.com/apache/pulsar/pull/3770#issuecomment-470326835 updating input topic has the associated overhead of cleaning up subscriptions and such. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3535: Deduplication causes a lot of Full GCs
merlimat commented on issue #3535: Deduplication causes a lot of Full GCs URL: https://github.com/apache/pulsar/issues/3535#issuecomment-470325628 As @hrsakai pointed out, the fix was ineffective because applied on a code path that's not being used. The problem is that that while the cursor is set as "inactive" in the beginning, a periodic check is flipping back the state to "active": https://github.com/apache/pulsar/blob/43380523c5269c152f61b2aa8f7b70281c770d1d/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L878-L885 Working on a fix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai opened a new issue #3773: Update documentation for topic partitions auto update
jiazhai opened a new issue #3773: Update documentation for topic partitions auto update URL: https://github.com/apache/pulsar/issues/3773 Update documentation for topic partitions auto update This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: On shutdown, kill the function instance thread after interrupt attempt (#3749)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 69faeaf On shutdown, kill the function instance thread after
interrupt attempt (#3749)
69faeaf is described below
commit 69faeaf80a70947fca18e36d47cdf7540560d637
Author: Matteo Merli
AuthorDate: Wed Mar 6 14:39:26 2019 -0800
On shutdown, kill the function instance thread after interrupt attempt
(#3749)
---
.../org/apache/pulsar/functions/runtime/ThreadRuntime.java | 14 +++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git
a/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/ThreadRuntime.java
b/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/ThreadRuntime.java
index ad1002c..93246f2 100644
---
a/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/ThreadRuntime.java
+++
b/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/ThreadRuntime.java
@@ -44,6 +44,8 @@ class ThreadRuntime implements Runtime {
// The thread that invokes the function
private Thread fnThread;
+private static final int THREAD_SHUTDOWN_TIMEOUT_MILLIS = 10_000;
+
@Getter
private InstanceConfig instanceConfig;
private JavaInstanceRunnable javaInstanceRunnable;
@@ -114,13 +116,19 @@ class ThreadRuntime implements Runtime {
}
}
+@SuppressWarnings("deprecation")
@Override
public void stop() {
if (fnThread != null) {
// interrupt the instance thread
fnThread.interrupt();
try {
-fnThread.join();
+// If the instance thread doesn't respond within some time,
attempt to
+// kill the thread
+fnThread.join(THREAD_SHUTDOWN_TIMEOUT_MILLIS, 0);
+if (fnThread.isAlive()) {
+fnThread.stop();
+}
} catch (InterruptedException e) {
// ignore this
}
@@ -152,8 +160,8 @@ class ThreadRuntime implements Runtime {
public CompletableFuture
getAndResetMetrics() {
return
CompletableFuture.completedFuture(javaInstanceRunnable.getAndResetMetrics());
}
-
-
+
+
@Override
public CompletableFuture getMetrics(int
instanceId) {
return
CompletableFuture.completedFuture(javaInstanceRunnable.getMetrics());
[GitHub] [pulsar] merlimat merged pull request #3749: On shutdown, kill the function instance thread after interrupt attempt
merlimat merged pull request #3749: On shutdown, kill the function instance thread after interrupt attempt URL: https://github.com/apache/pulsar/pull/3749 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription
lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription URL: https://github.com/apache/pulsar/pull/3746#issuecomment-470306023 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lovelle removed a comment on issue #3748: [cpp client] Bugfix prevent dup consumer for same topic subscription
lovelle removed a comment on issue #3748: [cpp client] Bugfix prevent dup consumer for same topic subscription URL: https://github.com/apache/pulsar/pull/3748#issuecomment-469436084 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lovelle removed a comment on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription
lovelle removed a comment on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription URL: https://github.com/apache/pulsar/pull/3746#issuecomment-470261127 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] grantwwu edited a comment on issue #3715: Pulsar functions worker fails to initialize
grantwwu edited a comment on issue #3715: Pulsar functions worker fails to initialize URL: https://github.com/apache/pulsar/issues/3715#issuecomment-470291412 In broker logs. Running in a 3 node cluster. Unfortunately I'm not sure we can reproduce this anymore - it was breaking clusters frequently enough (once every 1 or 2 days?, with no obvious path to recovery) that we've downgraded to 2.2.1 entirely. I'm going to test out 2.3.1 when it comes out - we were also hitting some Jetty thread pool issues that merlimat said were getting fixed then. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] grantwwu commented on issue #3715: Pulsar functions worker fails to initialize
grantwwu commented on issue #3715: Pulsar functions worker fails to initialize URL: https://github.com/apache/pulsar/issues/3715#issuecomment-470291412 In broker logs. Running in a 3 node cluster. Unfortunately I'm not sure we can reproduce this anymore - it was breaking clusters frequently enough (once every 1 or 2 days?, with no obvious path to recovery) that we've downgraded to 2.2.1 entirely. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern
jerrypeng commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern URL: https://github.com/apache/pulsar/pull/3770#issuecomment-470287776 @srkukarni but then why don't we allow users to update the actual input topic? Since serde and schema are kind of tied to the topic. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext
srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext URL: https://github.com/apache/pulsar/pull/3717#issuecomment-470284433 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3715: Pulsar functions worker fails to initialize
jerrypeng commented on issue #3715: Pulsar functions worker fails to initialize URL: https://github.com/apache/pulsar/issues/3715#issuecomment-470276983 @grantwwu how often have you seen this happen? Did you find the exceptions you listed above in function logs or worker/broker logs? How are you running cluster? Standalone? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription
lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription URL: https://github.com/apache/pulsar/pull/3746#issuecomment-470273529 @sijie @jiazhai Great! :+1: thanks you both for the feedback. So when checks are success this pr will be ready to be merged! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #3772: Force before and after methods to always run
aahmed-se opened a new pull request #3772: Force before and after methods to always run URL: https://github.com/apache/pulsar/pull/3772 This should improve test stability This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext
srkukarni commented on issue #3717: Expose Secrets in Source/SinkContext URL: https://github.com/apache/pulsar/pull/3717#issuecomment-470271049 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470269568 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern
srkukarni commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern URL: https://github.com/apache/pulsar/pull/3770#issuecomment-470262744 @jerrypeng I don't see a reason not to? Afterall one can experiment with different libraries of serialization changes to test which one works better. And also we do make sure that the types etc match. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on a change in pull request #3770: Allow users to update everything in inputspecs except for isregexpattern
srkukarni commented on a change in pull request #3770: Allow users to update
everything in inputspecs except for isregexpattern
URL: https://github.com/apache/pulsar/pull/3770#discussion_r263119829
##
File path:
pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/SinkConfigUtils.java
##
@@ -416,7 +416,7 @@ public static SinkConfig validateUpdate(SinkConfig
existingConfig, SinkConfig ne
if (!existingConfig.getInputSpecs().containsKey(topicName)) {
throw new IllegalArgumentException("Input Topics cannot be
altered");
}
-if
(!consumerConfig.equals(existingConfig.getInputSpecs().get(topicName))) {
+if (consumerConfig.isRegexPattern() !=
existingConfig.getInputSpecs().get(topicName).isRegexPattern()) {
throw new IllegalArgumentException("Input Specs mismatch");
Review comment:
Changed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #3764: Fixed Reader.HasNext() in Go client
merlimat commented on a change in pull request #3764: Fixed Reader.HasNext() in
Go client
URL: https://github.com/apache/pulsar/pull/3764#discussion_r263119822
##
File path: pulsar-client-go/pulsar/reader_test.go
##
@@ -218,3 +219,49 @@ func TestReaderCompaction(t *testing.T) {
assert.Nil(t, msg)
assert.NotNil(t, err)
}
+
+func TestReaderHasNext(t *testing.T) {
+ topic := fmt.Sprintf("TestReaderHasNext-%d", rand.Int())
+ ctx := context.Background()
+
+ client, err := NewClient(ClientOptions{
+ URL: "pulsar://localhost:6650",
+ //Logger: func(level logutil.LoggerLevel, file string, line
int, message string) {
Review comment:
Good point.. :) removing these
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] srkukarni commented on a change in pull request #3770: Allow users to update everything in inputspecs except for isregexpattern
srkukarni commented on a change in pull request #3770: Allow users to update
everything in inputspecs except for isregexpattern
URL: https://github.com/apache/pulsar/pull/3770#discussion_r263119861
##
File path:
pulsar-functions/utils/src/test/java/org/apache/pulsar/functions/utils/FunctionConfigUtilsTest.java
##
@@ -157,6 +157,15 @@ public void testMergeDifferentInputs() {
FunctionConfigUtils.validateUpdate(functionConfig, newFunctionConfig);
}
+@Test(expectedExceptions = IllegalArgumentException.class,
expectedExceptionsMessageRegExp = "Input Specs mismatch")
Review comment:
added
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription
lovelle commented on issue #3746: [java client] Bugfix prevent dup consumers for same topic subscription URL: https://github.com/apache/pulsar/pull/3746#issuecomment-470261127 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3749: On shutdown, kill the function instance thread after interrupt attempt
jerrypeng commented on issue #3749: On shutdown, kill the function instance thread after interrupt attempt URL: https://github.com/apache/pulsar/pull/3749#issuecomment-470257979 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern
jerrypeng commented on issue #3770: Allow users to update everything in inputspecs except for isregexpattern URL: https://github.com/apache/pulsar/pull/3770#issuecomment-470257404 @srkukarni a general question: We should definitely allow users to update receiverQueueSize config in ConsumerConfig but does it make sense for users to also be able to update schemaType and serdeClassName ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on a change in pull request #3770: Allow users to update everything in inputspecs except for isregexpattern
jerrypeng commented on a change in pull request #3770: Allow users to update
everything in inputspecs except for isregexpattern
URL: https://github.com/apache/pulsar/pull/3770#discussion_r263113441
##
File path:
pulsar-functions/utils/src/test/java/org/apache/pulsar/functions/utils/FunctionConfigUtilsTest.java
##
@@ -157,6 +157,15 @@ public void testMergeDifferentInputs() {
FunctionConfigUtils.validateUpdate(functionConfig, newFunctionConfig);
}
+@Test(expectedExceptions = IllegalArgumentException.class,
expectedExceptionsMessageRegExp = "Input Specs mismatch")
Review comment:
Can we also add a test case that tests that other configs e.g.
receiverQueueSize can be passed/updated without error
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jerrypeng commented on a change in pull request #3770: Allow users to update everything in inputspecs except for isregexpattern
jerrypeng commented on a change in pull request #3770: Allow users to update
everything in inputspecs except for isregexpattern
URL: https://github.com/apache/pulsar/pull/3770#discussion_r263112143
##
File path:
pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/SinkConfigUtils.java
##
@@ -416,7 +416,7 @@ public static SinkConfig validateUpdate(SinkConfig
existingConfig, SinkConfig ne
if (!existingConfig.getInputSpecs().containsKey(topicName)) {
throw new IllegalArgumentException("Input Topics cannot be
altered");
}
-if
(!consumerConfig.equals(existingConfig.getInputSpecs().get(topicName))) {
+if (consumerConfig.isRegexPattern() !=
existingConfig.getInputSpecs().get(topicName).isRegexPattern()) {
throw new IllegalArgumentException("Input Specs mismatch");
Review comment:
Can we modify the message of the exception to be more specific? That would
help a lot in debugging
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3768: [pulsar-function] intermittent test failure due to deadlock
merlimat commented on issue #3768: [pulsar-function] intermittent test failure due to deadlock URL: https://github.com/apache/pulsar/issues/3768#issuecomment-470254736 I think he's referring to the `PulsarWorkerAssignmentTest` test getting stuck during shutdown. There's already #3749 to fix that...though the ehm challenge is to get all these flaky to pass at once. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng opened a new pull request #3771: fix: function config cleanupSubscription update bug
jerrypeng opened a new pull request #3771: fix: function config cleanupSubscription update bug URL: https://github.com/apache/pulsar/pull/3771 ### Motivation The are currently two bugs involving the config cleanupSubscription and updating a function: 1. when a user updates a function without changing the cleanupSubscription config, the cleanupSubscription config gets reverted to the default i.e. true 2. When a user attempts to update the cleanupSubscription config in a function, the config doesn't get changed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3768: [pulsar-function] intermittent test failure due to deadlock
srkukarni commented on issue #3768: [pulsar-function] intermittent test failure due to deadlock URL: https://github.com/apache/pulsar/issues/3768#issuecomment-470253042 @rdhabalia Could you please add more color on what the dead lock is? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch retry_creation updated (bbfb1cb -> 024831f)
This is an automated email from the ASF dual-hosted git repository. sanjeevrk pushed a change to branch retry_creation in repository https://gitbox.apache.org/repos/asf/pulsar.git. from bbfb1cb Added header new ccfb949 When the loadmanager leader is not available, fall through regular least loaded selection (#3688) new 0142247 Revert "Remove broker mode to handle persistent/non-persistent topics separately (#3348)" (#3713) new 7c7c3fb Use python 3.5 runtime and pulsar client (#3723) new bb9a4fb fix pulsar_message_set_replication_clusters (#3729) new 40aecc1 Fix Test localhost resolution errors (#3730) new bc8b380 Fail the source record if the write fails (#3706) new 5fd5730 [logging] make root log level configurable (#3661) new 65daec1 fix Jenkins pulsar-website-build error 571 (#3691) new 548c726 [Issue #3436][pulsar-broker] Creating REST Endpoint for non-partitioned topic creation (#3625) new 37a2fda [Issue 3275][pulsar-io]Support source and sink of flume (#3597) new d84b687 Update README.md with IDE setup for lombok (#3711) new e06a894 add cli client consumer for standalone (#3697) new d847c35 Exposing getSchemaVersion in the client by making it public. (#3744) new ffa2a68 Avoid debug noise by consumer ack-tracker (#2953) new 9dc3df6 Adjust the serving threads to have a minimum of threads (#3698) new 07cebb1 On publish failures, log error and count them as sys exceptions (#3704) new 33f1c55 fix function termination cleanup (#3751) new c39e7d1 Broker considers fail-over consumer priority-level (#2954) new 77332b0 [docs] Fixed typo remove-backlog-quotas to remove-backlog-quota in docs (#3757) new a662757 Multi version generic schema provider (#3756) new 024831f Merge branch 'master' into retry_creation The 2979 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: README.md | 28 ++ bin/pulsar | 1 + conf/broker.conf | 8 +- conf/log4j2.yaml | 4 +- conf/proxy.conf| 2 +- conf/standalone.conf | 9 + deployment/terraform-ansible/templates/broker.conf | 6 + docker/pulsar/Dockerfile | 8 +- docker/pulsar/pom.xml | 6 +- ...ar-client-37.sh => install-pulsar-client-35.sh} | 2 +- .../apache/pulsar/broker/ServiceConfiguration.java | 14 +- .../org/apache/pulsar/broker/PulsarService.java| 15 +- .../broker/admin/impl/PersistentTopicsBase.java| 12 + .../pulsar/broker/admin/v2/PersistentTopics.java | 16 + .../broker/loadbalance/LeaderElectionService.java | 17 + .../broker/loadbalance/impl/LoadManagerShared.java | 34 +- .../loadbalance/impl/ModularLoadManagerImpl.java | 43 +- .../loadbalance/impl/SimpleLoadManagerImpl.java| 27 +- .../pulsar/broker/namespace/NamespaceService.java | 11 +- .../AbstractDispatcherSingleActiveConsumer.java| 29 +- .../pulsar/broker/service/BrokerService.java | 16 + .../pulsar/broker/admin/AdminApiTlsAuthTest.java | 28 +- .../pulsar/broker/admin/PersistentTopicsTest.java | 5 +- .../broker/auth/MockedPulsarServiceBaseTest.java | 16 +- .../broker/loadbalance/LoadBalancerTest.java | 70 ++- .../loadbalance/ModularLoadManagerImplTest.java| 84 ++-- .../broker/service/ReplicatorGlobalNSTest.java | 15 +- .../pulsar/broker/service/ReplicatorTestBase.java | 4 +- .../broker/service/v1/V1_ReplicatorTest.java | 6 +- .../pulsar/client/api/NonPersistentTopicTest.java | 151 ++ .../client/api/SimpleProducerConsumerTest.java | 93 .../pulsar/client/impl/TopicFromMessageTest.java | 5 - .../worker/PulsarWorkerAssignmentTest.java | 78 ++- .../apache/pulsar/io/PulsarFunctionAdminTest.java | 2 +- .../apache/pulsar/io/PulsarFunctionE2ETest.java| 13 +- .../stats/client/PulsarBrokerStatsClientTest.java | 22 +- .../org/apache/pulsar/client/admin/Topics.java | 19 + .../pulsar/client/admin/internal/TopicsImpl.java | 19 + .../apache/pulsar/client/api/ConsumerBuilder.java | 20 + .../java/org/apache/pulsar/client/api/Message.java | 7 + pulsar-client-cpp/include/pulsar/c/message.h | 4 +- pulsar-client-cpp/lib/c/c_Message.cc | 8 +- pulsar-client-go/pulsar/c_message.go | 2 +- .../pulsar/admin/cli/PulsarAdminToolTest.java | 3 + .../org/apache/pulsar/admin/cli/CmdTopics.java | 14 + .../client/impl/BinaryProtoLookupService.java | 11 +- .../pulsar/client/impl/HttpLookupService.java | 18 +-
[GitHub] [pulsar] merlimat commented on a change in pull request #3750: Negative acks in C++ client
merlimat commented on a change in pull request #3750: Negative acks in C++
client
URL: https://github.com/apache/pulsar/pull/3750#discussion_r263103180
##
File path: pulsar-client-cpp/tests/BasicEndToEndTest.cc
##
@@ -99,7 +100,7 @@ static void sendCallBackWithDelay(Result r, const Message&
msg, std::string pref
class EncKeyReader : public CryptoKeyReader {
private:
-void readFile(std::string fileName, std::string& fileContents) const {
+void readFile(std::string fileName, std::string ) const {
Review comment:
This is existing code and it was just one space reformatted by `make
format`.
In any case the functional change should be done in a different PR. Given
this is for test, I'm not particular concerned about a string copy when loading
the key file.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jai1 commented on issue #3750: Negative acks in C++ client
jai1 commented on issue #3750: Negative acks in C++ client URL: https://github.com/apache/pulsar/pull/3750#issuecomment-470243091 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jai1 commented on a change in pull request #3750: Negative acks in C++ client
jai1 commented on a change in pull request #3750: Negative acks in C++ client
URL: https://github.com/apache/pulsar/pull/3750#discussion_r263097629
##
File path: pulsar-client-cpp/tests/BasicEndToEndTest.cc
##
@@ -99,7 +100,7 @@ static void sendCallBackWithDelay(Result r, const Message&
msg, std::string pref
class EncKeyReader : public CryptoKeyReader {
private:
-void readFile(std::string fileName, std::string& fileContents) const {
+void readFile(std::string fileName, std::string ) const {
Review comment:
`const std::string& fileName`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] srkukarni commented on a change in pull request #3717: Expose Secrets in Source/SinkContext
srkukarni commented on a change in pull request #3717: Expose Secrets in
Source/SinkContext
URL: https://github.com/apache/pulsar/pull/3717#discussion_r263097424
##
File path:
pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/SchedulerManager.java
##
@@ -105,25 +105,12 @@ public SchedulerManager(WorkerConfig workerConfig,
PulsarClient pulsarClient, Pu
.compressionType(CompressionType.LZ4)
.sendTimeout(0, TimeUnit.MILLISECONDS)
.createAsync().get(10, TimeUnit.SECONDS);
+} catch (Exception e) {
Review comment:
Oops, looks like bad commit. Reverted
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jai1 commented on issue #3761: Updated consumer example to use negative acks
jai1 commented on issue #3761: Updated consumer example to use negative acks URL: https://github.com/apache/pulsar/pull/3761#issuecomment-470240572 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jai1 commented on a change in pull request #3633: [pulsar-broker] Fix deadlock: add zk-operation timeout for blocking call on zk-cache
jai1 commented on a change in pull request #3633: [pulsar-broker] Fix deadlock:
add zk-operation timeout for blocking call on zk-cache
URL: https://github.com/apache/pulsar/pull/3633#discussion_r263094693
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/web/PulsarWebResource.java
##
@@ -609,9 +608,10 @@ protected void validateTopicOwnership(TopicName
topicName, boolean authoritative
* @throws Exception
*/
protected void validateGlobalNamespaceOwnership(NamespaceName namespace) {
+int timeout =
pulsar().getConfiguration().getZooKeeperOperationTimeoutSeconds();
Review comment:
nit: var name -> timeoutInSecs
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jai1 commented on a change in pull request #3633: [pulsar-broker] Fix deadlock: add zk-operation timeout for blocking call on zk-cache
jai1 commented on a change in pull request #3633: [pulsar-broker] Fix deadlock: add zk-operation timeout for blocking call on zk-cache URL: https://github.com/apache/pulsar/pull/3633#discussion_r263094012 ## File path: conf/broker.conf ## @@ -61,6 +61,9 @@ failureDomainsEnabled=false # Zookeeper session timeout in milliseconds zooKeeperSessionTimeoutMillis=3 +# ZooKeeper operation timeout in seconds Review comment: nit: Other config around this are in Ms so I would suggest to keep the TimeUnit in millis. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jai1 commented on issue #3633: [pulsar-broker] Fix deadlock: add zk-operation timeout for blocking call on zk-cache
jai1 commented on issue #3633: [pulsar-broker] Fix deadlock: add zk-operation timeout for blocking call on zk-cache URL: https://github.com/apache/pulsar/pull/3633#issuecomment-470238478 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470238447 created issue #3768 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia opened a new issue #3768: [pulsar-function] intermittent test failure due to deadlock
rdhabalia opened a new issue #3768: [pulsar-function] intermittent test failure due to deadlock URL: https://github.com/apache/pulsar/issues/3768 [org.apache.pulsar.functions.worker.PulsarWorkerAssignmentTest.shutdown()](https://builds.apache.org/job/pulsar_precommit_java8/7046/testReport/junit/org.apache.pulsar.functions.worker/PulsarWorkerAssignmentTest/shutdown/) has intermittent failure due to deadlock into function-worker. Disabling this for now and we have to enable back once deadlock issue is fixed at function-worker. ``` Error Message Method org.apache.pulsar.functions.worker.PulsarWorkerAssignmentTest.shutdown() didn't finish within the time-out 6 Stacktrace org.testng.internal.thread.ThreadTimeoutException: Method org.apache.pulsar.functions.worker.PulsarWorkerAssignmentTest.shutdown() didn't finish within the time-out 6 at io.netty.channel.epoll.Native.eventFdWrite(Native Method) at io.netty.channel.epoll.EpollEventLoop.wakeup(EpollEventLoop.java:169) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:799) at io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:1013) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:825) at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:794) at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:837) at org.apache.pulsar.client.impl.ClientCnx.sendRequestWithId(ClientCnx.java:685) at org.apache.pulsar.client.impl.ProducerImpl.closeAsync(ProducerImpl.java:609) at org.apache.pulsar.functions.sink.PulsarSink$PulsarSinkProcessorBase.close(PulsarSink.java:130) at org.apache.pulsar.functions.sink.PulsarSink.close(PulsarSink.java:306) at org.apache.pulsar.functions.instance.JavaInstanceRunnable.close(JavaInstanceRunnable.java:482) at org.apache.pulsar.functions.runtime.ThreadRuntime.stop(ThreadRuntime.java:128) at org.apache.pulsar.functions.runtime.RuntimeSpawner.close(RuntimeSpawner.java:151) at org.apache.pulsar.functions.worker.FunctionActioner.stopFunction(FunctionActioner.java:243) at org.apache.pulsar.functions.worker.FunctionRuntimeManager.conditionallyStopFunction(FunctionRuntimeManager.java:868) at org.apache.pulsar.functions.worker.FunctionRuntimeManager.stopFunction(FunctionRuntimeManager.java:441) at org.apache.pulsar.functions.worker.FunctionRuntimeManager.lambda$stopAllOwnedFunctions$1(FunctionRuntimeManager.java:429) at org.apache.pulsar.functions.worker.FunctionRuntimeManager$$Lambda$795/1890737615.accept(Unknown Source) at java.util.HashMap$Values.forEach(HashMap.java:981) at org.apache.pulsar.functions.worker.FunctionRuntimeManager.stopAllOwnedFunctions(FunctionRuntimeManager.java:426) at org.apache.pulsar.functions.worker.FunctionRuntimeManager.close(FunctionRuntimeManager.java:813) at org.apache.pulsar.functions.worker.WorkerService.stop(WorkerService.java:218) at org.apache.pulsar.functions.worker.PulsarWorkerAssignmentTest.shutdown(PulsarWorkerAssignmentTest.java:135) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54) at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jai1 commented on a change in pull request #3764: Fixed Reader.HasNext() in Go client
jai1 commented on a change in pull request #3764: Fixed Reader.HasNext() in Go
client
URL: https://github.com/apache/pulsar/pull/3764#discussion_r263092680
##
File path: pulsar-client-go/pulsar/reader_test.go
##
@@ -218,3 +219,49 @@ func TestReaderCompaction(t *testing.T) {
assert.Nil(t, msg)
assert.NotNil(t, err)
}
+
+func TestReaderHasNext(t *testing.T) {
+ topic := fmt.Sprintf("TestReaderHasNext-%d", rand.Int())
+ ctx := context.Background()
+
+ client, err := NewClient(ClientOptions{
+ URL: "pulsar://localhost:6650",
+ //Logger: func(level logutil.LoggerLevel, file string, line
int, message string) {
Review comment:
Do you want to checkin with these comments
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
rdhabalia commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470237356 I am going to disable [org.apache.pulsar.functions.worker.PulsarWorkerAssignmentTest.shutdown()](https://builds.apache.org/job/pulsar_precommit_java8/7046/testReport/junit/org.apache.pulsar.functions.worker/PulsarWorkerAssignmentTest/shutdown/) test for now because it's blocking other PR. It's mainly happening due to deadlock in function-worker while stopping the function during shutdown. I will create separate issue to fix the deadlock and then we can enable this test again. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #3764: Fixed Reader.HasNext() in Go client
merlimat commented on a change in pull request #3764: Fixed Reader.HasNext() in
Go client
URL: https://github.com/apache/pulsar/pull/3764#discussion_r263083466
##
File path: pulsar-client-cpp/lib/ConsumerImpl.cc
##
@@ -1039,8 +1040,7 @@ void
ConsumerImpl::hasMessageAvailableAsync(HasMessageAvailableCallback callback
return;
}
-BrokerGetLastMessageIdCallback callback1 = [this, lastDequed,
callback](Result result,
-
MessageId messageId) {
+getLastMessageIdAsync([this, lastDequed, callback](Result result,
MessageId messageId) {
if (result == ResultOk) {
Review comment:
I don't think that this would improve readability
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jai1 commented on a change in pull request #3764: Fixed Reader.HasNext() in Go client
jai1 commented on a change in pull request #3764: Fixed Reader.HasNext() in Go
client
URL: https://github.com/apache/pulsar/pull/3764#discussion_r263080874
##
File path: pulsar-client-cpp/lib/ConsumerImpl.cc
##
@@ -1039,8 +1040,7 @@ void
ConsumerImpl::hasMessageAvailableAsync(HasMessageAvailableCallback callback
return;
}
-BrokerGetLastMessageIdCallback callback1 = [this, lastDequed,
callback](Result result,
-
MessageId messageId) {
+getLastMessageIdAsync([this, lastDequed, callback](Result result,
MessageId messageId) {
if (result == ResultOk) {
Review comment:
nit: can you refactor it to
```
const auto ret = (result == ResultOk && messageId > lastDequed &&
messageId.entryId() != -1);
callback(result, ret);
```
or even not use ret.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] srkukarni commented on issue #3722: Retry creation of assignment topic a few times before giving up
srkukarni commented on issue #3722: Retry creation of assignment topic a few times before giving up URL: https://github.com/apache/pulsar/pull/3722#issuecomment-470213817 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3749: On shutdown, kill the function instance thread after interrupt attempt
merlimat commented on issue #3749: On shutdown, kill the function instance thread after interrupt attempt URL: https://github.com/apache/pulsar/pull/3749#issuecomment-470194548 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager…
merlimat commented on issue #3760: Fix intermittent test: BrokerServiceLookupTest.testModularLoadManager… URL: https://github.com/apache/pulsar/pull/3760#issuecomment-470194520 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #3752: revise the schema default type not null
merlimat commented on a change in pull request #3752: revise the schema default
type not null
URL: https://github.com/apache/pulsar/pull/3752#discussion_r263034204
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java
##
@@ -167,49 +167,95 @@ default T decode(byte[] bytes, byte[] schemaVersion) {
}
/**
- * Create a Avro schema type by extracting the fields of the specified
class.
+ * Create a allow null Avro schema type by extracting the fields of the
specified class.
*
* @param clazz the POJO class to be used to extract the Avro schema
* @return a Schema instance
*/
static Schema AVRO(Class clazz) {
-return DefaultImplementation.newAvroSchema(clazz);
+return DefaultImplementation.newAvroSchema(clazz, true);
}
/**
- * Create a JSON schema type by extracting the fields of the specified
class.
+ * Create a Avro schema type by extracting the fields of the specified
class.
+ *
+ * @param clazz the POJO class to be used to extract the Avro schema
+ * allowNull the create a allow null or not null Avro schema
+ * @return a Schema instance
+ */
+static Schema AVRO(Class clazz, Boolean allowNull) {
Review comment:
`boolean`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #3752: revise the schema default type not null
merlimat commented on a change in pull request #3752: revise the schema default
type not null
URL: https://github.com/apache/pulsar/pull/3752#discussion_r263034112
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java
##
@@ -167,49 +167,95 @@ default T decode(byte[] bytes, byte[] schemaVersion) {
}
/**
- * Create a Avro schema type by extracting the fields of the specified
class.
+ * Create a allow null Avro schema type by extracting the fields of the
specified class.
*
* @param clazz the POJO class to be used to extract the Avro schema
* @return a Schema instance
*/
static Schema AVRO(Class clazz) {
-return DefaultImplementation.newAvroSchema(clazz);
+return DefaultImplementation.newAvroSchema(clazz, true);
}
/**
- * Create a JSON schema type by extracting the fields of the specified
class.
+ * Create a Avro schema type by extracting the fields of the specified
class.
+ *
+ * @param clazz the POJO class to be used to extract the Avro schema
+ * allowNull the create a allow null or not null Avro schema
+ * @return a Schema instance
+ */
+static Schema AVRO(Class clazz, Boolean allowNull) {
+return DefaultImplementation.newAvroSchema(clazz, allowNull);
+}
+
+
+/**
+ * Create a allow null JSON schema type by extracting the fields of the
specified class.
*
* @param clazz the POJO class to be used to extract the JSON schema
* @return a Schema instance
*/
static Schema JSON(Class clazz) {
-return DefaultImplementation.newJSONSchema(clazz);
+return DefaultImplementation.newJSONSchema(clazz, true);
+}
+
+/**
+ * Create a allow null JSON schema type by extracting the fields of the
specified class.
+ *
+ * @param clazz the POJO class to be used to extract the JSON schema
+ * allowNull the create a allow null or not null JSON schema
+ * @return a Schema instance
+ */
+static Schema JSON(Class clazz, Boolean allowNull) {
Review comment:
Since there's is no standard for JSON schemas, I'd prefer to leave it for
now with the fixed `allowNull=true` and don't offer the option here
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #3752: revise the schema default type not null
merlimat commented on a change in pull request #3752: revise the schema default
type not null
URL: https://github.com/apache/pulsar/pull/3752#discussion_r263035591
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java
##
@@ -167,49 +167,95 @@ default T decode(byte[] bytes, byte[] schemaVersion) {
}
/**
- * Create a Avro schema type by extracting the fields of the specified
class.
+ * Create a allow null Avro schema type by extracting the fields of the
specified class.
*
* @param clazz the POJO class to be used to extract the Avro schema
* @return a Schema instance
*/
static Schema AVRO(Class clazz) {
-return DefaultImplementation.newAvroSchema(clazz);
+return DefaultImplementation.newAvroSchema(clazz, true);
}
/**
- * Create a JSON schema type by extracting the fields of the specified
class.
+ * Create a Avro schema type by extracting the fields of the specified
class.
+ *
+ * @param clazz the POJO class to be used to extract the Avro schema
+ * allowNull the create a allow null or not null Avro schema
+ * @return a Schema instance
+ */
+static Schema AVRO(Class clazz, Boolean allowNull) {
+return DefaultImplementation.newAvroSchema(clazz, allowNull);
+}
+
+
+/**
+ * Create a allow null JSON schema type by extracting the fields of the
specified class.
*
* @param clazz the POJO class to be used to extract the JSON schema
* @return a Schema instance
*/
static Schema JSON(Class clazz) {
-return DefaultImplementation.newJSONSchema(clazz);
+return DefaultImplementation.newJSONSchema(clazz, true);
+}
+
+/**
+ * Create a allow null JSON schema type by extracting the fields of the
specified class.
+ *
+ * @param clazz the POJO class to be used to extract the JSON schema
+ * allowNull the create a allow null or not null JSON schema
+ * @return a Schema instance
+ */
+static Schema JSON(Class clazz, Boolean allowNull) {
+return DefaultImplementation.newJSONSchema(clazz, allowNull);
}
+
/**
* Key Value Schema using passed in schema type, support JSON and AVRO
currently.
*/
static Schema> KeyValue(Class key, Class
value, SchemaType type) {
-return DefaultImplementation.newKeyValueSchema(key, value, type);
+return DefaultImplementation.newKeyValueSchema(key, value, type, true);
}
+/**
+ * Key Value Schema using passed in schema type, support JSON and AVRO
currently.
+ */
+static Schema> KeyValue(Class key, Class
value, SchemaType type, Boolean allowNull) {
Review comment:
This becomes even more confusing, because `allowNull` here would be applied
to, say, a protobuf schema for which it will not make sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[pulsar] branch master updated: Multi version generic schema provider (#3756)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new a662757 Multi version generic schema provider (#3756)
a662757 is described below
commit a66275741237e59b76efe7d5339f25a3a89f2fb1
Author: penghui
AuthorDate: Thu Mar 7 00:46:53 2019 +0800
Multi version generic schema provider (#3756)
* Add multi version generic schema provider.
* fix log output and refactor get schema in lookup service.
---
.../client/impl/BinaryProtoLookupService.java | 11 ++-
.../pulsar/client/impl/HttpLookupService.java | 18 -
.../apache/pulsar/client/impl/LookupService.java | 15
.../generic/MultiVersionGenericSchemaProvider.java | 87 ++
.../MultiVersionGenericSchemaProviderTest.java | 67 +
.../pulsar/common/schema/BytesSchemaVersion.java | 40 ++
6 files changed, 234 insertions(+), 4 deletions(-)
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java
index 94773e3..8d07552 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java
@@ -44,6 +44,7 @@ import
org.apache.pulsar.common.api.proto.PulsarApi.CommandLookupTopicResponse.L
import org.apache.pulsar.common.naming.NamespaceName;
import org.apache.pulsar.common.naming.TopicName;
import org.apache.pulsar.common.partition.PartitionedTopicMetadata;
+import org.apache.pulsar.common.schema.BytesSchemaVersion;
import org.apache.pulsar.common.schema.SchemaInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -183,10 +184,16 @@ public class BinaryProtoLookupService implements
LookupService {
@Override
public CompletableFuture> getSchema(TopicName
topicName) {
+return getSchema(topicName, null);
+}
+
+
+@Override
+public CompletableFuture> getSchema(TopicName
topicName, byte[] version) {
return
client.getCnxPool().getConnection(serviceNameResolver.resolveHost()).thenCompose(clientCnx
-> {
long requestId = client.newRequestId();
-ByteBuf request = Commands.newGetSchema(requestId,
topicName.toString(), Optional.empty());
-
+ByteBuf request = Commands.newGetSchema(requestId,
topicName.toString(),
+Optional.ofNullable(BytesSchemaVersion.of(version)));
return clientCnx.sendGetSchema(request, requestId);
});
}
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/HttpLookupService.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/HttpLookupService.java
index 955c45e..8e26c87 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/HttpLookupService.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/HttpLookupService.java
@@ -24,7 +24,9 @@ import io.netty.channel.EventLoopGroup;
import java.net.InetSocketAddress;
import java.net.URI;
+import java.nio.charset.StandardCharsets;
import java.util.Arrays;
+import java.util.Base64;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
@@ -136,18 +138,30 @@ class HttpLookupService implements LookupService {
@Override
public CompletableFuture> getSchema(TopicName
topicName) {
+return getSchema(topicName, null);
+}
+
+@Override
+public CompletableFuture> getSchema(TopicName
topicName, byte[] version) {
CompletableFuture> future = new
CompletableFuture<>();
String schemaName = topicName.getSchemaName();
String path = String.format("admin/v2/schemas/%s/schema", schemaName);
-
+if (version != null) {
+path = String.format("admin/v2/schemas/%s/schema/%s",
+schemaName,
+new String(version, StandardCharsets.UTF_8));
+}
httpClient.get(path, GetSchemaResponse.class).thenAccept(response -> {
future.complete(Optional.of(SchemaInfoUtil.newSchemaInfo(schemaName,
response)));
}).exceptionally(ex -> {
if (ex.getCause() instanceof NotFoundException) {
future.complete(Optional.empty());
} else {
-log.warn("Failed to get schema for topic {} : {}", topicName,
ex.getCause().getClass());
+log.warn("Failed to get schema for topic {} version {}",
+topicName,
+version != null ?
Base64.getEncoder().encodeToString(version) : null,
+ex.getCause());
future.completeExceptionally(ex);
}
return null;
