[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542068298 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542056765 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542022333 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542052087 run Integration Tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542010745 run Integration Tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542052087 run Integration Tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5381: [Issue 5380][deployment]Fixing pod anti-affinity rules in Kubernetes files including Helm chart
jiazhai commented on issue #5381: [Issue 5380][deployment]Fixing pod anti-affinity rules in Kubernetes files including Helm chart URL: https://github.com/apache/pulsar/pull/5381#issuecomment-542049773 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rocketraman commented on issue #5202: Incorrect msgBacklog value
rocketraman commented on issue #5202: Incorrect msgBacklog value
URL: https://github.com/apache/pulsar/issues/5202#issuecomment-542044367
I have this issue as well... one of my subscriptions to a partitioned topic
is showing a backlog of 1 message, but that is incorrect -- the backlog is
actually 0.
Partitioned stats:
```
{
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"averageMsgSize" : 0.0,
"storageSize" : 2189,
"publishers" : [ {
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"averageMsgSize" : 0.0,
"producerId" : 0
} ],
"subscriptions" : {
"__compaction" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"msgRateRedeliver" : 0.0,
"msgBacklog" : 0,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"unackedMessages" : 0,
"msgRateExpired" : 0.0,
"consumers" : [ ],
"isReplicated" : false
},
"test-xxx" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"msgRateRedeliver" : 0.0,
"msgBacklog" : 1,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"unackedMessages" : 1,
"msgRateExpired" : 0.0,
"consumers" : [ {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"msgRateRedeliver" : 0.0,
"availablePermits" : 998,
"unackedMessages" : 1,
"blockedConsumerOnUnackedMsgs" : false
} ],
"isReplicated" : false
}
},
"replication" : { },
"metadata" : {
"partitions" : 1
},
"partitions" : { }
}
```
and internal stats:
```
{
"entriesAddedCounter" : 28,
"numberOfEntries" : 8,
"totalSize" : 19297,
"currentLedgerEntries" : 7,
"currentLedgerSize" : 17114,
"lastLedgerCreatedTimestamp" : "2019-10-14T14:46:51.57Z",
"waitingCursorsCount" : 268,
"pendingAddEntriesCount" : 0,
"lastConfirmedEntry" : "81:6",
"state" : "LedgerOpened",
"ledgers" : [ {
"ledgerId" : 81,
"entries" : 0,
"size" : 0,
"offloaded" : false
} ],
"cursors" : {
"__compaction" : {
"markDeletePosition" : "81:6",
"readPosition" : "81:7",
"waitingReadOp" : false,
"pendingReadOps" : 0,
"messagesConsumedCounter" : 28,
"cursorLedger" : 85,
"cursorLedgerLastEntry" : 3,
"individuallyDeletedMessages" : "[]",
"lastLedgerSwitchTimestamp" : "2019-10-15T05:22:53.273Z",
"state" : "Open",
"numberOfEntriesSinceFirstNotAckedMessage" : 1,
"totalNonContiguousDeletedMessagesRange" : 0,
"properties" : {
"CompactedTopicLedger" : 86
}
},
"test-xxx" : {
"markDeletePosition" : "81:6",
"readPosition" : "81:7",
"waitingReadOp" : true,
"pendingReadOps" : 0,
"messagesConsumedCounter" : 27,
"cursorLedger" : 84,
"cursorLedgerLastEntry" : 5,
"individuallyDeletedMessages" : "[]",
"lastLedgerSwitchTimestamp" : "2019-10-15T05:19:49.576Z",
"state" : "Open",
"numberOfEntriesSinceFirstNotAckedMessage" : 1,
"totalNonContiguousDeletedMessagesRange" : 0,
"properties" : { }
}
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] k2la opened a new issue #5387: When I create many PulsarClients and Consumers, failed to resolve hostname
k2la opened a new issue #5387: When I create many PulsarClients and Consumers,
failed to resolve hostname
URL: https://github.com/apache/pulsar/issues/5387
I have use cases that needs to create many `PulsarClient` and `Consumer`
instances.
```
List clients = new ArrayList<>();
List> consumers = new ArrayList<>();
for (int i = 0; i < 100; i++) {
clients.add(PulsarClient.builder()
.serviceUrl(url)
.build());
consumers.add(clients.get(i).newConsumer()
.topic(topic)
.subscriptionName(subs)
.messageListener(new MessageListener() {
@Override
public void received(Consumer consumer, Message
message) {
System.out.println("Received: " + new String(message.getData()));
consumer.acknowledgeAsync(message);
}
})
.subscriptionType(SubscriptionType.Shared)
.subscribe());
}
```
When I set MaxDirectMemorySize of JVM 1GB and create 100 each `PulsarClient`
and `Consumer` instances like above code, the client program failed to resolve
hostname with the following logs.
```
[pulsar-client-io-164-1] WARN org.apache.pulsar.client.impl.ConnectionPool -
Failed to open connection to example.broker.dev.pulsar.jp:6650 :
org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException:
Search domain query failed. Original hostname: 'example.broker.dev.pulsar.jp'
failed to resolve 'example.broker.dev.pulsar.jp' after 4 queries
[pulsar-client-io-164-1] WARN org.apache.pulsar.client.impl.PulsarClientImpl
- [persistent://tenant1/ns1/t1] Failed to get partitioned topic metadata
java.util.concurrent.CompletionException:
org.apache.pulsar.client.api.PulsarClientException:
java.util.concurrent.CompletionException:
org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException:
Search domain query failed. Original hostname: 'example.broker.dev.pulsar.jp'
failed to resolve 'example.broker.dev.pulsar.jp' after 4 queries
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at
java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:647)
at
java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at
org.apache.pulsar.client.impl.ConnectionPool.lambda$null$9(ConnectionPool.java:202)
at
org.apache.pulsar.shade.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at
org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at
org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
at
org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
at
org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.pulsar.client.api.PulsarClientException:
java.util.concurrent.CompletionException:
org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException:
Search domain query failed. Original hostname: 'example.broker.dev.pulsar.jp'
failed to resolve 'example.broker.dev.pulsar.jp' after 4 queries
... 7 more
Caused by: java.util.concurrent.CompletionException:
org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException:
Search domain query failed. Original hostname: 'example.broker.dev.pulsar.jp'
failed to resolve 'example.broker.dev.pulsar.jp' after 4 queries
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943)
at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at
org.apache.pulsar.client.impl.ConnectionPool.lambda$resolveName$16(ConnectionPool.java:259)
at
org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at
org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(Defau
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542022333 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on a change in pull request #5148: Validate topic name before creating partition/non partition topic via admin cli.
MarvinCai commented on a change in pull request #5148: Validate topic name
before creating partition/non partition topic via admin cli.
URL: https://github.com/apache/pulsar/pull/5148#discussion_r334737624
##
File path:
pulsar-broker/src/test/java/org/apache/pulsar/client/impl/TopicsConsumerImplTest.java
##
@@ -685,92 +685,6 @@ public void testMultiTopicsMessageListener() throws
Exception {
consumer.close();
}
-
-/**
- * Test topic partitions auto subscribed.
- *
- * Steps:
- * 1. Create a consumer with 2 topics, and each topic has 2 partitions:
xx-partition-0, xx-partition-1.
- * 2. produce message to xx-partition-2, and verify consumer could not
receive message.
Review comment:
sure, updated to keep test case to still test can auto subscribe newly added
partitions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] MarvinCai commented on a change in pull request #5148: Validate topic name before creating partition/non partition topic via admin cli.
MarvinCai commented on a change in pull request #5148: Validate topic name before creating partition/non partition topic via admin cli. URL: https://github.com/apache/pulsar/pull/5148#discussion_r334733254 ## File path: site2/docs/admin-api-partitioned-topics.md ## @@ -80,6 +88,12 @@ command and specifying the topic name as an argument. This is an example command $ bin/pulsar-admin topics create persistent://my-tenant/my-namespace/my-topic ``` +> Note +> +> It's only allow to create non partitioned topic of name contains suffix '-partition-' followed by numeric value like +> 'xyz-topic-partition-10', if there's already a partitioned topic with same name, in this case 'xyz-topic', and has +> number of partition larger then that numeric value in this case 10. Else creation of such topic will fail. Review comment: updated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542010745 run Integration Tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542009264 @sijie if user set the auto update strategy by new way, we will use the new setting, otherwise will use old setting. So this change can read the old setting, if user update the auto update strategy by new way and then update it by the old way(May be a different version of the broker), new change will be ignored. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf
codelipenghui commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf URL: https://github.com/apache/pulsar/pull/5359#issuecomment-542007371 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (7506092 -> 3a9854e)
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 7506092 Update schema-understand.md (#5233) add 3a9854e make skip all messages async. (#5375) No new revisions were added by this update. Summary of changes: .../broker/admin/impl/PersistentTopicsBase.java| 31 +- 1 file changed, 18 insertions(+), 13 deletions(-)
[GitHub] [pulsar] codelipenghui commented on issue #5378: Trim messages which less than mark delete position for message redelivery
codelipenghui commented on issue #5378: Trim messages which less than mark delete position for message redelivery URL: https://github.com/apache/pulsar/pull/5378#issuecomment-542007258 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui merged pull request #5375: [pulsar-admin]make skip all messages async
codelipenghui merged pull request #5375: [pulsar-admin]make skip all messages async URL: https://github.com/apache/pulsar/pull/5375 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 opened a new pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 opened a new pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227 Master Issue: #4737 ### Motivation Implement of [PIP-44](https://github.com/apache/pulsar/wiki/PIP-44%3A-Separate-schema-compatibility-checker-for-producer-and-consumer). ### Modifications - Add two flag : 1. schema_auto_update_compatibility_strategy for whether producer automatically creates schema under compatibility checking conditions 2. schema_compatibility_strategy for pulsar schema compatibility check - Consumer and producer has their own schema compatibility check, specific in [PIP-44](https://github.com/apache/pulsar/wiki/PIP-44%3A-Separate-schema-compatibility-checker-for-producer-and-consumer) ### Verifying this change Add the tests for it Does this pull request potentially affect one of the following parts: If yes was chosen, please highlight the changes Dependencies (does it add or upgrade a dependency): (no) The public API: (no) The schema: (yes) The default values of configurations: (no) The wire protocol: (no) The rest endpoints: (no) The admin cli options: (yes) Anything that affects deployment: (no) ### Documentation Does this pull request introduce a new feature? (yes) If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) If a feature is not applicable for documentation, explain why? If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 closed pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 closed pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542005621 > @congbobo184 @codelipenghui > > I see you introduced two settings, one is the flag to control auto update and the other flag is to set the compatibility check strategy. These two settings are fine. I am just wondering how are we going to do with the existing auto update schema strategy. Are we going to deprecated the existing one? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-542005621 > @congbobo184 @codelipenghui > > I see you introduced two settings, one is the flag to control auto update and the other flag is to set the compatibility check strategy. These two settings are fine. I am just wondering how are we going to do with the existing auto update schema strategy. Are we going to deprecated the existing one? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar-manager] branch master updated: Make default environment persistent (#197)
This is an automated email from the ASF dual-hosted git repository. guangning pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git The following commit(s) were added to refs/heads/master by this push: new fc0c422 Make default environment persistent (#197) fc0c422 is described below commit fc0c4225efdde3bd474035e7437d3f829b0650d5 Author: Enrico Olivelli AuthorDate: Tue Oct 15 02:47:10 2019 +0200 Make default environment persistent (#197) - By default start embedded HerdDB in standalone embedded mode - By default persist data locally - Update HerdDB client/server to 0.12.2 - Add more examples in the default application.properties --- .gitignore| 1 + gradle.properties | 2 +- src/main/resources/application.properties | 23 +++ 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/.gitignore b/.gitignore index 0e3afd8..a892a4f 100644 --- a/.gitignore +++ b/.gitignore @@ -44,3 +44,4 @@ build/ *.db front-end/data/ *.log +dbdata diff --git a/gradle.properties b/gradle.properties index 52301f6..5e3de66 100644 --- a/gradle.properties +++ b/gradle.properties @@ -14,4 +14,4 @@ apiMockitoVersion=1.7.1 mockitoJunit4Version=1.7.1 gsonVersion=2.8.2 postgresqlVersion=42.2.5 -herddbVersion=0.12.0 +herddbVersion=0.12.2 diff --git a/src/main/resources/application.properties b/src/main/resources/application.properties index 1e2e276..f92afb6 100644 --- a/src/main/resources/application.properties +++ b/src/main/resources/application.properties @@ -35,13 +35,20 @@ mybatis.type-aliases-package=io.streamnative.pulsar.manager #spring.datasource.password= # HerdDB JDBC Driver - spring.datasource.driver-class-name=herddb.jdbc.Driver - HerdDB - local in memory-only (check docs for starting an embedded server or embedded replicated server) - spring.datasource.url=jdbc:herddb:server:localhost:7000?server.base.dir=. - spring.datasource.schema=classpath:/META-INF/sql/herddb-schema.sql - spring.datasource.username=sa - spring.datasource.password=hdb - spring.datasource.initialization-mode=always +spring.datasource.driver-class-name=herddb.jdbc.Driver +# HerdDB - local in memory-only +#spring.datasource.url=jdbc:herddb:local +# HerdDB - start embedded server, data persisted on local disk (directory 'dbdata'), listening on localhost:7000 +spring.datasource.url=jdbc:herddb:server:localhost:7000?server.start=true&server.base.dir=dbdata +# HerdDB - connect to standalone server at localhost:7000 +#spring.datasource.url=jdbc:herddb:server:localhost:7000 +# HerdDB - connect to cluster, uses ZooKeeper for service discovery +#spring.datasource.url=jdbc:herddb:zookeeper:localhost:2181/herd + +spring.datasource.schema=classpath:/META-INF/sql/herddb-schema.sql +spring.datasource.username=sa +spring.datasource.password=hdb +spring.datasource.initialization-mode=always # postgresql configuration #spring.datasource.driver-class-name=org.postgresql.Driver @@ -57,7 +64,7 @@ zuul.routes.lookup.url=http://localhost:8080/lookup/ # pagehelper plugin #pagehelper.helperDialect=sqlite -# force 'mysql' for HerdDB +# force 'mysql' for HerdDB, comment out for postgresql pagehelper.helperDialect=mysql backend.directRequestBroker=true
[GitHub] [pulsar] Anonymitaet commented on issue #5296: [Doc] Update connector guide overall
Anonymitaet commented on issue #5296: [Doc] Update connector guide overall URL: https://github.com/apache/pulsar/pull/5296#issuecomment-541984267 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5386: Update presto to 0.226
aahmed-se commented on issue #5386: Update presto to 0.226 URL: https://github.com/apache/pulsar/pull/5386#issuecomment-541970815 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #5386: Update presto to 0.226
aahmed-se opened a new pull request #5386: Update presto to 0.226 URL: https://github.com/apache/pulsar/pull/5386 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat opened a new pull request #5385: Standalone should use crc32c by default as well
merlimat opened a new pull request #5385: Standalone should use crc32c by default as well URL: https://github.com/apache/pulsar/pull/5385 ### Motivation Fixed the default for digest type to match what we're already using in regular broker. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] addisonj edited a comment on issue #5383: deleteSubscription fails and leaves a dangling thread in jetty threadpool
addisonj edited a comment on issue #5383: deleteSubscription fails and leaves a dangling thread in jetty threadpool URL: https://github.com/apache/pulsar/issues/5383#issuecomment-541929505 ~Okay, after a bit of research, this looks like it likely is mostly a non issue in master with the changes to move to an async http handler. There may still be some exceptions in this code path that aren't getting handled properly, but this issue may well be fixed already~ Scratch that, it still is probably an issue due to the way it was converted This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] addisonj commented on issue #5383: deleteSubscription fails and leaves a dangling thread in jetty threadpool
addisonj commented on issue #5383: deleteSubscription fails and leaves a dangling thread in jetty threadpool URL: https://github.com/apache/pulsar/issues/5383#issuecomment-541929505 Okay, after a bit of research, this looks like it likely is mostly a non issue in master with the changes to move to an async http handler. There may still be some exceptions in this code path that aren't getting handled properly, but this issue may well be fixed already This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
sijie commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-541914451 @congbobo184 @codelipenghui I see you introduced two settings, one is the flag to control auto update and the other flag is to set the compatibility check strategy. These two settings are fine. I am just wondering how are we going to do with the existing auto update schema strategy. Are we going to deprecated the existing one? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Lanayx opened a new issue #5384: Topic compaction fails with 500 error
Lanayx opened a new issue #5384: Topic compaction fails with 500 error URL: https://github.com/apache/pulsar/issues/5384 **Describe the bug** I'm running pulsar 2.4.1 on minikube installed with Helm template. I create topic in `retention` namespace where some retention is set ( in my case --time 3h --size 1G). Then I sent 100 messages to the topic with the same key, and then wanted to enable compacting for this topic. Tried several ways, `bin/pulsar-admin topics compact persistent://public/retention/topic-2` returns ``` HTTP 500 Internal Server Error Reason: HTTP 500 Internal Server Error ``` Direct REST call `PUT /admin/v2/persistent/public/retention/topic-2/compaction HTTP/1.1` returns ``` --- An unexpected error occurred in the server --- Message: org.apache.pulsar.broker.PulsarServerException: java.lang.IllegalArgumentException: Param serviceUrl must not be blank. Stacktrace: org.apache.pulsar.broker.PulsarServerException: org.apache.pulsar.broker.PulsarServerException: java.lang.IllegalArgumentException: Param serviceUrl must not be blank. at org.apache.pulsar.broker.PulsarService.getCompactor(PulsarService.java:809) at org.apache.pulsar.broker.service.persistent.PersistentTopic.triggerCompaction(PersistentTopic.java:1775) at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.internalTriggerCompaction(PersistentTopicsBase.java:1262) at org.apache.pulsar.broker.admin.v2.PersistentTopics.compact(PersistentTopics.java:838) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:183) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:53) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
[GitHub] [pulsar] addisonj opened a new issue #5383: deleteSubscription fails and leaves a dangling thread in jetty threadpool
addisonj opened a new issue #5383: deleteSubscription fails and leaves a
dangling thread in jetty threadpool
URL: https://github.com/apache/pulsar/issues/5383
**Describe the bug**
For unknown reasons, a `deleteSubscription` call can fail and it leaves
around a dangling thread in the the broker, specifically the Jetty server
threadpool. If this happens too many times, the broker can start having issues
as Jetty is starved of threads.
**To Reproduce**
Not sure of exactly what happens, but here appears to be what usually gets
me into this state
Steps to reproduce the behavior:
1. Use `pulsar-flink` (https://github.com/streamnative/pulsar-flink) to
create a subscription with the `earliest` start position
2. Have your flink app fail very quickly
3. Using jstack, noticing the danging thread waiting to delete the
subscription
**Expected behavior**
The subscription deletion should either succeed or result in a proper
failure that doesn't leave a dangling thread.
**Screenshots**
In the logs, this is what I see when the call fails:
```
0:51:09.507 [pulsar-web-33-3] INFO
org.apache.pulsar.broker.service.persistent.PersistentSubscription -
[persistent://code/gerrit-events/bridge-talent-management:ref-updated][flink-pulsar-16a30186-aa62-4c3b-b576-612da2bfbf51]
Unsubscribing
00:51:09.507 [pulsar-web-33-3] INFO
org.apache.pulsar.broker.service.persistent.PersistentSubscription -
[persistent://code/gerrit-events/bridge-talent-management:ref-updated][flink-pulsar-16a30186-aa62-4c3b-b576-612da2bfbf51]
Successfully closed subscription
[ManagedCursorImpl{ledger=code/gerrit-events/persistent/bridge-talent-management%3Aref-updated,
name=flink-pulsar-16a30186-aa62-4c3b-b576-612da2bfbf51, ackPos=608:-1,
readPos=608:-1}]
00:51:09.508 [pulsar-web-33-3] INFO
org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper -
[code/gerrit-events/persistent/bridge-talent-management%3Aref-updated] Remove
consumer=flink-pulsar-16a30186-aa62-4c3b-b576-612da2bfbf51
00:51:09.510 [bookkeeper-ml-workers-OrderedExecutor-2-0] ERROR
org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught
java.lang.IllegalArgumentException: inconsistent range
at
java.util.concurrent.ConcurrentSkipListMap$SubMap.(ConcurrentSkipListMap.java:2620)
~[?:1.8.0_212]
at
java.util.concurrent.ConcurrentSkipListMap.subMap(ConcurrentSkipListMap.java:2078)
~[?:1.8.0_212]
at
org.apache.bookkeeper.mledger.util.RangeCache.removeRange(RangeCache.java:132)
~[org.apache.pulsar-managed-ledger-original-2.4.1-inst2.jar:2.4.1-inst2]
at
org.apache.bookkeeper.mledger.impl.EntryCacheImpl.invalidateEntries(EntryCacheImpl.java:151)
~[org.apache.pulsar-managed-ledger-original-2.4.1-inst2.jar:2.4.1-inst2]
at
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$7.operationComplete(ManagedLedgerImpl.java:777)
~[org.apache.pulsar-managed-ledger-original-2.4.1-inst2.jar:2.4.1-inst2]
at
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$7.operationComplete(ManagedLedgerImpl.java:765)
~[org.apache.pulsar-managed-ledger-original-2.4.1-inst2.jar:2.4.1-inst2]
at
org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper.lambda$null$13(MetaStoreImplZookeeper.java:308)
~[org.apache.pulsar-managed-ledger-original-2.4.1-inst2.jar:2.4.1-inst2]
at
org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32)
~[org.apache.pulsar-managed-ledger-original-2.4.1-inst2.jar:2.4.1-inst2]
at
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)
[org.apache.bookkeeper-bookkeeper-common-4.9.2.jar:4.9.2]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_212]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_212]
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
```
This then results in the following thread sticking around waiting for a
future to complete:
```
"pulsar-web-33-11" #95 prio=5 os_prio=0 tid=0x7ff06a5b nid=0xe2
waiting on condition [0x7ff0048b7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x8553c2b0> (a
java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at
java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at
java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at
org.apache.pulsa
[GitHub] [pulsar-client-go] xujianhai666 opened a new pull request #75: [ISSUE #74]refactor: gofmt code
xujianhai666 opened a new pull request #75: [ISSUE #74]refactor: gofmt code URL: https://github.com/apache/pulsar-client-go/pull/75 Motivation: code is not fmt, should be with gofmt Modifications: - gofmt once Fixes #74 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] xujianhai666 opened a new issue #74: fmt code
xujianhai666 opened a new issue #74: fmt code URL: https://github.com/apache/pulsar-client-go/issues/74 **Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] fmt code **Describe the solution you'd like** A clear and concise description of what you want to happen. gofmt **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** Add any other context or screenshots about the feature request here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] xujianhai666 opened a new pull request #73: [ISSUE #64]fix: avoid assert panic
xujianhai666 opened a new pull request #73: [ISSUE #64]fix: avoid assert panic URL: https://github.com/apache/pulsar-client-go/pull/73 ### Motivation when poll return nil, type assert will panic. ### Modifications - add assert check Fixes #64 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #5354: Test do not merge
aahmed-se closed pull request #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #5371: fix potential deadlock that can occur in addConsumer
jerrypeng commented on issue #5371: fix potential deadlock that can occur in addConsumer URL: https://github.com/apache/pulsar/pull/5371#issuecomment-541809404 rerun java8 tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liangyuanpeng commented on issue #5348: Load tls file fail on Windows
liangyuanpeng commented on issue #5348: Load tls file fail on Windows URL: https://github.com/apache/pulsar/issues/5348#issuecomment-541771171 @merlimat Thanks,Just solved the problem, I realized what a fool I was Now,i got the error `` org.apache.pulsar.broker.PulsarServerException: org.apache.pulsar.broker.PulsarServerException: java.lang.IllegalArgumentException: Private key must be accompanied by certificate chain `` I am checking my configuration If this is a silly question, I'm sorry for that This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5348: Load tls file fail on Windows
merlimat commented on issue #5348: Load tls file fail on Windows URL: https://github.com/apache/pulsar/issues/5348#issuecomment-541765194 @liangyuanpeng From the error looks like you're connecting with TLS to a non TLS port. Typically, the TLS port is `6651`, so the URL should be `pulsar+ssl://192.168.1.11:6651` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liangyuanpeng commented on issue #5348: Load tls file fail on Windows
liangyuanpeng commented on issue #5348: Load tls file fail on Windows URL: https://github.com/apache/pulsar/issues/5348#issuecomment-541754848 Update, I confirm that my problem is on configuring TLS transport and can i not use proxy? Actually, I tried to start proxy,but i just got error ``` 23:40:17.052 [main] INFO org.apache.pulsar.broker.authorization.AuthorizationService - Authorization is disabled 2019-10-14 11:40:17,127 [sun.misc.Launcher$AppClassLoader@58d25a40] error Uncaught exception in thread main: Failed to bind Pulsar Proxy on port 6650 ``` and my ``proxy.conf`` ``` # The ZooKeeper quorum connection string (as a comma-separated list) zookeeperServers=192.168.1.11 # Configuration store connection string (as a comma-separated list) configurationStoreServers=192.168.1.11 # if Service Discovery is Disabled this url should point to the discovery service provider. brokerServiceURL=pulsar://192.168.1.11:6650 brokerServiceURLTLS= # These settings are unnecessary if `zookeeperServers` is specified brokerWebServiceURL=http://192.168.1.11:8080 brokerWebServiceURLTLS= ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5376: [Issue 3009][docs] Add presto sql REST APIs doc
wolfstudy commented on issue #5376: [Issue 3009][docs] Add presto sql REST APIs doc URL: https://github.com/apache/pulsar/pull/5376#issuecomment-541747624 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5382: Update client-libraries-go.md
wolfstudy commented on issue #5382: Update client-libraries-go.md URL: https://github.com/apache/pulsar/pull/5382#issuecomment-541747003 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #4811: [Issue: 3161][Go client] Return Message ID for send
wolfstudy commented on issue #4811: [Issue: 3161][Go client] Return Message ID for send URL: https://github.com/apache/pulsar/pull/4811#issuecomment-541744951 ping @massakam PTAL again thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #65: [Issue 64] fix bug: type assert cause panic
wolfstudy commented on issue #65: [Issue 64] fix bug: type assert cause panic URL: https://github.com/apache/pulsar-client-go/pull/65#issuecomment-541744405 ping @mileschao PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on a change in pull request #65: [Issue 64] fix bug: type assert cause panic
wolfstudy commented on a change in pull request #65: [Issue 64] fix bug: type
assert cause panic
URL: https://github.com/apache/pulsar-client-go/pull/65#discussion_r334536144
##
File path: pulsar/impl_partition_producer.go
##
@@ -293,14 +293,17 @@ func (p *partitionProducer) internalFlushCurrentBatch() {
func (p *partitionProducer) internalFlush(fr *flushRequest) {
p.internalFlushCurrentBatch()
- pi := p.pendingQueue.PeekLast().(*pendingItem)
- pi.sendRequests = append(pi.sendRequests, &sendRequest{
- msg: nil,
- callback: func(id MessageID, message *ProducerMessage, e error)
{
- fr.err = e
- fr.waitGroup.Done()
- },
- })
+ pi, ok := p.pendingQueue.PeekLast().(*pendingItem)
+ if ok {
+ pi.sendRequests = append(pi.sendRequests, &sendRequest{
+ msg: nil,
+ callback: func(id MessageID, message *ProducerMessage,
e error) {
+ fr.err = e
+ fr.waitGroup.Done()
+ },
+ })
+ }
+
Review comment:
If not ok, `fr.waitGroup.Done()` will not be executed, `wg.wait()` in send
will not be released, and sending will be blocked indefinitely
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #69: [ISSUE #68][feat]add Option config for init (#68)
wolfstudy commented on issue #69: [ISSUE #68][feat]add Option config for init (#68) URL: https://github.com/apache/pulsar-client-go/pull/69#issuecomment-541742558 ping @merlimat PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5381: [Issue 5380][deployment]Fixing pod anti-affinity rules in Kubernetes files including Helm chart
jiazhai commented on issue #5381: [Issue 5380][deployment]Fixing pod anti-affinity rules in Kubernetes files including Helm chart URL: https://github.com/apache/pulsar/pull/5381#issuecomment-541731805 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5376: [Issue 3009][docs] Add presto sql REST APIs doc
jiazhai commented on issue #5376: [Issue 3009][docs] Add presto sql REST APIs doc URL: https://github.com/apache/pulsar/pull/5376#issuecomment-541730453 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar-manager] branch master updated (44273c9 -> f250835)
This is an automated email from the ASF dual-hosted git repository. guangning pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git. from 44273c9 Support HerdDB database (#194) add f250835 Use only standard well-known repositories and fix license header in build.gradle (#200) No new revisions were added by this update. Summary of changes: build.gradle | 18 -- 1 file changed, 16 insertions(+), 2 deletions(-)
[GitHub] [pulsar] boncheff opened a new pull request #5382: Update client-libraries-go.md
boncheff opened a new pull request #5382: Update client-libraries-go.md URL: https://github.com/apache/pulsar/pull/5382 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5296: [Doc] Update connector guide overall
Anonymitaet commented on issue #5296: [Doc] Update connector guide overall URL: https://github.com/apache/pulsar/pull/5296#issuecomment-541619879 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5296: [Doc] Update connector guide overall
Anonymitaet commented on issue #5296: [Doc] Update connector guide overall URL: https://github.com/apache/pulsar/pull/5296#issuecomment-541617591 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5296: [Doc] Update connector guide overall
Anonymitaet commented on issue #5296: [Doc] Update connector guide overall URL: https://github.com/apache/pulsar/pull/5296#issuecomment-541601493 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] a1onzo commented on issue #2799: Receiving "Connection Already Closed Error on Creating Producer"
a1onzo commented on issue #2799: Receiving "Connection Already Closed Error on Creating Producer" URL: https://github.com/apache/pulsar/issues/2799#issuecomment-541567106 hi, is the issue above solved? i am confused with that error "Error Checking/Getting Partition Metadata while creating producer" too @gurgen2727 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5296: [Doc] Update connector guide overall
Anonymitaet commented on issue #5296: [Doc] Update connector guide overall URL: https://github.com/apache/pulsar/pull/5296#issuecomment-541545145 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] yittg commented on a change in pull request #5165: [PIP-43] Support producer to send msg with different schema
yittg commented on a change in pull request #5165: [PIP-43] Support producer to
send msg with different schema
URL: https://github.com/apache/pulsar/pull/5165#discussion_r334362812
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
##
@@ -1289,16 +1364,15 @@ private void batchMessageAndSend() {
}
if (!batchMessageContainer.isEmpty()) {
try {
+List opSendMsgs;
if (batchMessageContainer.isMultiBatches()) {
-List opSendMsgs =
batchMessageContainer.createOpSendMsgs();
-for (OpSendMsg opSendMsg : opSendMsgs) {
-processOpSendMsg(opSendMsg);
-}
+opSendMsgs = batchMessageContainer.createOpSendMsgs();
} else {
-OpSendMsg opSendMsg =
batchMessageContainer.createOpSendMsg();
-if (opSendMsg != null) {
-processOpSendMsg(opSendMsg);
-}
+opSendMsgs =
Collections.singletonList(batchMessageContainer.createOpSendMsg());
Review comment:
It's ok to leave it as it is.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] yittg commented on a change in pull request #5165: [PIP-43] Support producer to send msg with different schema
yittg commented on a change in pull request #5165: [PIP-43] Support producer to
send msg with different schema
URL: https://github.com/apache/pulsar/pull/5165#discussion_r334361724
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerBase.java
##
@@ -73,6 +81,14 @@ public MessageId send(T message) throws
PulsarClientException {
return new TypedMessageBuilderImpl<>(this, schema);
}
+// TODO: same parameterized type `T` with producer for the moment,
Review comment:
Should I just edit the master issue, with a checklist, or new a issue for
each TODO?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] yittg commented on a change in pull request #5165: [PIP-43] Support producer to send msg with different schema
yittg commented on a change in pull request #5165: [PIP-43] Support producer to
send msg with different schema
URL: https://github.com/apache/pulsar/pull/5165#discussion_r334355056
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
##
@@ -1356,6 +1342,58 @@ protected void handleGetSchema(CommandGetSchema
commandGetSchema) {
});
}
+@Override
+protected void handleGetOrCreateSchema(CommandGetOrCreateSchema
commandGetOrCreateSchema) {
+if (log.isDebugEnabled()) {
+log.debug("Received CommandGetOrCreateSchema call from {}",
remoteAddress);
+}
+long requestId = commandGetOrCreateSchema.getRequestId();
+String topicName = commandGetOrCreateSchema.getTopic();
+SchemaData schemaData =
getSchema(commandGetOrCreateSchema.getSchema());
+SchemaData schema = schemaData.getType() == SchemaType.NONE ? null :
schemaData;
+service.getTopicIfExists(topicName).thenAccept(topicOpt -> {
+if (topicOpt.isPresent()) {
+Topic topic = topicOpt.get();
+CompletableFuture schemaVersionFuture =
tryAddSchema(topic, schema);
+schemaVersionFuture.exceptionally(ex -> {
+ServerError errorCode =
BrokerServiceException.getClientErrorCode(ex);
+
ctx.writeAndFlush(Commands.newGetOrCreateSchemaResponseError(
+requestId, errorCode, ex.getMessage()));
+return null;
+}).thenAccept(schemaVersion -> {
+
ctx.writeAndFlush(Commands.newGetOrCreateSchemaResponse(
+requestId, schemaVersion));
+});
+} else {
+ctx.writeAndFlush(Commands.newGetOrCreateSchemaResponseError(
+requestId, ServerError.TopicNotFound, "Topic not
found"));
+}
+}).exceptionally(ex -> {
+ServerError errorCode =
BrokerServiceException.getClientErrorCode(ex);
+ctx.writeAndFlush(Commands.newGetOrCreateSchemaResponseError(
+requestId, errorCode, ex.getMessage()));
+return null;
+});
+}
+
+private CompletableFuture tryAddSchema(Topic topic,
SchemaData schema) {
+if (schema != null) {
+return topic.addSchema(schema);
+} else {
+return topic.hasSchema().thenCompose((hasSchema) -> {
+log.info("[{}] {} configured with schema {}",
Review comment:
`producer_id` not carried in `CommandGetOrCreateSchema`, actually need it?
Or a `-1` as placeholder?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] yittg commented on a change in pull request #5165: [PIP-43] Support producer to send msg with different schema
yittg commented on a change in pull request #5165: [PIP-43] Support producer to
send msg with different schema
URL: https://github.com/apache/pulsar/pull/5165#discussion_r334352968
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
##
@@ -1312,10 +1386,16 @@ private void batchMessageAndSend() {
private void processOpSendMsg(OpSendMsg op) {
try {
-batchMessageContainer.clear();
+if (op.msg != null && isBatchMessagingEnabled()) {
Review comment:
`op.msg` means a single non-batch message. we should flush the current batch
if needed.
`batchMessageContainer.clear();` was moved into `batchMessageAndSend`.
Then `processOpSendMsg` not only for batch message.
Don't we need to do the flush, to keep the order or something else?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5374: [Issue 5362][website]Update logo path for Pulsar users
Jennifer88huang commented on issue #5374: [Issue 5362][website]Update logo path for Pulsar users URL: https://github.com/apache/pulsar/pull/5374#issuecomment-541527874 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5372: Use StandardCharsets.UTF_8 for converting String to bytes
sijie commented on issue #5372: Use StandardCharsets.UTF_8 for converting String to bytes URL: https://github.com/apache/pulsar/pull/5372#issuecomment-541527551 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5374: [Issue 5362][website]Update logo path for Pulsar users
sijie commented on issue #5374: [Issue 5362][website]Update logo path for Pulsar users URL: https://github.com/apache/pulsar/pull/5374#issuecomment-541525072 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5378: Trim messages which less than mark delete position for message redelivery
sijie commented on issue #5378: Trim messages which less than mark delete position for message redelivery URL: https://github.com/apache/pulsar/pull/5378#issuecomment-541524805 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Guchman commented on issue #5366: Deploy function without starting it
Guchman commented on issue #5366: Deploy function without starting it URL: https://github.com/apache/pulsar/issues/5366#issuecomment-541524677 @aahmed-se I mean external control of function, stop and start that with API. Anyway, it's may be useful to deploy full stream pipeline before starting that Actually I have stream processing pipeline, but complex one, so I need to merge two streams sequentially. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
