[GitHub] [pulsar] zwzch commented on issue #5610: Fluentd connector for Apache Pulsar
zwzch commented on issue #5610: Fluentd connector for Apache Pulsar URL: https://github.com/apache/pulsar/issues/5610#issuecomment-554610825 I made a [plugin](https://github.com/zwzch/fluent-plugin-pulsar) for this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] srkukarni opened a new pull request #5674: Parametrize grpc and metrics ports so that they can be configured
srkukarni opened a new pull request #5674: Parametrize grpc and metrics ports so that they can be configured URL: https://github.com/apache/pulsar/pull/5674 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation Currently when running functions as kubernetes pods, the grpc/metrics port that the container binds to is hardcoded. This pr allows this to be configurable. ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5572: Fix create consumer on partitioned topic while disable topic auto creation.
codelipenghui commented on issue #5572: Fix create consumer on partitioned topic while disable topic auto creation. URL: https://github.com/apache/pulsar/pull/5572#issuecomment-554583693 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] merlimat commented on a change in pull request #101: Add client function to list topics in a namespace.
merlimat commented on a change in pull request #101: Add client function to
list topics in a namespace.
URL: https://github.com/apache/pulsar-client-go/pull/101#discussion_r347062977
##
File path: pulsar/client_impl_test.go
##
@@ -222,3 +221,84 @@ func TestTopicPartitions(t *testing.T) {
assert.Equal(t, len(partitions), 1)
assert.Equal(t, partitions[0], topic)
}
+
+func TestNamespaceTopicsNamespaceDoesNotExit(t *testing.T) {
+ c, err := NewClient(ClientOptions{
+ URL: serviceURL,
+ })
+ if err != nil {
+ t.Errorf("failed to create client error: %+v", err)
+ return
+ }
+ defer c.Close()
+ ci := c.(*client)
+
+ // fetch from namespace that does not exist
+ name := generateRandomName()
+ topics, err := ci.namespaceTopics(fmt.Sprintf("%s/%s", name, name))
+ assert.Equal(t, 0, len(topics))
+}
+
+func TestNamespaceTopics(t *testing.T) {
+ name := generateRandomName()
+ namespace := fmt.Sprintf("public/%s", name)
+ namespaceUrl :=
fmt.Sprintf("http://localhost:8080/admin/v2/namespaces/%s;, namespace)
Review comment:
In `test_helper.go` we have:
```
serviceURL= "pulsar://localhost:6650"
serviceURLTLS = "pulsar+ssl://localhost:6651
```
we could add the `http://localhost:8080` there as well
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-go] cckellogg opened a new pull request #101: Add client function to list topics in a namespace.
cckellogg opened a new pull request #101: Add client function to list topics in a namespace. URL: https://github.com/apache/pulsar-client-go/pull/101 ### Motivation This patch is needed to implement the regex consumer. Doing this in a separate patch to make reviews easier. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #5599: [pulsar-broker] close managed-ledgers before giving up bundle ownership to avoid bad zk-version
rdhabalia commented on issue #5599: [pulsar-broker] close managed-ledgers before giving up bundle ownership to avoid bad zk-version URL: https://github.com/apache/pulsar/pull/5599#issuecomment-554475085 @merlimat can you please review it.. somehow we are seeing this issue with 2.4 very often when we restart the broker. we also want to merge #5604 as part of this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #5628: Fix: Netty threads may still linger after client close
jerrypeng commented on issue #5628: Fix: Netty threads may still linger after client close URL: https://github.com/apache/pulsar/pull/5628#issuecomment-554457080 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5642: Add Github workflow for gated checkin
merlimat commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-554456806 @aahmed-se Might also be interesting to add a cache for Maven deps: https://help.github.com/en/actions/automating-your-workflow-with-github-actions/caching-dependencies-to-speed-up-workflows This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5590: [Issue 5474][pulsar-io-debezium] Support CDC Connector for MongoDB
tuteng commented on issue #5590: [Issue 5474][pulsar-io-debezium] Support CDC Connector for MongoDB URL: https://github.com/apache/pulsar/pull/5590#issuecomment-554417838 @jiazhai @murong00 PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5590: [Issue 5474][pulsar-io-debezium] Support CDC Connector for MongoDB
tuteng commented on issue #5590: [Issue 5474][pulsar-io-debezium] Support CDC Connector for MongoDB URL: https://github.com/apache/pulsar/pull/5590#issuecomment-554408485 Thank you for your contributing This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng edited a comment on issue #5590: [Issue 5474][pulsar-io-debezium] Support CDC Connector for MongoDB
tuteng edited a comment on issue #5590: [Issue 5474][pulsar-io-debezium] Support CDC Connector for MongoDB URL: https://github.com/apache/pulsar/pull/5590#issuecomment-554408485 Thank you for your contributing @huangdx0726 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ArtemShaban opened a new issue #5673: pulsar-broker] fix ConcurrentModificationException while load-report serialization/updateLocalBrokerData
ArtemShaban opened a new issue #5673: pulsar-broker] fix
ConcurrentModificationException while load-report
serialization/updateLocalBrokerData
URL: https://github.com/apache/pulsar/issues/5673
**Issue desctiption:**
Pulsar version = 2.4.1
Sometimes I see 2 ConcurrentModificationException in broker logs.
```
06:24:51.990 [pulsar-load-manager-4-1] WARN
org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Error when
attempting to update local broker data: {}
java.util.ConcurrentModificationException: null
#011at java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)
~[?:1.8.0_231]
#011at java.util.HashMap$EntryIterator.next(HashMap.java:1479)
~[?:1.8.0_231]
#011at java.util.HashMap$EntryIterator.next(HashMap.java:1477)
~[?:1.8.0_231]
#011at
org.apache.pulsar.policies.data.loadbalancer.LocalBrokerData.updateBundleData(LocalBrokerData.java:175)
~[org.apache.pulsar-pulsar-common-2.4.1.jar:2.4.1]
#011at
org.apache.pulsar.policies.data.loadbalancer.LocalBrokerData.update(LocalBrokerData.java:121)
~[org.apache.pulsar-pulsar-common-2.4.1.jar:2.4.1]
#011at
org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.updateLocalBrokerData(ModularLoadManagerImpl.java:847)
~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at
org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.writeBrokerDataOnZooKeeper(ModularLoadManagerImpl.java:860)
~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at
org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerWrapper.writeLoadReportOnZookeeper(ModularLoadManagerWrapper.java:120)
~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at
org.apache.pulsar.broker.loadbalance.LoadReportUpdaterTask.run(LoadReportUpdaterTask.java:41)
~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_231]
#011at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
~[?:1.8.0_231]
#011at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
~[?:1.8.0_231]
#011at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
~[?:1.8.0_231]
#011at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_231]
#011at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_231]
#011at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
#011at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
```
```
06:24:51.990 [pulsar-load-manager-4-1] WARN
org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Error
writing broker data on ZooKeeper: {}
com.fasterxml.jackson.databind.JsonMappingException: (was
java.util.ConcurrentModificationException) (through reference chain:
org.apache.pulsar.policies.data.loadbalancer.LocalBrokerData["lastStats"])
#011at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:394)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:353)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:727)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3906)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
com.fasterxml.jackson.databind.ObjectMapper.writeValueAsBytes(ObjectMapper.java:3244)
~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at
org.apache.pulsar.policies.data.loadbalancer.JSONWritable.getJsonBytes(JSONWritable.java:39)
~[org.apache.pulsar-pulsar-common-2.4.1.jar:2.4.1]
#011at
[GitHub] [pulsar] frejonb opened a new issue #5672: [C++ client] Deadletter queue functionality
frejonb opened a new issue #5672: [C++ client] Deadletter queue functionality
URL: https://github.com/apache/pulsar/issues/5672
Is anyone working on porting DLQ from the Java client to Cpp
https://github.com/apache/pulsar/pull/2508? I'd like to help/contribute.
I'm thinking on the following API
```c++
Client client("pulsar://localhost:6650");
Consumer consumer;
ConsumerConfiguration config;
config.setMaxRedeliverCount(10);
config.deadLetterTopic("my-dlq-topic");
Result result = client.subscribe("persistent://prop/r1/ns1/my-topic",
"consumer-1", config, consumer);
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5671: [Doc] Add HBase sink connector guide
Anonymitaet commented on issue #5671: [Doc] Add HBase sink connector guide URL: https://github.com/apache/pulsar/pull/5671#issuecomment-554277117 @tuteng could you please help review? Thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet opened a new pull request #5671: [Doc] Add HBase sink connector guide
Anonymitaet opened a new pull request #5671: [Doc] Add HBase sink connector guide URL: https://github.com/apache/pulsar/pull/5671 Fix https://github.com/apache/pulsar/issues/5667 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5670: [Doc] Re-organize built-in connector guide
Anonymitaet commented on issue #5670: [Doc] Re-organize built-in connector guide URL: https://github.com/apache/pulsar/pull/5670#issuecomment-554267107 @tuteng could you please help review? Thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet opened a new pull request #5670: [Doc] Re-organize built-in connector guide
Anonymitaet opened a new pull request #5670: [Doc] Re-organize built-in connector guide URL: https://github.com/apache/pulsar/pull/5670 Fix https://github.com/apache/pulsar/issues/5667 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
