[GitHub] [pulsar] wolfstudy merged pull request #7847: Update release.json and versions.json
wolfstudy merged pull request #7847: URL: https://github.com/apache/pulsar/pull/7847 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang edited a comment on pull request #7848: URL: https://github.com/apache/pulsar/pull/7848#issuecomment-675890018 @sijie @jiazhai CI failed when build pulsar image all the time, error information ```shell [ERROR] The command '/bin/sh -c /pulsar/bin/install-pulsar-client-27.sh' returned a non-zero code: 1 [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 20:24 min [INFO] Finished at: 2020-08-19T06:51:53Z [INFO] [ERROR] Failed to execute goal com.spotify:dockerfile-maven-plugin:1.4.13:build (default) on project pulsar-docker-image: Could not build image: The command '/bin/sh -c /pulsar/bin/install-pulsar-client-27.sh' returned a non-zero code: 1 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException ##[error]Process completed with exit code 1. ``` It seems have nothing to do with this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang removed a comment on pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang removed a comment on pull request #7848: URL: https://github.com/apache/pulsar/pull/7848#issuecomment-675854276 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang commented on pull request #7848: URL: https://github.com/apache/pulsar/pull/7848#issuecomment-675890018 @sijie @jiazhai CI failed when build pulsar image all the time, error information ```shell [ERROR] The command '/bin/sh -c /pulsar/bin/install-pulsar-client-27.sh' returned a non-zero code: 1 [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 20:24 min [INFO] Finished at: 2020-08-19T06:51:53Z [INFO] [ERROR] Failed to execute goal com.spotify:dockerfile-maven-plugin:1.4.13:build (default) on project pulsar-docker-image: Could not build image: The command '/bin/sh -c /pulsar/bin/install-pulsar-client-27.sh' returned a non-zero code: 1 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException ##[error]Process completed with exit code 1. ``` It seems have nothing to do with this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-go] sneerin opened a new issue #351: Ability to receive/wait for nacked messages, pause a topic
sneerin opened a new issue #351: URL: https://github.com/apache/pulsar-client-go/issues/351 Currently, if you send 5 messages, and back #5 , it will arrive the last one, breaking the order. i want to have a use-case, when once a message nacked, the order is not broken, the topic is on pause, until the message is returned or completely failed to DLQ. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang commented on pull request #7848: URL: https://github.com/apache/pulsar/pull/7848#issuecomment-675873392 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-go] sneerin opened a new issue #350: ReceiverQueueSize -1 leads to crash
sneerin opened a new issue #350:
URL: https://github.com/apache/pulsar-client-go/issues/350
Expected behavior
According to doc -1 is a correct value
Tell us what should happen
Actual behavior
App is crashing
Tell us what happens instead
Steps to reproduce
options := pulsar.ConsumerOptions{
Topic: topic,
SubscriptionName: cg,
Type: pulsar.Shared,
ReceiverQueueSize: -1,
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang commented on pull request #7848: URL: https://github.com/apache/pulsar/pull/7848#issuecomment-675854276 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang removed a comment on pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang removed a comment on pull request #7848: URL: https://github.com/apache/pulsar/pull/7848#issuecomment-675838846 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] wolfstudy merged pull request #7835: Add 2.6.2 docs
wolfstudy merged pull request #7835: URL: https://github.com/apache/pulsar/pull/7835 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7821: Support deduplication on topic level
codelipenghui commented on pull request #7821: URL: https://github.com/apache/pulsar/pull/7821#issuecomment-675840887 @315157973 Could you please resolve the conflicts? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang commented on pull request #7848: URL: https://github.com/apache/pulsar/pull/7848#issuecomment-675838846 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (23d795c -> 809c6be)
This is an automated email from the ASF dual-hosted git repository. sanjeevrk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 23d795c Fix pending batchIndexAcks bitSet batchSize in PersistentAcknowledgmentsGroupingTracker (#7828) add 809c6be Limit cpu count for proxy unit test cases (#7845) No new revisions were added by this update. Summary of changes: .../workflows/ci-integration-backwards-compatibility.yaml| 2 +- .github/workflows/ci-integration-cli.yaml| 2 +- .github/workflows/ci-integration-function-state.yaml | 2 +- .github/workflows/ci-integration-messaging.yaml | 2 +- .github/workflows/ci-integration-process.yaml| 2 +- .github/workflows/ci-integration-schema.yaml | 2 +- .github/workflows/ci-integration-sql.yaml| 2 +- .github/workflows/ci-integration-standalone.yaml | 2 +- .github/workflows/ci-integration-thread.yaml | 2 +- .github/workflows/ci-integration-tiered-filesystem.yaml | 2 +- .github/workflows/ci-integration-tiered-jcloud.yaml | 2 +- .github/workflows/ci-license.yaml| 2 +- .github/workflows/ci-unit-broker-broker-gp1.yaml | 6 +- .github/workflows/ci-unit-broker-client-api.yaml | 12 pulsar-client-cpp/python/setup.py| 4 ++-- .../apache/pulsar/proxy/server/ProxyIsAHttpProxyTest.java| 3 +++ 16 files changed, 30 insertions(+), 19 deletions(-)
[pulsar] branch master updated (23d795c -> 809c6be)
This is an automated email from the ASF dual-hosted git repository. sanjeevrk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 23d795c Fix pending batchIndexAcks bitSet batchSize in PersistentAcknowledgmentsGroupingTracker (#7828) add 809c6be Limit cpu count for proxy unit test cases (#7845) No new revisions were added by this update. Summary of changes: .../workflows/ci-integration-backwards-compatibility.yaml| 2 +- .github/workflows/ci-integration-cli.yaml| 2 +- .github/workflows/ci-integration-function-state.yaml | 2 +- .github/workflows/ci-integration-messaging.yaml | 2 +- .github/workflows/ci-integration-process.yaml| 2 +- .github/workflows/ci-integration-schema.yaml | 2 +- .github/workflows/ci-integration-sql.yaml| 2 +- .github/workflows/ci-integration-standalone.yaml | 2 +- .github/workflows/ci-integration-thread.yaml | 2 +- .github/workflows/ci-integration-tiered-filesystem.yaml | 2 +- .github/workflows/ci-integration-tiered-jcloud.yaml | 2 +- .github/workflows/ci-license.yaml| 2 +- .github/workflows/ci-unit-broker-broker-gp1.yaml | 6 +- .github/workflows/ci-unit-broker-client-api.yaml | 12 pulsar-client-cpp/python/setup.py| 4 ++-- .../apache/pulsar/proxy/server/ProxyIsAHttpProxyTest.java| 3 +++ 16 files changed, 30 insertions(+), 19 deletions(-)
[GitHub] [pulsar] srkukarni merged pull request #7845: Limit cpu count for proxy unit test cases
srkukarni merged pull request #7845: URL: https://github.com/apache/pulsar/pull/7845 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
zhanghaou commented on pull request #7839: URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675830637 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] zhanghaou removed a comment on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
zhanghaou removed a comment on pull request #7839: URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675774838 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] zymap commented on issue #7754: Remove redundant ci job
zymap commented on issue #7754: URL: https://github.com/apache/pulsar/issues/7754#issuecomment-675830186 Yes. JIRA issue: https://issues.apache.org/jira/browse/INFRA-20708 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang opened a new pull request #7848: take seek time into account in Reader#hasMessageAvailable
aloyszhang opened a new pull request #7848: URL: https://github.com/apache/pulsar/pull/7848 Fixes #7796 ### Motivation This PR takes seek time into account in Reader#hasMessageAvailable. ### Modifications 1. mark the timestamp in `Reader` whenever `seek(timestamp)` 2. compare the marked timestamp with the publish time of last messag to decide `Reader#hasMessageAvailable` give true or false This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] wolfstudy commented on pull request #7847: Update release.json and versions.json
wolfstudy commented on pull request #7847: URL: https://github.com/apache/pulsar/pull/7847#issuecomment-675809269  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #7838: Add The Ability To Set Log Level From Environment Variables In Docker
jiazhai commented on issue #7838: URL: https://github.com/apache/pulsar/issues/7838#issuecomment-675807753 May need this change (https://github.com/apache/pulsar/pull/7789) as well. It will be available in 2.6.1 release. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] wolfstudy opened a new pull request #7847: Update release.json and versions.json
wolfstudy opened a new pull request #7847: URL: https://github.com/apache/pulsar/pull/7847 Signed-off-by: xiaolong.ran ### Motivation Currently, when the official website 2.6.1 document is released, we should first let 2.6.0 continue as the default version, wait for version 2.6.1 to appear in the historical version, and then try to use version 2.6.1 as the default version. If we directly publish the 2.6.1 version of the document, the search function will be unavailable. ### Modifications - Update release.json - Update versions.json This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #7846: Multi-Topic subscription across tenant and namespace using regex in Java client throws IllegalArgumentException
jiazhai commented on issue #7846: URL: https://github.com/apache/pulsar/issues/7846#issuecomment-675801881 @rnowacoski currently regex subscription supports the topics in the same namespace. change this issue into a feature request. from my view, it is not hard to do, maybe we should pay more attention to the authorization. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] rnowacoski opened a new issue #7846: Multi-Topic subscription across tenant and namespace using regex in Java client is broken
rnowacoski opened a new issue #7846:
URL: https://github.com/apache/pulsar/issues/7846
**Describe the bug**
When using a regex to create a consumer and a regex pattern is present in
the tenant or namespace section of the pattern this error is thrown.
```
java.lang.IllegalArgumentException: Invalid named entity: \w+
at
org.apache.pulsar.common.naming.NamedEntity.checkName(NamedEntity.java:39)
at
org.apache.pulsar.common.naming.NamespaceName.validateNamespaceName(NamespaceName.java:179)
at
org.apache.pulsar.common.naming.NamespaceName.get(NamespaceName.java:53)
at org.apache.pulsar.common.naming.TopicName.(TopicName.java:151)
at org.apache.pulsar.common.naming.TopicName.(TopicName.java:38)
at org.apache.pulsar.common.naming.TopicName$1.load(TopicName.java:63)
at org.apache.pulsar.common.naming.TopicName$1.load(TopicName.java:60)
at
org.apache.pulsar.shade.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at
org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2276)
at
org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at
org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at
org.apache.pulsar.shade.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
at
org.apache.pulsar.shade.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3973)
at
org.apache.pulsar.shade.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4957)
at org.apache.pulsar.common.naming.TopicName.get(TopicName.java:88)
at
org.apache.pulsar.client.impl.PulsarClientImpl.patternTopicSubscribeAsync(PulsarClientImpl.java:403)
at
org.apache.pulsar.client.impl.PulsarClientImpl.subscribeAsync(PulsarClientImpl.java:333)
at
org.apache.pulsar.client.impl.ConsumerBuilderImpl.subscribeAsync(ConsumerBuilderImpl.java:142)
at
org.apache.pulsar.client.impl.ConsumerBuilderImpl.subscribe(ConsumerBuilderImpl.java:99)
```
Example pattern: `persistent://\w+/Test/Test1` or
`persistent://Test/\w+/Test1`
This is because `PulsarClientImpl.patternTopicSubscribeAsync` is called when
creating a consumer using a Pattern. This calls `topicName.get(regex);` where
regex is the Pattern passed in. This method attempts to validate the tenant and
namespace names against this regex `^[-=:.\\w]*$` in
org.apache.pulsar.common.naming.NamedEntity. This fails for any regex pattern.
The namespace name is needed in the current code because it is used by
`LookupService.getTopicsUnderNamespace` to find the topics to subscribe to.
**To Reproduce**
Steps to reproduce the behavior:
1. Create a Consumer like below
```
client
.newConsumer(Schema.BYTES)
.topicsPattern(Pattern.compile("persistent://\w+/Test/Test1"))
.subscribe()
```
2. Observe `java.lang.IllegalArgumentException: Invalid named entity: \w+`
is thrown
**Expected behavior**
I given a wildcard in a tenant or namespace I would expect the code to look
up all tenants/namespaces that match and then lookup all topics to subscribe to.
**Proposed Solution**
1. LookupService implementations should implement a method like
`getNamespaces()` that will return all tenant/namespaces.
2. `PulsarClientImpl.patternTopicSubscribeAsync` should filter these
namespaces based on the regex given
3. For each renaming namespace that matches the given regex
`PulsarClientImpl.patternTopicSubscribeAsync` should call
`LookupService.getTopicsUnderNamespace`
4. List of topics for each matching namespace should be combined into one
list and the consumer should subscribe to all of those topics
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
zhanghaou commented on pull request #7839: URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675789050 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on pull request #7840: [Client]Add autoPartitionsUpdateInterval for producer and consumer
RobertIndie commented on pull request #7840: URL: https://github.com/apache/pulsar/pull/7840#issuecomment-675787881 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
zhanghaou commented on pull request #7839: URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675774838 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se opened a new pull request #7845: Limit cpu count for proxy unit test cases
aahmed-se opened a new pull request #7845: URL: https://github.com/apache/pulsar/pull/7845 * Limit cpu count for running unit tests. * CI env are generally limited in core count and requesting too many resources causes failures. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] nlu90 opened a new pull request #7844: allow kubernetes runtime to customize function instance class path
nlu90 opened a new pull request #7844: URL: https://github.com/apache/pulsar/pull/7844 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] srkukarni opened a new pull request #7843: Add More producer knobs
srkukarni opened a new pull request #7843: URL: https://github.com/apache/pulsar/pull/7843 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation This pr adds more Producer Config knobs to FunctionConfig and SourceConfig. ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] merlimat opened a new pull request #7842: [C++] Allow to configure KeyShared with out of order delivery
merlimat opened a new pull request #7842: URL: https://github.com/apache/pulsar/pull/7842 ### Motivation Similar to #7188, allow to configure key-shared with relaxed ordering in C++ client This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] massakam opened a new pull request #7841: [broker] Fix deadlock when adding consumer
massakam opened a new pull request #7841: URL: https://github.com/apache/pulsar/pull/7841 ### Motivation A deadlock occurred on our Pulsar 2.4.2 broker server. The cause is the following two threads: ``` "prometheus-stats-36-1" #410 prio=5 os_prio=0 tid=0x7f4b70019800 nid=0x30ca waiting for monitor entry [0x7f4bbe3b7000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.pulsar.broker.service.persistent.PersistentSubscription.getNumberOfEntriesDelayed(PersistentSubscription.java:1013) - waiting to lock <0x7f913d098dd0> (a org.apache.pulsar.broker.service.persistent.PersistentSubscription) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$getTopicStats$8(NamespaceStatsAggregator.java:129) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$523/2109257042.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.getTopicStats(NamespaceStatsAggregator.java:122) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$null$0(NamespaceStatsAggregator.java:64) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$521/1017174654.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$null$1(NamespaceStatsAggregator.java:63) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$520/1098830264.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$generate$2(NamespaceStatsAggregator.java:62) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$316/212211274.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.generate(NamespaceStatsAggregator.java:59) at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsGenerator.generate(PrometheusMetricsGenerator.java:73) at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet.lambda$doGet$0(PrometheusMetricsServlet.java:70) at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet$$Lambda$315/1221766138.run(Unknown Source) at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) "ForkJoinPool.commonPool-worker-104" #953 daemon prio=5 os_prio=0 tid=0x7f4dc8030800 nid=0x3b87 waiting on condition [0x7f48f6ce1000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x7f913d08b5c8> (a org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section) at java.util.concurrent.locks.StampedLock.acquireRead(StampedLock.java:1215) at java.util.concurrent.locks.StampedLock.readLock(StampedLock.java:428) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:377) at org.apache.pulsar.co
[GitHub] [pulsar] jiazhai commented on issue #7754: Remove redundant ci job
jiazhai commented on issue #7754: URL: https://github.com/apache/pulsar/issues/7754#issuecomment-675546222 @zymap have you already opened a ticket in ASF Infra for this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] RobertIndie opened a new pull request #7840: [Client]Add autoPartitionsUpdateInterval for producer and consumer
RobertIndie opened a new pull request #7840: URL: https://github.com/apache/pulsar/pull/7840 ### Motivation Add auto partitions update interval setting for producer and consumer. ### Modifications * add autoUpdatePartitionsInterval to partitioned producer and consumer ### Verifying this change - [ ] Make sure that the change passes the CI checks. This change is a trivial rework / code cleanup without any test coverage. ### Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation - Does this pull request introduce a new feature? (yes) - If yes, how is the feature documented? (JavaDocs) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] moshonb6 commented on issue #7829: Upgrading the presto to the last version
moshonb6 commented on issue #7829: URL: https://github.com/apache/pulsar/issues/7829#issuecomment-675490874 We are using http header "X-Presto-Max-Size" but seems that the header is being ignored. We always get chunks of 2MB. may it be also related to the old presto db version? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] zhanghaou opened a new pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
zhanghaou opened a new pull request #7839: URL: https://github.com/apache/pulsar/pull/7839 Link [https://github.com/apache/pulsar/issues/7758](https://github.com/apache/pulsar/issues/7758) and master issue [https://github.com/apache/pulsar/issues/2688](https://github.com/apache/pulsar/issues/2688) ### Motivation Support set/get/remove maxProducers on a topic level. ### Verifying this change new unit test added. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-go] ernado commented on issue #349: Failed to lookup topic (request timeout) after pulsar broker restart
ernado commented on issue #349: URL: https://github.com/apache/pulsar-client-go/issues/349#issuecomment-675474730 Should I try 2.6.1 or 2.5? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #7756: Add release 2.6.1 blog for website
Jennifer88huang commented on a change in pull request #7756: URL: https://github.com/apache/pulsar/pull/7756#discussion_r472178807 ## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ## @@ -0,0 +1,295 @@ +--- +author: XiaoLong Ran +authorURL: https://twitter.com/wolf4j1 +title: Apache Pulsar 2.6.1 +--- +We are very glad to see the Apache Pulsar community has successfully released the wonderful 2.6.1 version after accumulated hard work. It is a great milestone for this fast-growing project and the whole Pulsar community. This is the result of a huge effort from the community, with over 90 commits and a long list of improvements and bug fixes. + +Here is a selection of some of the most interesting and major features added to Pulsar 2.6.1. + + + +## Broker + + Limit the batch size to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` + +The Batch size is not limited to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` from the `BatchReceive` policy. + +1. Batch size is not limited to the minimum of the maxNumberOfMessages and maxSizeOfMessages from the BatchReceive policy. +2. When the batch size is greater than the receiveQ of the consumer (I used a batch size of 3000 and a receiveQ of 500), I notice the following issues: + + In a multi-topic (pattern) consumer, the client stops receiving any messages. I think it gets paused and never resumed when setting a timeout in the batch policy. Only one batch is fetched and the client is never resumed. + +For more information about implementation details, see [PR-6865](https://github.com/apache/pulsar/pull/6865). + + Fix hash range conflict issue in Key_Shared subscription with sticky hash range +In `Key_Shared` subscription where the`stickyHashRange` is used, consumers are not allowed to use interleaving hashes. Review comment: add a space between the two words. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] toneill818 opened a new issue #7838: Add The Ability To Set Log Level From Environment Variables In Docker
toneill818 opened a new issue #7838: URL: https://github.com/apache/pulsar/issues/7838 Currently there are not scripts or set to configure the log YAML file from environment variables. There are for other non YAML configuration files. **Describe the solution you'd like** Add a script to the docker container that can read PULSAR_PREFIX_LOG_LEVEL and set it in the yaml file. **Describe alternatives you've considered** Being able to set the log level within the configuration files such as the broker.conf. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] wolfstudy commented on a change in pull request #7756: Add release 2.6.1 blog for website
wolfstudy commented on a change in pull request #7756: URL: https://github.com/apache/pulsar/pull/7756#discussion_r472066961 ## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ## @@ -0,0 +1,295 @@ +--- +author: XiaoLong Ran +authorURL: https://twitter.com/wolf4j1 +title: Apache Pulsar 2.6.1 +--- +We are very glad to see the Apache Pulsar community has successfully released the wonderful 2.6.1 version after accumulated hard work. It is a great milestone for this fast-growing project and the whole Pulsar community. This is the result of a huge effort from the community, with over 90 commits and a long list of improvements and bug fixes. + +Here is a selection of some of the most interesting and major features added to Pulsar 2.6.1. + + + +## Broker + + Limit the batch size to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` + +The Batch size is not limited to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` from the `BatchReceive` policy. + +1. Batch size is not limited to the minimum of the maxNumberOfMessages and maxSizeOfMessages from the BatchReceive policy. +2. When the batch size is greater than the receiveQ of the consumer (I used a batch size of 3000 and a receiveQ of 500), I notice the following issues: + + In a multi-topic (pattern) consumer, the client stops receiving any messages. I think it gets paused and never resumed when setting a timeout in the batch policy. Only one batch is fetched and the client is never resumed. + +For more information about implementation details, see [PR-6865](https://github.com/apache/pulsar/pull/6865). + + Fix hash range conflict issue in Key_Shared subscription with sticky hash range +In `Key_Shared` subscription where the`stickyHashRange` is used, consumers are not allowed to use interleaving hashes. Review comment: What is difference? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #7756: Add release 2.6.1 blog for website
Jennifer88huang commented on a change in pull request #7756: URL: https://github.com/apache/pulsar/pull/7756#discussion_r472056698 ## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ## @@ -0,0 +1,295 @@ +--- +author: XiaoLong Ran +authorURL: https://twitter.com/wolf4j1 +title: Apache Pulsar 2.6.1 +--- +We are very glad to see the Apache Pulsar community has successfully released the wonderful 2.6.1 version after accumulated hard work. It is a great milestone for this fast-growing project and the whole Pulsar community. This is the result of a huge effort from the community, with over 90 commits and a long list of improvements and bug fixes. + +Here is a selection of some of the most interesting and major features added to Pulsar 2.6.1. + + + +## Broker + + Limit the batch size to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` + +The Batch size is not limited to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` from the `BatchReceive` policy. + +1. Batch size is not limited to the minimum of the maxNumberOfMessages and maxSizeOfMessages from the BatchReceive policy. +2. When the batch size is greater than the receiveQ of the consumer (I used a batch size of 3000 and a receiveQ of 500), I notice the following issues: + + In a multi-topic (pattern) consumer, the client stops receiving any messages. I think it gets paused and never resumed when setting a timeout in the batch policy. Only one batch is fetched and the client is never resumed. + +For more information about implementation details, see [PR-6865](https://github.com/apache/pulsar/pull/6865). + + Fix hash range conflict issue in Key_Shared subscription with sticky hash range +In `Key_Shared` subscription where the`stickyHashRange` is used, consumers are not allowed to use interleaving hashes. + +The pull request will fix hash range conflict issue in `Key_Shared` with sticky hash range. Review comment: will fix --> fixes ## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ## @@ -0,0 +1,295 @@ +--- +author: XiaoLong Ran +authorURL: https://twitter.com/wolf4j1 +title: Apache Pulsar 2.6.1 +--- +We are very glad to see the Apache Pulsar community has successfully released the wonderful 2.6.1 version after accumulated hard work. It is a great milestone for this fast-growing project and the whole Pulsar community. This is the result of a huge effort from the community, with over 90 commits and a long list of improvements and bug fixes. + +Here is a selection of some of the most interesting and major features added to Pulsar 2.6.1. + + + +## Broker + + Limit the batch size to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` + +The Batch size is not limited to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` from the `BatchReceive` policy. + +1. Batch size is not limited to the minimum of the maxNumberOfMessages and maxSizeOfMessages from the BatchReceive policy. +2. When the batch size is greater than the receiveQ of the consumer (I used a batch size of 3000 and a receiveQ of 500), I notice the following issues: Review comment: Take care of the following: 1. Item 1 is identical to the sentence above. 2. It's release blog, could we avoid using "I used ..., I notice)? ## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ## @@ -0,0 +1,295 @@ +--- +author: XiaoLong Ran +authorURL: https://twitter.com/wolf4j1 +title: Apache Pulsar 2.6.1 +--- +We are very glad to see the Apache Pulsar community has successfully released the wonderful 2.6.1 version after accumulated hard work. It is a great milestone for this fast-growing project and the whole Pulsar community. This is the result of a huge effort from the community, with over 90 commits and a long list of improvements and bug fixes. + +Here is a selection of some of the most interesting and major features added to Pulsar 2.6.1. + + + +## Broker + + Limit the batch size to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` + +The Batch size is not limited to the minimum of the `maxNumberOfMessages` and `maxSizeOfMessages` from the `BatchReceive` policy. + +1. Batch size is not limited to the minimum of the maxNumberOfMessages and maxSizeOfMessages from the BatchReceive policy. +2. When the batch size is greater than the receiveQ of the consumer (I used a batch size of 3000 and a receiveQ of 500), I notice the following issues: + + In a multi-topic (pattern) consumer, the client stops receiving any messages. I think it gets paused and never resumed when setting a timeout in the batch policy. Only one batch is fetched and the client is never resumed. Review comment: what does "it" mean in "I think it gets paused..."? ## File path: site2/website/blog/2020-08-17-Apache-Pulsar-2-6-1.md ## @@ -0,0 +1,295 @@ +--- +author:
[GitHub] [pulsar] ErmirZaimi opened a new issue #7837: DB2 Connector for Pulsar
ErmirZaimi opened a new issue #7837: URL: https://github.com/apache/pulsar/issues/7837 **Is your enhancement request related to a problem? Please describe.** We want to receive events in Pulsar for any changes done to the DB2 (in AS400 IBM machine) database, **Describe the solution you'd like** A new connector for pulsar **Describe alternatives you've considered** Using the incubating version of debezium API for kafka This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #7828: Fix pending batchIndexAcks bitSet batchSize in PersistentAcknowledgmentsGroupingTracker
codelipenghui merged pull request #7828: URL: https://github.com/apache/pulsar/pull/7828 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated: Fix pending batchIndexAcks bitSet batchSize in PersistentAcknowledgmentsGroupingTracker (#7828)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 23d795c Fix pending batchIndexAcks bitSet batchSize in
PersistentAcknowledgmentsGroupingTracker (#7828)
23d795c is described below
commit 23d795c08ed54606e71ac8b1de1bc32420e2fbf1
Author: ran
AuthorDate: Tue Aug 18 17:11:19 2020 +0800
Fix pending batchIndexAcks bitSet batchSize in
PersistentAcknowledgmentsGroupingTracker (#7828)
### Motivation
The pending batchIndexAcks bitSet batchSize is not correct.
### Modifications
Fix the bitSet batchSize.
---
.../apache/pulsar/client/impl/BatchMessageIndexAckTest.java | 12 ++--
.../impl/PersistentAcknowledgmentsGroupingTracker.java | 3 +--
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BatchMessageIndexAckTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BatchMessageIndexAckTest.java
index 8f76561..ae10141 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BatchMessageIndexAckTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BatchMessageIndexAckTest.java
@@ -28,6 +28,7 @@ import org.apache.pulsar.client.api.ProducerConsumerBase;
import org.apache.pulsar.client.api.PulsarClientException;
import org.apache.pulsar.client.api.Schema;
import org.apache.pulsar.client.api.SubscriptionType;
+import org.apache.pulsar.common.policies.data.PersistentTopicInternalStats;
import org.apache.pulsar.common.util.FutureUtil;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
@@ -58,13 +59,14 @@ public class BatchMessageIndexAckTest extends
ProducerConsumerBase {
}
@Test
-public void testBatchMessageIndexAckForSharedSubscription() throws
PulsarClientException, ExecutionException, InterruptedException {
+public void testBatchMessageIndexAckForSharedSubscription() throws
Exception {
final String topic = "testBatchMessageIndexAckForSharedSubscription";
+final String subscriptionName = "sub";
@Cleanup
Consumer consumer = pulsarClient.newConsumer(Schema.INT32)
.topic(topic)
-.subscriptionName("sub")
+.subscriptionName(subscriptionName)
.receiverQueueSize(100)
.subscriptionType(SubscriptionType.Shared)
.enableBatchIndexAcknowledgment(true)
@@ -115,6 +117,12 @@ public class BatchMessageIndexAckTest extends
ProducerConsumerBase {
Message moreMessage = consumer.receive(2, TimeUnit.SECONDS);
Assert.assertNull(moreMessage);
+// check the mark delete position was changed
+BatchMessageIdImpl ackedMessageId = (BatchMessageIdImpl)
received.get(0);
+PersistentTopicInternalStats stats =
admin.topics().getInternalStats(topic);
+String markDeletePosition =
stats.cursors.get(subscriptionName).markDeletePosition;
+Assert.assertEquals(ackedMessageId.ledgerId + ":" +
ackedMessageId.entryId, markDeletePosition);
+
futures.clear();
for (int i = 0; i < 50; i++) {
futures.add(producer.sendAsync(i));
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java
index 17e8c22..6a4deef 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java
@@ -169,8 +169,7 @@ public class PersistentAcknowledgmentsGroupingTracker
implements Acknowledgments
ConcurrentBitSetRecyclable bitSet =
pendingIndividualBatchIndexAcks.computeIfAbsent(
new MessageIdImpl(msgId.getLedgerId(), msgId.getEntryId(),
msgId.getPartitionIndex()), (v) -> {
ConcurrentBitSetRecyclable value =
ConcurrentBitSetRecyclable.create();
-value.set(0, batchSize + 1);
-value.clear(batchIndex);
+value.set(0, batchSize);
return value;
});
bitSet.clear(batchIndex);
[GitHub] [pulsar] massakam closed pull request #7836: [broker] Make PersistentSubscription#getNumberOfEntriesDelayed() non-synchronized
massakam closed pull request #7836: URL: https://github.com/apache/pulsar/pull/7836 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] massakam opened a new pull request #7836: [broker] Make PersistentSubscription#getNumberOfEntriesDelayed() non-synchronized
massakam opened a new pull request #7836: URL: https://github.com/apache/pulsar/pull/7836 ### Motivation A deadlock occurred on our Pulsar 2.4.2 broker server. The cause is the following two threads: ``` "prometheus-stats-36-1" #410 prio=5 os_prio=0 tid=0x7f4b70019800 nid=0x30ca waiting for monitor entry [0x7f4bbe3b7000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.pulsar.broker.service.persistent.PersistentSubscription.getNumberOfEntriesDelayed(PersistentSubscription.java:1013) - waiting to lock <0x7f913d098dd0> (a org.apache.pulsar.broker.service.persistent.PersistentSubscription) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$getTopicStats$8(NamespaceStatsAggregator.java:129) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$523/2109257042.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.getTopicStats(NamespaceStatsAggregator.java:122) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$null$0(NamespaceStatsAggregator.java:64) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$521/1017174654.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$null$1(NamespaceStatsAggregator.java:63) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$520/1098830264.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.lambda$generate$2(NamespaceStatsAggregator.java:62) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator$$Lambda$316/212211274.accept(Unknown Source) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:385) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159) at org.apache.pulsar.broker.stats.prometheus.NamespaceStatsAggregator.generate(NamespaceStatsAggregator.java:59) at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsGenerator.generate(PrometheusMetricsGenerator.java:73) at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet.lambda$doGet$0(PrometheusMetricsServlet.java:70) at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet$$Lambda$315/1221766138.run(Unknown Source) at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) "ForkJoinPool.commonPool-worker-104" #953 daemon prio=5 os_prio=0 tid=0x7f4dc8030800 nid=0x3b87 waiting on condition [0x7f48f6ce1000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x7f913d08b5c8> (a org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section) at java.util.concurrent.locks.StampedLock.acquireRead(StampedLock.java:1215) at java.util.concurrent.locks.StampedLock.readLock(StampedLock.java:428) at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:377) at org.apache.pulsar.co
[GitHub] [pulsar] wolfstudy opened a new pull request #7835: Add 2.6.2 docs
wolfstudy opened a new pull request #7835: URL: https://github.com/apache/pulsar/pull/7835 Signed-off-by: xiaolong.ran ### Motivation Currently, the 2.6.2 docs are a complete copy based on 2.6.1. In the next pull request, if there is a fix for the 2.6.2 document, we can modify it based on the current 2.6.2, so A release manager can directly reuse the contents of the 2.6.2 document when releasing 2.6.2 ### Modifications - Add docs of 2.6.2 - Add sidebars of 2.6.2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated: Update 2.6.1 docs (#7832)
This is an automated email from the ASF dual-hosted git repository. rxl pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 8de22bd Update 2.6.1 docs (#7832) 8de22bd is described below commit 8de22bdff0227fd6008afe293e50c0deb4f0331f Author: 冉小龙 AuthorDate: Tue Aug 18 16:39:03 2020 +0800 Update 2.6.1 docs (#7832) Signed-off-by: xiaolong.ran ### Motivation Update 2.6.1 docs ### Modifications - update 2.6.1 docs - add 2.6.1 sidebars.json - update releases.json - update versions.json --- site2/website/releases.json| 1 + .../version-2.6.1/admin-api-namespaces.md | 11 +- .../version-2.6.1/admin-api-schemas.md | 7 - .../version-2.6.1/administration-zk-bk.md | 92 ++-- .../version-2.6.1/client-libraries-go.md | 19 + .../version-2.6.1/client-libraries-java.md | 49 +- .../version-2.6.1/client-libraries-python.md | 27 -- .../version-2.6.1/concepts-clients.md | 2 +- .../version-2.6.1/concepts-messaging.md| 158 +++--- .../version-2.6.1/concepts-proxy-sni-routing.md| 121 + .../version-2.6.1/concepts-schema-registry.md | 6 + .../version-2.6.1/cookbooks-deduplication.md | 10 +- .../version-2.6.1/cookbooks-encryption.md | 2 +- .../version-2.6.1/cookbooks-tiered-storage.md | 2 - .../version-2.6.1/deploy-bare-metal.md | 2 +- .../versioned_docs/version-2.6.1/deploy-dcos.md| 4 +- .../versioned_docs/version-2.6.1/functions-cli.md | 2 +- .../version-2.6.1/functions-debug.md | 2 +- .../version-2.6.1/functions-deploy.md | 2 +- .../version-2.6.1/functions-develop.md | 2 +- .../version-2.6.1/functions-metrics.md | 7 - .../version-2.6.1/functions-runtime.md | 5 +- .../version-2.6.1/functions-worker.md | 46 +- .../getting-started-concepts-and-architecture.md | 16 - .../version-2.6.1/getting-started-helm.md | 33 +- .../versioned_docs/version-2.6.1/helm-install.md | 2 +- .../versioned_docs/version-2.6.1/io-cdc-canal.md | 6 + .../website/versioned_docs/version-2.6.1/io-cli.md | 12 +- .../versioned_docs/version-2.6.1/io-hdfs2-sink.md | 6 +- .../versioned_docs/version-2.6.1/io-quickstart.md | 4 - .../versioned_docs/version-2.6.1/io-twitter.md | 7 - .../version-2.6.1/reference-cli-tools.md | 8 +- .../version-2.6.1/reference-configuration.md | 5 +- .../version-2.6.1/reference-pulsar-admin.md| 2 +- .../version-2.6.1/schema-get-started.md| 4 - .../version-2.6.1/security-bouncy-castle.md| 7 +- .../version-2.6.1/security-oauth2.md | 135 ++ .../version-2.6.1/security-overview.md | 4 + .../version-2.6.1/sql-deployment-configurations.md | 6 +- .../version-2.6.1/window-functions-context.md | 529 + .../versioned_sidebars/version-2.6.1-sidebars.json | 155 ++ site2/website/versions.json| 1 + 42 files changed, 1237 insertions(+), 284 deletions(-) diff --git a/site2/website/releases.json b/site2/website/releases.json index b82be5f..bad6d43 100644 --- a/site2/website/releases.json +++ b/site2/website/releases.json @@ -1,4 +1,5 @@ [ + "2.6.1", "2.6.0", "2.5.2", "2.5.1", diff --git a/site2/website/versioned_docs/version-2.6.1/admin-api-namespaces.md b/site2/website/versioned_docs/version-2.6.1/admin-api-namespaces.md index c1cdd44..d437cf5 100644 --- a/site2/website/versioned_docs/version-2.6.1/admin-api-namespaces.md +++ b/site2/website/versioned_docs/version-2.6.1/admin-api-namespaces.md @@ -555,11 +555,18 @@ admin.namespaces().getRetention(namespace) set dispatch throttling -It sets message dispatch rate for all the topics under a given namespace. +It sets message dispatch rate for all the topics under a given namespace. Dispatch rate can be restricted by number of message per X seconds (`msg-dispatch-rate`) or by number of message-bytes per X second (`byte-dispatch-rate`). dispatch rate is in second and it can be configured with `dispatch-rate-period`. Default value of `msg-dispatch-rate` and `byte-dispatch-rate` is -1 which disables the throttling. + Note +> - If neither `clusterDispatchRate` nor `topicDispatchRate` is configured, dispatch throttling is disabled. +> > +> - If `topicDispatchRate` is not configured, `clusterDispatchRate` takes effect. +> > +> - If `topicDispatchRate` is configured, `topicDispatchRate` takes effect. + ## CLI ``` @@ -670,7 +677,7 @@ $ pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1 admin.namespaces().getSubscriptionDispatchRate(namespace) ``` - set dispatch throttling for subscription
[GitHub] [pulsar] wolfstudy merged pull request #7832: Update 2.6.1 docs
wolfstudy merged pull request #7832: URL: https://github.com/apache/pulsar/pull/7832 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Huanli-Meng commented on issue #7463: [docs] Instructions for using Go client are unclear and confusing
Huanli-Meng commented on issue #7463: URL: https://github.com/apache/pulsar/issues/7463#issuecomment-675331269 @devinbost, when you use the CGo to build the Pulsar library, you can use the `go get -u github.com/apache/pulsar/pulsar-client-go/pulsar` command to get the pulsar library. when you use the native Go to build the Pulsar library, you can use the `go get -u "github.com/apache/pulsar-client-go/pulsar` command to get the pulsar library. The native Go is introduced in Pulsar 2.5.1 and the CGo is going to be deprecated. Here is the link about native go client: https://pulsar.apache.org/docs/en/client-libraries-go/ here is the link about CGo client: https://pulsar.apache.org/docs/en/2.5.1/client-libraries-cgo This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Huanli-Meng commented on pull request #7834: [Doc]--Update BookKeeper config
Huanli-Meng commented on pull request #7834: URL: https://github.com/apache/pulsar/pull/7834#issuecomment-675313886 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Jennifer88huang commented on pull request #7832: Update 2.6.1 docs
Jennifer88huang commented on pull request #7832: URL: https://github.com/apache/pulsar/pull/7832#issuecomment-675303922 @Huanli-Meng could you please also help check whether doc PR labeled with release/2.6.1 are picked in this PR. Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
