[GitHub] [pulsar] tuteng merged pull request #6081: [Hotfix]Fixed website build error
tuteng merged pull request #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Fixed crowdin parse error (#6081)
This is an automated email from the ASF dual-hosted git repository. guangning pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 21660e9 Fixed crowdin parse error (#6081) 21660e9 is described below commit 21660e9402f82afdbfa47a24dbd7626b9ce9c68d Author: Guangning AuthorDate: Sat Jan 18 15:07:59 2020 +0800 Fixed crowdin parse error (#6081) ### Motivation https://github.com/apache/pulsar/issues/5816 After the release of the new version, crowdin parsing label error blocked release note. ### Modifications * Temporary replacement of Chinese version files ### Verifying this change yarn build pass --- site2/tools/build-site.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/site2/tools/build-site.sh b/site2/tools/build-site.sh index 6a9f469..37e4aeb 100755 --- a/site2/tools/build-site.sh +++ b/site2/tools/build-site.sh @@ -33,6 +33,8 @@ yarn run crowdin-download # TODO: remove this after figuring out why crowdin removed code tab when generating translated files # https://github.com/apache/pulsar/issues/5816 cp versioned_docs/version-2.4.2/functions-develop.md translated_docs/zh-CN/version-2.4.2/functions-develop.md +cp versioned_docs/version-2.5.0/functions-develop.md translated_docs/zh-CN/version-2.5.0/functions-develop.md +cp versioned_docs/version-2.5.0/io-overview.md translated_docs/zh-CN/version-2.5.0/io-overview.md cp versioned_docs/version-2.4.2/functions-develop.md translated_docs/ja/version-2.4.2/functions-develop.md cp versioned_docs/version-2.4.2/functions-develop.md translated_docs/fr/version-2.4.2/functions-develop.md
[GitHub] [pulsar] tuteng commented on issue #6081: [Hotfix]Fixed website build error
tuteng commented on issue #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081#issuecomment-575865790 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #6081: [Hotfix]Fixed website build error
tuteng commented on issue #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081#issuecomment-575865735 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #6081: [Hotfix]Fixed website build error
tuteng commented on issue #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081#issuecomment-575857283 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up
codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up URL: https://github.com/apache/pulsar/pull/6077#issuecomment-575848046 @sijie I have addressed your comments, please help take a look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5716: [Issues 5709]remove the namespace checking
tuteng commented on issue #5716: [Issues 5709]remove the namespace checking URL: https://github.com/apache/pulsar/pull/5716#issuecomment-575835911 Move to 2.5.1, If it is necessary to merge into 2.4.3, please reply here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #6081: [Hotfix]Fixed website build error
tuteng commented on issue #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081#issuecomment-575834027 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] hanbo1990 opened a new pull request #6082: [Issue 5326] make bookie ranges stored in path/to/bkdata/ranges
hanbo1990 opened a new pull request #6082: [Issue 5326] make bookie ranges stored in path/to/bkdata/ranges URL: https://github.com/apache/pulsar/pull/6082 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #6074: PIP-55: Refresh Authentication Credentials
merlimat commented on a change in pull request #6074: PIP-55: Refresh Authentication Credentials URL: https://github.com/apache/pulsar/pull/6074#discussion_r368105348 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java ## @@ -127,6 +127,10 @@ AuthenticationDataSource authenticationData; AuthenticationProvider authenticationProvider; AuthenticationState authState; +// In case of proxy, if the authentication credentials are forwardable, +// it will hold the credentials of the original client +AuthenticationState originalAuthState; +private boolean pendingAuthChallengeResponse = false; Review comment: This variable will be always accessed only the connection IO thread so there won't be any race condition. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #6078: Make log flush immediately
merlimat commented on issue #6078: Make log flush immediately URL: https://github.com/apache/pulsar/pull/6078#issuecomment-575724621 > Is there any performance penalty for this? Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] bry00 commented on issue #5899: Cannot import Python pulsar-client==2.4.2 on Mac Os Catalina 10.15.2
bry00 commented on issue #5899: Cannot import Python pulsar-client==2.4.2 on Mac Os Catalina 10.15.2 URL: https://github.com/apache/pulsar/issues/5899#issuecomment-575721147 I've succeed with installing and using `pulsar-client==2.5.0` using regular (brew not pyenv) Python 3.7.6 installation. It seems, that this segmentation fault is caused by some incompatibility of` pulsar-client` with pyenv, but it would be another issue. I close this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] bry00 closed issue #5899: Cannot import Python pulsar-client==2.4.2 on Mac Os Catalina 10.15.2
bry00 closed issue #5899: Cannot import Python pulsar-client==2.4.2 on Mac Os Catalina 10.15.2 URL: https://github.com/apache/pulsar/issues/5899 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5816: website failed to build due to "Invalid code tab markdown"
sijie commented on issue #5816: website failed to build due to "Invalid code tab markdown" URL: https://github.com/apache/pulsar/issues/5816#issuecomment-575705375 @Jennifer88huang just one though, I think this issue might be related to the duplicated string setting in crowdin. We might try to disable that setting. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie edited a comment on issue #5816: website failed to build due to "Invalid code tab markdown"
sijie edited a comment on issue #5816: website failed to build due to "Invalid code tab markdown" URL: https://github.com/apache/pulsar/issues/5816#issuecomment-575705375 @Jennifer88huang just one thought, I think this issue might be related to the duplicated string setting in crowdin. We might try to disable that setting. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5278: Fixed reader cursor seek with disconnection
sijie commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-575699228 --- @codelipenghui @jiazhai @merlimat without sending a close consumer command to the client, the client will not clean the pre-fetched items in its queue. Hence resetting a cursor can result in dispatching duplicated items and unexpected behavior. For example, in the following sequence, 1) a consumer is receiving messages. it already pre-prefetches 5 messages (1, 2, 3, 4, 5). 2) it only calls `receive` once. It receives a message `1`. there are 4 messages in the queue. 3) it calls `reset` to reset to message `0`. it is expecting to receive `1` for next `receive` call. but since 2-5 are still in the client cache. when it calls `receive`, it will receive `2`. IMO this violates the guarantee that pulsar provides. Looking more into this issue and checking the current implementation, it seems the current implementation seems to be ok and shouldn't have any side-effects on connections. So I would like to step back and understand a bit more about the *problematic* behavior driving the motivation of this change. @bsideup you mentioned the followings in your previous comment: ``` Since Liiklus is not using Pulsar's offset storage, it performs "seek" on every partition, and it causes the other connections to disconnect, and, when they connect it seeks again 😂 ``` Can you clarify what do you observe "it causes the other connections to disconnect"? As far as I know, "close consumer" will not close a multiplexed connection. I am trying to understand what you have observed before. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-575692466 @tuteng In my development environment I have two HA-Proxy servers that load balance between 2 - Pulsar proxy servers. I then have 3 brokers, these brokers then point to 4 bookkeepers. I also have 3 Zookeeper nodes. The difference between my development and production environments are that the development environment's configuration store is also the Zookeeper quorum. In my Production environment I have a separate quorum zookeeper cluster (3) and configuration store (3), and also have 8 Bookkeepers in production. Just to note, I am seeing this issue in our production instance as well. Here is the broker.conf I'm using on all three of the brokers and is pretty much the same in the production environment just different host names, cert names, and bucket names: https://gist.github.com/digi691/2a27c8a6055145e98450fc7efce8c0c4. FYI - I had to scrub the file for hostnames, Identifications in Cert Names, etc... As you'll notice: TLS is turned on though authentication is currently turned off. I timeout when going through the HA-Proxy and Pulsar Proxies as I would expect when hitting the admin api to list non-persistent volumes. When I point directly to the brokers and hit that that admin api, it just sits and waits forever as I explained above and never produces a response. I also cannot use the pulsar-admin tool sub command `topics list` as I believe it's trying to list non-persistent topics as well as persistent. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6047: [docs] Update configuration information for 2.4.x releases
sijie commented on issue #6047: [docs] Update configuration information for 2.4.x releases URL: https://github.com/apache/pulsar/pull/6047#issuecomment-575674858 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #6081: [Hotfix]Fixed website build error
tuteng commented on issue #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081#issuecomment-575671963 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #6077: Support delete inactive topic when subscriptions caught up
sijie commented on a change in pull request #6077: Support delete inactive
topic when subscriptions caught up
URL: https://github.com/apache/pulsar/pull/6077#discussion_r367988673
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
##
@@ -1586,19 +1594,34 @@ public long getBacklogSize() {
return ledger.getEstimatedBacklogSize();
}
-public boolean isActive() {
+public boolean isActive(InactiveTopicDeleteMode deleteMode) {
+switch (deleteMode) {
+case delete_when_no_subscriptions:
+if (!subscriptions.isEmpty()) {
+return true;
+}
+case delete_when_subscriptions_caught_up:
+if (hasLocalConsumers() || hasBacklogs()) {
Review comment:
I think in `delete_when_subscriptions_caught_up` mode, we just need to check
if the backlog is empty or not. we don't need to check if there are consumers
connected.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #6077: Support delete inactive topic when subscriptions caught up
sijie commented on a change in pull request #6077: Support delete inactive
topic when subscriptions caught up
URL: https://github.com/apache/pulsar/pull/6077#discussion_r367987983
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
##
@@ -1586,19 +1594,34 @@ public long getBacklogSize() {
return ledger.getEstimatedBacklogSize();
}
-public boolean isActive() {
+public boolean isActive(InactiveTopicDeleteMode deleteMode) {
+switch (deleteMode) {
+case delete_when_no_subscriptions:
+if (!subscriptions.isEmpty()) {
+return true;
+}
Review comment:
break here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #6077: Support delete inactive topic when subscriptions caught up
sijie commented on a change in pull request #6077: Support delete inactive
topic when subscriptions caught up
URL: https://github.com/apache/pulsar/pull/6077#discussion_r367985450
##
File path:
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##
@@ -254,6 +255,23 @@
doc = "How often to check for inactive topics"
)
private int brokerDeleteInactiveTopicsFrequencySeconds = 60;
+
+@FieldContext(
+category = CATEGORY_POLICIES,
+doc = "Set the inactive topic delete mode. Default is
delete_when_no_subscriptions\n"
++ "'delete_when_no_subscriptions' mode only delete the topic which has
no subscriptions and no active producers\n"
++ "'delete_when_subscriptions_caught_up' mode only delete the topic
that all subscriptions has no backlogs(caught up)"
++ "and no active producers/consumers"
+)
+private InactiveTopicDeleteMode brokerDeleteInactiveTopicsMode =
InactiveTopicDeleteMode.delete_when_no_subscriptions;
+
+@FieldContext(
+category = CATEGORY_POLICIES,
+doc = "Max duration of topic inactivity in seconds, default is 60s\n"
++ "Topics that are inactive for longer than this value will be deleted"
+)
+private int brokerDeleteInactiveTopicsMaxInactiveDurationSeconds = 60;
Review comment:
I like the idea of introducing
`brokerDeleteInactiveTopicsMaxInactiveDurationSeconds`. However, I think we
need to keep backward compatibility here since it was using
`brokerDeleteInactiveTopicsFrequencySeconds`. So what you can do here are the
followings:
1. use boxed type not primitive type. so it is `null` when this setting is
not specified.
```
private Integer brokerDeleteInactiveTopicsMaxInactiveDurationSeconds = null
```
2. implementing the getter for this field.
```
int getBrokerDeleteInactiveTopicsMaxInactiveDurationSeconds() {
if (brokerDeleteInactiveTopicsMaxInactiveDurationSeconds == null) {
return brokerDeleteInactiveTopicsFrequencySeconds;
} else {
return brokerDeleteInactiveTopicsMaxInactiveDurationSeconds;
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #6081: [Hotfix]Fixed website build error
tuteng commented on issue #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081#issuecomment-575669420 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng opened a new pull request #6081: [Hotfix]Fixed website build error
tuteng opened a new pull request #6081: [Hotfix]Fixed website build error URL: https://github.com/apache/pulsar/pull/6081 ### Motivation https://github.com/apache/pulsar/issues/5816 After the release of the new version, crowdin parsing label error blocked release note. ### Modifications * Temporary replacement of Chinese version files ### Verifying this change yarn build pass This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for batch message local index
sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for
batch message local index
URL: https://github.com/apache/pulsar/pull/6052#discussion_r367980477
##
File path: pulsar-common/src/main/proto/PulsarApi.proto
##
@@ -477,6 +484,8 @@ message CommandAck {
optional uint64 txnid_least_bits = 6 [default = 0];
optional uint64 txnid_most_bits = 7 [default = 0];
+
+repeated BatchMessageIndexesAckData batch_message_ack_indexes = 8;
Review comment:
If we added an `ackSet` bitset to MessageIdData, we don't need to introduce
additional structure, no?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for batch message local index
sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for
batch message local index
URL: https://github.com/apache/pulsar/pull/6052#discussion_r367978978
##
File path: managed-ledger/src/main/proto/MLDataFormats.proto
##
@@ -66,6 +66,9 @@ message PositionInfo {
// Additional custom properties associated with
// the current cursor position
repeated LongProperty properties = 4;
+
+// Store which index in the batch message has been deleted
+repeated BatchDeletedIndexInfo batchDeletedIndexes = 5;
Review comment:
Instead of introducing a new structure for storing index information, why
not just store a bitset in NestedPositionInfo?
```
message NestedPositionInfo {
required int64 ledgerId = 1;
required int64 entryId = 2;
// using one integer for storing the acknowledgement set for the indexes
within a entry
optional uint32 ackset = 0;
}
```
The `ackset` in `NestedPositionInfo` is used for indicating whether an
*entry* is fully acknowledged or *partially* acknowledged. Hence you don't need
to introduce more data structure. You can still use `individualDeletedMessages`
for representing a delete range. In a message range, all *entries* between
`lowerEndpoint` and `higherEndpoint` are *fully* acknowledged. The ackSets in
`lowerEndpoint` and `upperEndpoint` are used for indicating whether these two
entries are *fully* or *partially* acked.
Using this format, you don't need to introduce redundant structure and it is
a very efficient storage format. Also, more importantly, it provides a
consistent view for people to realize the acknowledgment behavior without
looking into multiple different places.
In the implementation, it should be straightforward as well. As you just
need to introduce ackSet to current acknowledgment logic and make sure the
range merging logic handle ackSet correctly. And when exposing the
acknowledgment information to topic stats, you can just simply add the `ackSet`
in the individual delete set.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for batch message local index
sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for
batch message local index
URL: https://github.com/apache/pulsar/pull/6052#discussion_r367979779
##
File path: pulsar-common/src/main/proto/PulsarApi.proto
##
@@ -447,6 +453,7 @@ message CommandMessage {
required uint64 consumer_id = 1;
required MessageIdData message_id = 2;
optional uint32 redelivery_count = 3 [default = 0];
+repeated IntRange acked_indexes = 4;
Review comment:
I think an `integer` should be enough for representing the `ackSet`, no?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for batch message local index
sijie commented on a change in pull request #6052: [PIP-54] Acknowledgement for batch message local index URL: https://github.com/apache/pulsar/pull/6052#discussion_r367965258 ## File path: managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java ## @@ -148,6 +152,10 @@ return position; }; private final LongPairRangeSet individualDeletedMessages; + +// Maintain the indexes deleted status of batch messages that not deleted completely +// (ledgerId, entryId, batchSize) -> deleted indexes +private final ConcurrentHashMap batchDeletedIndexes; Review comment: I think we can initialize this lazily since people might not enable this feature at all. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder
Jennifer88huang commented on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder URL: https://github.com/apache/pulsar/pull/6067#issuecomment-575660854 @codelipenghui Thanks for your feedback, I've update the content on Key_Shared accordingly. @zymap Thanks for your review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang edited a comment on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder
Jennifer88huang edited a comment on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder URL: https://github.com/apache/pulsar/pull/6067#issuecomment-575660854 @codelipenghui Thanks for your feedback, I've updated the content on Key_Shared accordingly. @zymap Thanks for your review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #6007: [Issue 5996][docs] Update links in 2.4.1 and 2.4.2 releases
Jennifer88huang commented on issue #6007: [Issue 5996][docs] Update links in 2.4.1 and 2.4.2 releases URL: https://github.com/apache/pulsar/pull/6007#issuecomment-575655220 ok. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui edited a comment on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder
codelipenghui edited a comment on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder URL: https://github.com/apache/pulsar/pull/6067#issuecomment-575643279 @Jennifer88huang I think we can delete the beta indicator, we have noticed that some users have been using it in production. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder
codelipenghui commented on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder URL: https://github.com/apache/pulsar/pull/6067#issuecomment-575643720 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder
codelipenghui commented on issue #6067: [Issue 6057][docs] Add description and usage on TypedMessageBuilder URL: https://github.com/apache/pulsar/pull/6067#issuecomment-575643279 @Jennifer88huang I think we can delete the beta indicator, we have learned that some users have been using it in production. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui edited a comment on issue #5822: Expose bookkeeper expose explicit lac configuration in broker.conf
codelipenghui edited a comment on issue #5822: Expose bookkeeper expose explicit lac configuration in broker.conf URL: https://github.com/apache/pulsar/pull/5822#issuecomment-575641280 run integration tests >org.apache.pulsar.tests.integration.cli.CLITest.pulsar-test-suite org.apache.pulsar.tests.integration.schema.SchemaTest.pulsar-test-suite org.apache.pulsar.tests.integration.functions.PulsarFunctionsProcessTest.pulsar-test-suite org.apache.pulsar.tests.integration.presto.TestBasicPresto.pulsar-test-suite org.apache.pulsar.tests.integration.functions.PulsarFunctionsThreadTest.pulsar-test-suite org.apache.pulsar.tests.integration.offload.TestS3Offload.tiered-storage-test-suite org.apache.pulsar.tests.integration.offload.TestFileSystemOffload.tiered-storage-test-suite org.apache.pulsar.tests.integration.functions.PulsarStateTest.pulsar-standalone-suite org.apache.pulsar.tests.integration.messaging.PersistentTopicMessagingTest.pulsar-test-suite This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5822: Expose bookkeeper expose explicit lac configuration in broker.conf
codelipenghui commented on issue #5822: Expose bookkeeper expose explicit lac configuration in broker.conf URL: https://github.com/apache/pulsar/pull/5822#issuecomment-575641280 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5837: [release] add release notes for 2.5.0 release
codelipenghui commented on issue #5837: [release] add release notes for 2.5.0 release URL: https://github.com/apache/pulsar/pull/5837#issuecomment-575637830 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar.wiki] branch master updated: Updated Home (markdown)
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 038c011 Updated Home (markdown) 038c011 is described below commit 038c01168109fcfd7f5bdd261bbb572767b03850 Author: lipenghui AuthorDate: Fri Jan 17 21:57:55 2020 +0800 Updated Home (markdown) --- Home.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Home.md b/Home.md index 5cd4168..c59e944 100644 --- a/Home.md +++ b/Home.md @@ -8,7 +8,7 @@ We encourage to document any big change or feature or any addition to public use ### Proposed * [[PIP 55: Refresh Authentication Credentials]] -* [[PIP 54: Acknowledgement for batch message local index]] +* [PIP 54: Acknowledgement for batch message local index](https://github.com/apache/pulsar/wiki/PIP-54:-Acknowledgement-for-local-index-of-batch-message) * [[PIP 53: Contribute DotPulsar to Apache Pulsar]] * [[PIP 52: Message dispatch throttling relative to publish rate]] * [[PIP 51: Tenant policy support]]
[pulsar.wiki] branch master updated: Add PIP-54
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git
The following commit(s) were added to refs/heads/master by this push:
new 2ac7305 Add PIP-54
2ac7305 is described below
commit 2ac73050f5ee2d84b39d49d409c69dcea63b29a8
Author: lipenghui
AuthorDate: Fri Jan 17 21:56:26 2020 +0800
Add PIP-54
---
...owledgement-for-local-index-of-batch-message.md | 76 ++
1 file changed, 76 insertions(+)
diff --git a/PIP-54:-Acknowledgement-for-local-index-of-batch-message.md
b/PIP-54:-Acknowledgement-for-local-index-of-batch-message.md
new file mode 100644
index 000..1b35fdb
--- /dev/null
+++ b/PIP-54:-Acknowledgement-for-local-index-of-batch-message.md
@@ -0,0 +1,76 @@
+* **Status**: Proposal
+* **Author**: Penghui Li
+* **Pull Request**: https://github.com/apache/pulsar/pull/6052
+* **Mailing List discussion**:
+* **Release**:
+
+
+## Motivation
+
+Currently, the managed cursor maintain the acked messages by mark delete
position and individual delete messages. All of the mark delete position and
individual delete messages are faced to batch messages(ledgerId,entryId). For
users, the single message is the publishing and consuming unit. Since the
managed cursor can’t maintain the local index of a batch message, broker will
dispatch acked messages(not batch) to consumers like issue
https://github.com/apache/pulsar/issues/5969.
+
+So this PIP is to support the ability to track the ack status of each batch
index to avoid dispatch acked messages to users.
+
+## Approach
+
+This approach requires the cooperation of the client and server. When the
broker dispatch messages, it will carry the batch index that has been acked.
The client will filter out the batch index that has been acked.
+
+Client need to send the batch index ack information to broker, so that broker
can maintain the batch index ack status.
+
+The managed cursor maintain the batch index ack status in memory by using a
BitSet and the BitSet can be persisted to ledger and metastore, so that can
avoid broker crash. When broker receive the batch index ack request, the acked
batch index will be add the BitSet. When broker dispatch messages to client
will get the batch message index ack status from the managed cursor and send it
to client. When all indexes of the batch message are acked, the cursor will
delete the batch message.
+
+Be careful when calculating consumer permits because client filter out the
acked batch index, so the broker need to increase the available permits equals
to acked batch indexes. Otherwise, the broker will stop dispatch messages to
that consumer because it does not have enough available permits.
+
+For save resources, transfer the acked batch indexes ranges between client and
broker. And in managed cursor also persist batch indexes ranges to the cursor
ledger and metadata store.
+
+
+## Changes
+
+### Wire protocol
+
+```proto
+message BatchMessageIndexesAckData {
+required MessageIdData message_id = 1;
+required int32 batchSize = 2;
+repeated IntRange ack_indexes = 3;
+}
+```
+
+```proto
+
+message CommandMessage {
+repeated IntRange acked_indexes = 4;
+}
+```
+
+```proto
+
+message CommandAck {
+repeated BatchMessageIndexesAckData batch_message_ack_indexes = 8;
+}
+```
+
+### Managed ledger data formats
+
+```proto
+message BatchDeletedIndexRange {
+required uint32 lowerIndex = 1;
+required uint32 upperIndex = 2;
+}
+
+message BatchDeletedIndexInfo {
+required NestedPositionInfo position = 1;
+repeated BatchDeletedIndexRange batchDeletedIndexes = 2;
+}
+```
+
+```proto
+
+message PositionInfo {
+// Store which index in the batch message has been deleted
+repeated BatchDeletedIndexInfo batchDeletedIndexes = 5;
+}
+```
+
+## Configuration
+Added a flag named `batchIndexAcknowledgeEnable` in broker.conf to enable or
disable the batch index acknowledgement
\ No newline at end of file
[GitHub] [pulsar] sijie closed pull request #6007: [Issue 5996][docs] Update links in 2.4.1 and 2.4.2 releases
sijie closed pull request #6007: [Issue 5996][docs] Update links in 2.4.1 and 2.4.2 releases URL: https://github.com/apache/pulsar/pull/6007 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6007: [Issue 5996][docs] Update links in 2.4.1 and 2.4.2 releases
sijie commented on issue #6007: [Issue 5996][docs] Update links in 2.4.1 and 2.4.2 releases URL: https://github.com/apache/pulsar/pull/6007#issuecomment-575626131 @Jennifer88huang it seems that this pull request was created based on your master, which I can help rebase the branch. I have re-created #6080 for you. so I will close this one and use #6080 for merging this change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie opened a new pull request #6080: [Issue 5996][docs] Update broken links in 2.4.1 and 2.4.2 releases
sijie opened a new pull request #6080: [Issue 5996][docs] Update broken links in 2.4.1 and 2.4.2 releases URL: https://github.com/apache/pulsar/pull/6080 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6031: [Issue 4175] [pulsar-function-go] Add Go Function heartbeat (and gRPC service) for production usage
sijie commented on issue #6031: [Issue 4175] [pulsar-function-go] Add Go Function heartbeat (and gRPC service) for production usage URL: https://github.com/apache/pulsar/pull/6031#issuecomment-575624328 @devinbost can you rebase this pull request? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6028: [Issue 5690][pulsar-io] Make type name of Elasticsearch sink connector configurable.
sijie commented on issue #6028: [Issue 5690][pulsar-io] Make type name of Elasticsearch sink connector configurable. URL: https://github.com/apache/pulsar/pull/6028#issuecomment-575623805 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5837: [release] add release notes for 2.5.0 release
sijie commented on issue #5837: [release] add release notes for 2.5.0 release URL: https://github.com/apache/pulsar/pull/5837#issuecomment-575623272 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5837: [release] add release notes for 2.5.0 release
sijie commented on issue #5837: [release] add release notes for 2.5.0 release URL: https://github.com/apache/pulsar/pull/5837#issuecomment-575623002 @Anonymitaet @codelipenghui I have addressed your comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] VitalyShein edited a comment on issue #6079: Cannot create producer via websocket in standalone mode
VitalyShein edited a comment on issue #6079: Cannot create producer via websocket in standalone mode URL: https://github.com/apache/pulsar/issues/6079#issuecomment-575618435 I think this is the same issue: https://github.com/apache/pulsar/issues/5997 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] VitalyShein commented on issue #6079: Cannot create producer via websocket in standalone mode
VitalyShein commented on issue #6079: Cannot create producer via websocket in standalone mode URL: https://github.com/apache/pulsar/issues/6079#issuecomment-575618435 I think this is the same promblem: https://github.com/apache/pulsar/issues/5997 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5278: Fixed reader cursor seek with disconnection
tuteng commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-575617076 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up
codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up URL: https://github.com/apache/pulsar/pull/6077#issuecomment-575615610 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up
codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up URL: https://github.com/apache/pulsar/pull/6077#issuecomment-575615644 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] VitalyShein opened a new issue #6079: Cannot create producer via websocket in standalone mode
VitalyShein opened a new issue #6079: Cannot create producer via websocket in
standalone mode
URL: https://github.com/apache/pulsar/issues/6079
**Describe the bug**
Websocket api is broken - it is impossible to connect with producer to any
existing topic.
**To Reproduce**
Steps to reproduce the behavior:
1. Run pulsar in standalone mode (docker-compose file):
```
standalone:
image: apachepulsar/pulsar-standalone:2.5.0
expose:
- 8080
- 6650
ports:
- "6650:6650"
- "8080:8080"
```
2. Try to create producer via websocket (python 3.7)
```
import asyncio
import websockets
async def connect_test():
TOPIC =
'ws://localhost:8080/ws/v2/producer/persistent/public/default/my-topic'
async with websockets.connect(TOPIC) as websocket:
print('connected!')
asyncio.get_event_loop().run_until_complete(connect_test())
```
3. An exception will occur: error 400
4. Pulsar logs in docker:
```
12:46:38.002 [pulsar-web-57-6] WARN
org.apache.pulsar.websocket.ProducerHandler - [172.18.0.1:48562] Failed in
creating producer on topic persistent://public/default/my-topic: Param
serviceUrl must not be blank.
12:46:38.002 [pulsar-web-57-6] INFO org.eclipse.jetty.server.RequestLog -
172.18.0.1 - - [17/Jan/2020:12:46:38 +] "GET
/ws/v2/producer/persistent/public/default/my-topic HTTP/1.1" 400 462 "-"
"Python/3.7 websockets/8.1" 1
```
5. Change version of pulsar to 2.4.2 - all is working fine!
**Desktop (please complete the following information):**
- OS: Windows 10
- Docker-engine: 19.0.3.5
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5716: [Issues 5709]remove the namespace checking
tuteng commented on issue #5716: [Issues 5709]remove the namespace checking URL: https://github.com/apache/pulsar/pull/5716#issuecomment-575612001 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6071: [website] Download page is messed up
sijie commented on issue #6071: [website] Download page is messed up URL: https://github.com/apache/pulsar/issues/6071#issuecomment-575604822 The new website has been built. The issue is fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #6071: [website] Download page is messed up
sijie closed issue #6071: [website] Download page is messed up URL: https://github.com/apache/pulsar/issues/6071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar.wiki] branch master updated: Updated Release process (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 65a4d56 Updated Release process (markdown) 65a4d56 is described below commit 65a4d5684eba34c5b4ea20b60346778ba3d55974 Author: Sijie Guo AuthorDate: Fri Jan 17 20:18:02 2020 +0800 Updated Release process (markdown) --- Release-process.md | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/Release-process.md b/Release-process.md index 1b67641..c99e715 100644 --- a/Release-process.md +++ b/Release-process.md @@ -246,17 +246,6 @@ svn move https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.X.0-candidate-1 https://dist.apache.org/repos/dist/release/pulsar/pulsar-2.X.0 ``` -Remove the old releases (if any). We only need the latest release there, older releases are -available through the Apache archive: - -```shell -# Get the list of releases -svn ls https://dist.apache.org/repos/dist/release/pulsar - -# Delete each release (except for the last one) -svn rm https://dist.apache.org/repos/dist/release/pulsar/pulsar-2.Y.0 -``` - Promote the Maven staging repository for release. Login to `https://repository.apache.org` and select the staging repository associated with the RC candidate that was approved. The naming will be like `orgapachepulsar-XYZ`. Select the repository and click on "Release". Artifacts @@ -409,3 +398,16 @@ The Pulsar Team It is encouraged to write a blog post to summarize the features introduced in this release, especially for feature releases. You can follow the example [here](https://github.com/apache/pulsar/pull/2308) + +17. Remove old releases + +Remove the old releases (if any). We only need the latest release there, older releases are +available through the Apache archive: + +```shell +# Get the list of releases +svn ls https://dist.apache.org/repos/dist/release/pulsar + +# Delete each release (except for the last one) +svn rm https://dist.apache.org/repos/dist/release/pulsar/pulsar-2.Y.0 +```
[GitHub] [pulsar] murong00 opened a new pull request #6078: Make log flush immediately
murong00 opened a new pull request #6078: Make log flush immediately URL: https://github.com/apache/pulsar/pull/6078 ### Motivation Some users may confuse by pulsar/bookie log without flushing immediately. ### Modifications Change `immediateFlush` in `conf/log4j2.yaml` default to `true`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #6073: Unacked message timeout redelivery for ZeroQueueConsumer
sijie closed issue #6073: Unacked message timeout redelivery for ZeroQueueConsumer URL: https://github.com/apache/pulsar/issues/6073 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6073: Unacked message timeout redelivery for ZeroQueueConsumer
sijie commented on issue #6073: Unacked message timeout redelivery for ZeroQueueConsumer URL: https://github.com/apache/pulsar/issues/6073#issuecomment-575602365 This is fixed by #6076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5837: [release] add release notes for 2.5.0 release
sijie commented on a change in pull request #5837: [release] add release notes for 2.5.0 release URL: https://github.com/apache/pulsar/pull/5837#discussion_r367895583 ## File path: site2/website/release-notes.md ## @@ -1,5 +1,134 @@ -## Apache +## Apache Pulsar Release Notes + +### 2.5.0 — 2019-12-06 + + PIPs + +- [PIP-41] Introduce Protocol Handler API [#5130](https://github.com/apache/pulsar/pull/5130) [#5743](https://github.com/apache/pulsar/pull/5473) +- [PIP-45] Pluggable metadata interface [#5330](https://github.com/apache/pulsar/pull/5330) +- [PIP-44] Separate schema compatibility checker for producer and consumer [#5227](https://github.com/apache/pulsar/pull/5227) +- [PIP-43] Producer send messages with different schema [#5141](https://github.com/apache/pulsar/issues/5141) [#5517](https://github.com/apache/pulsar/pull/5517) +- [PIP-51] Introduce sticky consumer [#5388](https://github.com/apache/pulsar/pull/5388) +- [PIP-38] Support batch receive in java client. [#4621](https://github.com/apache/pulsar/pull/4621) +- [PIP-52] PIP-52: [pulsar-sever] Add support of dispatch throttling relative to publish-rate [#5797](https://github.com/apache/pulsar/pull/5797) + + Fixes + +- [Broker] Avoid retrying deleting namespace when topic is already deleted/fenced [#4665](https://github.com/apache/pulsar/pull/4665) +- [pulsar-broker] Fix expiry monitor to continue on non-recoverable error [#4818](https://github.com/apache/pulsar/pull/4818) Review comment: yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5837: [release] add release notes for 2.5.0 release
sijie commented on a change in pull request #5837: [release] add release notes for 2.5.0 release URL: https://github.com/apache/pulsar/pull/5837#discussion_r367895672 ## File path: site2/website/release-notes.md ## @@ -1,5 +1,134 @@ -## Apache +## Apache Pulsar Release Notes + +### 2.5.0 — 2019-12-06 + + PIPs + +- [PIP-41] Introduce Protocol Handler API [#5130](https://github.com/apache/pulsar/pull/5130) [#5743](https://github.com/apache/pulsar/pull/5473) +- [PIP-45] Pluggable metadata interface [#5330](https://github.com/apache/pulsar/pull/5330) +- [PIP-44] Separate schema compatibility checker for producer and consumer [#5227](https://github.com/apache/pulsar/pull/5227) +- [PIP-43] Producer send messages with different schema [#5141](https://github.com/apache/pulsar/issues/5141) [#5517](https://github.com/apache/pulsar/pull/5517) +- [PIP-51] Introduce sticky consumer [#5388](https://github.com/apache/pulsar/pull/5388) +- [PIP-38] Support batch receive in java client. [#4621](https://github.com/apache/pulsar/pull/4621) +- [PIP-52] PIP-52: [pulsar-sever] Add support of dispatch throttling relative to publish-rate [#5797](https://github.com/apache/pulsar/pull/5797) + + Fixes + +- [Broker] Avoid retrying deleting namespace when topic is already deleted/fenced [#4665](https://github.com/apache/pulsar/pull/4665) +- [pulsar-broker] Fix expiry monitor to continue on non-recoverable error [#4818](https://github.com/apache/pulsar/pull/4818) +- [pulsar-broker] fix ns-isolation api to fetch policy for specific broker [#5314](https://github.com/apache/pulsar/pull/5314) +- [broker] external protocols not set to local broker data [#5749](https://github.com/apache/pulsar/pull/5479) +- [broker]Add handle exception KeeperException.BadVersionException [#5563](https://github.com/apache/pulsar/pull/5563) +- Fix message deduplicate issue while using external sequence id with batch produce [#5491](https://github.com/apache/pulsar/pull/5491) +- Remove cursor while remove non-durable subscription [#5719](https://github.com/apache/pulsar/pull/5719) + + Enhancements + +- Process requests asynchronously on some REST APIs [4765](https://github.com/apache/pulsar/pull/4765) [4778](https://github.com/apache/pulsar/pull/4778) [4795](https://github.com/apache/pulsar/pull/4795) +- Reduce char[] creation on jvm heap [#5055](https://github.com/apache/pulsar/pull/5055) +- Fixes not owned bundles got selected when broker overloading [#5002](https://github.com/apache/pulsar/pull/5002) +- [pulsar-broker] support update partition for global topic [#5306](https://github.com/apache/pulsar/pull/5306) +- Ensure the handling of PartitionMetadataRequest is async end-to-end [#5307](https://github.com/apache/pulsar/pull/5307) +- [pulsar-broker] Introduce publish rate-limiting on topic [#3986](https://github.com/apache/pulsar/pull/3986) +- Allow to automatically assign TCP ports when starting a broker [#3555](https://github.com/apache/pulsar/pull/3555) +- [zookeeper] bump zookeeper to version 3.5.6 [#5043](https://github.com/apache/pulsar/pull/5043) +- Validate topic name before creating partition/non partition topic via admin cli. [#5148](https://github.com/apache/pulsar/pull/5148) Review comment: sure thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (a75cd7d -> 787bee1)
This is an automated email from the ASF dual-hosted git repository. zhaijia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from a75cd7d [website] Update website to include release 2.5.0 (#6075) add 787bee1 Fix zero queue consumer message redelivery (#6076) No new revisions were added by this update. Summary of changes: .../pulsar/client/impl/ZeroQueueSizeTest.java | 73 ++ .../pulsar/client/impl/ZeroQueueConsumerImpl.java | 5 +- 2 files changed, 77 insertions(+), 1 deletion(-)
[GitHub] [pulsar] jiazhai merged pull request #6076: Fix zero queue consumer message redelivery
jiazhai merged pull request #6076: Fix zero queue consumer message redelivery URL: https://github.com/apache/pulsar/pull/6076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] bsideup commented on issue #5278: Fixed reader cursor seek with disconnection
bsideup commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-575567826 @tuteng thank you so much! I will continue keeping my eye on this change and try to test it once merged to double check that the issue is gone (or maybe even before the merge if you can ping me) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5278: Fixed reader cursor seek with disconnection
tuteng commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-575563125 @bsideup I got it, move to 2.4.3, ping @merlimat Please rebase for handle conflict This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] bsideup commented on issue #5278: Fixed reader cursor seek with disconnection
bsideup commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-575558729 @tuteng this is a major issue when manual offset management is applied and blocks projects like Liiklus, see my previous comment: https://github.com/apache/pulsar/pull/5278#issuecomment-555088289 It's been re-scheduled many times already, with promises to be fixed in the next patch release, yet the issue is still there and blocks us from progressing with Pulsar 😕 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #6075: [website] Update website to include release 2.5.0
sijie merged pull request #6075: [website] Update website to include release 2.5.0 URL: https://github.com/apache/pulsar/pull/6075 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (fd4d0dd -> a75cd7d)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from fd4d0dd [Go Functions] Replace cgo client with native go client in Go Function (#6059) add a75cd7d [website] Update website to include release 2.5.0 (#6075) No new revisions were added by this update. Summary of changes: site2/website/releases.json|1 + .../versioned_docs/version-2.5.0/adaptors-kafka.md | 264 +++ .../admin-api-non-partitioned-topics.md| 104 + .../version-2.5.0/admin-api-partitioned-topics.md | 376 .../version-2.5.0/admin-api-persistent-topics.md | 661 ++ .../version-2.5.0/admin-api-schemas.md |7 + .../version-2.5.0/administration-dashboard.md | 63 + .../version-2.5.0/administration-geo.md| 158 ++ .../version-2.5.0/administration-load-balance.md | 182 ++ .../version-2.5.0/administration-proxy.md | 105 + .../version-2.5.0/administration-pulsar-manager.md | 133 ++ .../version-2.5.0/administration-stats.md | 64 + .../version-2.5.0/administration-upgrade.md| 151 ++ .../version-2.5.0/administration-zk-bk.md | 322 +++ .../version-2.5.0/client-libraries-cpp.md | 178 ++ .../version-2.5.0/client-libraries-go.md | 493 + .../version-2.5.0/client-libraries-java.md | 825 .../version-2.5.0/client-libraries-node.md | 402 .../version-2.5.0/client-libraries-python.md | 249 +++ .../version-2.5.0/concepts-clients.md | 82 + .../version-2.5.0/concepts-messaging.md| 445 .../version-2.5.0/concepts-overview.md | 31 + .../version-2.5.0/concepts-tiered-storage.md | 18 + .../version-2.5.0/cookbooks-deduplication.md | 121 ++ .../version-2.5.0/cookbooks-retention-expiry.md| 291 +++ .../version-2.5.0/cookbooks-tiered-storage.md | 296 +++ .../versioned_docs/version-2.5.0/deploy-aws.md | 224 ++ .../deploy-bare-metal-multi-cluster.md | 426 .../version-2.5.0/deploy-bare-metal.md | 461 .../versioned_docs/version-2.5.0/deploy-dcos.md| 183 ++ .../version-2.5.0/deploy-kubernetes.md | 394 .../version-2.5.0/deploy-monitoring.md | 90 + .../versioned_docs/version-2.5.0/functions-cli.md | 198 ++ .../version-2.5.0/functions-debug.md | 455 .../version-2.5.0/functions-develop.md | 983 + .../version-2.5.0/functions-metrics.md |7 + .../version-2.5.0/functions-overview.md| 200 ++ .../version-2.5.0/functions-runtime.md | 173 ++ .../version-2.5.0/functions-worker.md | 242 +++ .../version-2.5.0/getting-started-clients.md | 59 + .../version-2.5.0/getting-started-docker.md| 161 ++ .../version-2.5.0/getting-started-standalone.md| 226 ++ .../version-2.5.0/io-aerospike-sink.md | 26 + .../version-2.5.0/io-canal-source.md | 203 ++ .../version-2.5.0/io-cassandra-sink.md | 54 + .../version-2.5.0/io-cdc-debezium.md | 475 + .../website/versioned_docs/version-2.5.0/io-cdc.md | 26 + .../website/versioned_docs/version-2.5.0/io-cli.md | 601 ++ .../versioned_docs/version-2.5.0/io-connectors.md | 189 ++ .../version-2.5.0/io-debezium-source.md| 350 .../versioned_docs/version-2.5.0/io-develop.md | 230 ++ .../version-2.5.0/io-elasticsearch-sink.md | 50 + .../versioned_docs/version-2.5.0/io-file-source.md | 138 ++ .../versioned_docs/version-2.5.0/io-flume-sink.md | 52 + .../version-2.5.0/io-flume-source.md | 52 + .../versioned_docs/version-2.5.0/io-hbase-sink.md | 64 + .../versioned_docs/version-2.5.0/io-hdfs2-sink.md | 54 + .../versioned_docs/version-2.5.0/io-hdfs3-sink.md | 54 + .../version-2.5.0/io-influxdb-sink.md | 62 + .../versioned_docs/version-2.5.0/io-jdbc-sink.md | 57 + .../versioned_docs/version-2.5.0/io-kafka-sink.md | 69 + .../version-2.5.0/io-kafka-source.md | 171 ++ .../version-2.5.0/io-kinesis-sink.md | 73 + .../version-2.5.0/io-kinesis-source.md | 77 + .../versioned_docs/version-2.5.0/io-mongo-sink.md | 52 + .../version-2.5.0/io-netty-source.md | 205 ++ .../versioned_docs/version-2.5.0/io-overview.md| 136 ++ .../versioned_docs/version-2.5.0/io-quickstart.md | 824 .../version-2.5.0/io-rabbitmq-sink.md | 81 + .../version-2.5.0/io-rabbitmq-source.md| 78 + .../versioned_docs/version-2.5.0/io-redis-sink.md | 70 + .../versioned_docs/version-2.5.0/io-solr-sink.md | 61 + .../version-2.5.0/io-twitter-source.md | 28 + .../versioned_docs/version-2.5.0/io-t
[GitHub] [pulsar] sijie commented on a change in pull request #6075: [website] Update website to include release 2.5.0
sijie commented on a change in pull request #6075: [website] Update website to include release 2.5.0 URL: https://github.com/apache/pulsar/pull/6075#discussion_r367843047 ## File path: site2/website/releases.json ## @@ -1,5 +1,6 @@ [ "2.4.2", + "2.5.0", Review comment: this was done intentionally. because we want algolia index the website first before making it the stable release. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up
codelipenghui commented on issue #6077: Support delete inactive topic when subscriptions caught up URL: https://github.com/apache/pulsar/pull/6077#issuecomment-575547692 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui opened a new pull request #6077: Support delete inactive topic when subscriptions caught up
codelipenghui opened a new pull request #6077: Support delete inactive topic when subscriptions caught up URL: https://github.com/apache/pulsar/pull/6077 Fixes #6061 #4824 ### Motivation Currently, pulsar support delete inactive topic which has no active producers and no subscriptions. This pull request is support to delete inactive topics that all subscriptions of the topic are caught up and no active producers/consumer. ### Modifications Expose inactive topic delete mode in broker.conf, future more we can support namespace level configuration for the inactive topic delete mode. ### Verifying this change New unit tests added for each inactive topic delete mode. ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation - Does this pull request introduce a new feature? (no) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (63db362 -> fd4d0dd)
This is an automated email from the ASF dual-hosted git repository. rxl pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 63db362 Fixed website generate site (#6026) add fd4d0dd [Go Functions] Replace cgo client with native go client in Go Function (#6059) No new revisions were added by this update. Summary of changes: pulsar-function-go/examples/test/consumer.go | 2 +- pulsar-function-go/examples/test/producer.go | 8 +++-- pulsar-function-go/go.mod| 12 --- pulsar-function-go/go.sum| 50 pulsar-function-go/pf/instance.go| 13 +++- pulsar-function-go/pf/logAppender.go | 11 ++ 6 files changed, 50 insertions(+), 46 deletions(-)
[GitHub] [pulsar] wolfstudy merged pull request #6059: [Go Functions] Replace cgo client with native go client in Go Function
wolfstudy merged pull request #6059: [Go Functions] Replace cgo client with native go client in Go Function URL: https://github.com/apache/pulsar/pull/6059 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (8cf56f2 -> 63db362)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 8cf56f2 Fix maven broken link (#6068) add 63db362 Fixed website generate site (#6026) No new revisions were added by this update. Summary of changes: site2/tools/build-site.sh| 4 site2/tools/generate-api-docs.sh | 3 --- 2 files changed, 4 insertions(+), 3 deletions(-)
[GitHub] [pulsar] sijie commented on issue #6051: Expose lastConsumedTimestamp and lastAckedTimestamp to consumer stats
sijie commented on issue #6051: Expose lastConsumedTimestamp and lastAckedTimestamp to consumer stats URL: https://github.com/apache/pulsar/pull/6051#issuecomment-575540132 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #6026: [website]Hot fixed website generate document order error
sijie merged pull request #6026: [website]Hot fixed website generate document order error URL: https://github.com/apache/pulsar/pull/6026 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on a change in pull request #6075: [website] Update website to include release 2.5.0
jiazhai commented on a change in pull request #6075: [website] Update website to include release 2.5.0 URL: https://github.com/apache/pulsar/pull/6075#discussion_r367828027 ## File path: site2/website/releases.json ## @@ -1,5 +1,6 @@ [ "2.4.2", + "2.5.0", Review comment: nit: Put it at the first line? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai edited a comment on issue #6076: Fix zero queue consumer message redelivery
jiazhai edited a comment on issue #6076: Fix zero queue consumer message redelivery URL: https://github.com/apache/pulsar/pull/6076#issuecomment-575535255 run java8 tests Failed: org.apache.pulsar.broker.admin.AdminApiOffloadTest.testOffloadV1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #6076: Fix zero queue consumer message redelivery
jiazhai commented on issue #6076: Fix zero queue consumer message redelivery URL: https://github.com/apache/pulsar/pull/6076#issuecomment-575535255 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
