[GitHub] [pulsar] merlimat commented on issue #1190: Deleting a topic should succeed even if data is gone from bookies
merlimat commented on issue #1190: Deleting a topic should succeed even if data is gone from bookies URL: https://github.com/apache/pulsar/issues/1190#issuecomment-540379391 The error was not referring to the `NoSuchLedgerExistsException`. That is when the metadata for the ledger is not found. There are still examples in which the topic will fail to get deleted: 1. Create a topic, create some subscription and publish some data 2. Delete the data from the bookie (all the replicas) 3. Kill broker 4. Try to delete topic What will happen is that a new broker will first try to load the topic, triggering the ledger recovery operation, which will fail since the data is gone. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on issue #1190: Deleting a topic should succeed even if data is gone from bookies
MarvinCai commented on issue #1190: Deleting a topic should succeed even if data is gone from bookies URL: https://github.com/apache/pulsar/issues/1190#issuecomment-540366659 @merlimat Looking into the code, [PersistentTopic#delete](https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L838) will first remove cursor metadata then remove cursor ledgers, but it doesn't care about result of deleting cursor ledger so it'll proceed even with failure. Then it'll try [delete all ledgers](https://github.com/apache/pulsar/blob/master/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L2195-L2202), and this method will treat BKException.Code.NoSuchLedgerExistsException as succeed and proceed to delete ledger metadata from ZK if all ledger either deleted successfully or doesn't exist anymore. So seems current logic to delete topic can handle the situation you mentioned properly when ledgers are already gone from bookkeeper. Is there any further change need to made? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540350598 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5354: Test do not merge
aahmed-se commented on issue #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354#issuecomment-540339546 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404
aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540338028 @tuteng rebase this run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5296: [Doc] Update connector guide overall
aahmed-se commented on issue #5296: [Doc] Update connector guide overall URL: https://github.com/apache/pulsar/pull/5296#issuecomment-540337665 @Anonymitaet rebase this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5338: [Docs] Add putstate cli docs for functions
aahmed-se commented on issue #5338: [Docs] Add putstate cli docs for functions URL: https://github.com/apache/pulsar/pull/5338#issuecomment-540335333 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5296: [Doc] Update connector guide overall
aahmed-se commented on issue #5296: [Doc] Update connector guide overall URL: https://github.com/apache/pulsar/pull/5296#issuecomment-540335616 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide*
Anonymitaet commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide* URL: https://github.com/apache/pulsar/pull/5290#issuecomment-540334602 @aahmed-se thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5211: [Doc] Add *Kinesis source connector guide*
Anonymitaet commented on issue #5211: [Doc] Add *Kinesis source connector guide* URL: https://github.com/apache/pulsar/pull/5211#issuecomment-540334441 @aahmed-se thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (5df6488 -> 1611470)
This is an automated email from the ASF dual-hosted git repository.
hjf pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 5df6488 Use Buffered channels to avoid blocking on callback (#5336)
add 1611470 [Doc] Add *HDFS2 sink connector guide* (#5226)
No new revisions were added by this update.
Summary of changes:
site2/docs/io-connectors.md | 4 ++--
site2/docs/io-hdfs.md | 26 ---
site2/docs/{io-hdfs3-sink.md => io-hdfs2-sink.md} | 12 +--
3 files changed, 8 insertions(+), 34 deletions(-)
delete mode 100644 site2/docs/io-hdfs.md
copy site2/docs/{io-hdfs3-sink.md => io-hdfs2-sink.md} (89%)
[GitHub] [pulsar] aahmed-se commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide*
aahmed-se commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide* URL: https://github.com/apache/pulsar/pull/5290#issuecomment-540333571 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang merged pull request #5226: [Doc] Add *HDFS2 sink connector guide*
Jennifer88huang merged pull request #5226: [Doc] Add *HDFS2 sink connector guide* URL: https://github.com/apache/pulsar/pull/5226 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5338: [Docs] Add putstate cli docs for functions
aahmed-se commented on issue #5338: [Docs] Add putstate cli docs for functions URL: https://github.com/apache/pulsar/pull/5338#issuecomment-540332577 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404
aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540331922 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5226: [Doc] Add *HDFS2 sink connector guide*
Anonymitaet commented on issue #5226: [Doc] Add *HDFS2 sink connector guide* URL: https://github.com/apache/pulsar/pull/5226#issuecomment-540332163 @aahmed-se thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide*
aahmed-se commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide* URL: https://github.com/apache/pulsar/pull/5290#issuecomment-540331479 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5211: [Doc] Add *Kinesis source connector guide*
aahmed-se commented on issue #5211: [Doc] Add *Kinesis source connector guide* URL: https://github.com/apache/pulsar/pull/5211#issuecomment-540331132 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5226: [Doc] Add *HDFS2 sink connector guide*
aahmed-se commented on issue #5226: [Doc] Add *HDFS2 sink connector guide* URL: https://github.com/apache/pulsar/pull/5226#issuecomment-540330882 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404
aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540328352 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5354: Test do not merge
aahmed-se commented on issue #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354#issuecomment-540328083 rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5354: Test do not merge
aahmed-se commented on issue #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354#issuecomment-540324121 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540318972 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5342: Use simple GC settings when running integration tests
aahmed-se commented on issue #5342: Use simple GC settings when running integration tests URL: https://github.com/apache/pulsar/pull/5342#issuecomment-540318440 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5354: Test do not merge
aahmed-se commented on issue #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354#issuecomment-540317294 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5354: Test do not merge
aahmed-se commented on issue #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354#issuecomment-540317128 rerun tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5354: Test do not merge
aahmed-se commented on issue #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354#issuecomment-540316028 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5354: Test do not merge
aahmed-se commented on issue #5354: Test do not merge URL: https://github.com/apache/pulsar/pull/5354#issuecomment-540316071 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #5353: [Doc] Test do not merge
aahmed-se closed pull request #5353: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5353 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #5354: Test
aahmed-se opened a new pull request #5354: Test URL: https://github.com/apache/pulsar/pull/5354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #5352: [Doc] Test do not merge
aahmed-se closed pull request #5352: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5352 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #4811: [Issue: 3161][Go client] Return Message ID for send
wolfstudy edited a comment on issue #4811: [Issue: 3161][Go client] Return Message ID for send URL: https://github.com/apache/pulsar/pull/4811#issuecomment-540312794 Sorry for the late response. @massakam @merlimat PTAL again thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #4811: [Issue: 3161][Go client] Return Message ID for send
wolfstudy commented on issue #4811: [Issue: 3161][Go client] Return Message ID for send URL: https://github.com/apache/pulsar/pull/4811#issuecomment-540312794 Sorry for the late response. @massakam PTAL again thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5353: [Doc] Test do not merge
aahmed-se commented on issue #5353: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5353#issuecomment-540311798 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch xiaolong/send_return_msg deleted (was 933d5ef)
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch xiaolong/send_return_msg in repository https://gitbox.apache.org/repos/asf/pulsar.git. was 933d5ef replace SendWithMsgID with SendAndGetMsgID This change permanently discards the following revisions: discard 933d5ef replace SendWithMsgID with SendAndGetMsgID
[GitHub] [pulsar] aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404
aahmed-se commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540310969 rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch xiaolong/send_return_msg created (now 933d5ef)
This is an automated email from the ASF dual-hosted git repository. rxl pushed a change to branch xiaolong/send_return_msg in repository https://gitbox.apache.org/repos/asf/pulsar.git. at 933d5ef replace SendWithMsgID with SendAndGetMsgID This branch includes the following new commits: new 933d5ef replace SendWithMsgID with SendAndGetMsgID The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[pulsar] 01/01: replace SendWithMsgID with SendAndGetMsgID
This is an automated email from the ASF dual-hosted git repository.
rxl pushed a commit to branch xiaolong/send_return_msg
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 933d5efca2f055dc3116dd9c02720c8aaab6cf13
Merge: 5564bc8 5df6488
Author: xiaolong.ran
AuthorDate: Thu Oct 10 10:44:54 2019 +0800
replace SendWithMsgID with SendAndGetMsgID
Signed-off-by: xiaolong.ran
.github/PULL_REQUEST_TEMPLATE.md | 2 +-
.gitignore | 1 +
README.md | 4 +-
bin/pulsar | 2 +-
conf/broker.conf |37 +
conf/functions_worker.yml | 6 +
conf/log4j2.yaml | 3 +-
conf/presto/catalog/pulsar.properties |27 +-
conf/standalone.conf |40 +
dashboard/Dockerfile | 6 +-
dashboard/django/collector.py | 4 +-
dashboard/django/stats/models.py | 6 +
dashboard/django/stats/static/stats/additional.css |54 +
.../django/stats/templates/stats/messages.html |58 +-
dashboard/django/stats/templates/stats/peek.html |22 -
dashboard/django/stats/urls.py | 7 +-
dashboard/django/stats/views.py| 289 +-
dashboard/django/utils/__init__.py |19 +
dashboard/django/utils/import_utils.py |27 +
dashboard/requirements.txt | 8 +
deployment/kubernetes/README.md| 2 +-
deployment/kubernetes/generic/admin.yaml |44 -
.../generic/k8s-1-9-and-above/admin.yaml |44 +
.../generic/k8s-1-9-and-above/bookie.yaml | 134 +
.../generic/k8s-1-9-and-above/broker.yaml | 108 +
.../{ => k8s-1-9-and-above}/cluster-metadata.yaml | 0
.../generic/k8s-1-9-and-above/monitoring.yaml | 207 +
.../generic/k8s-1-9-and-above/proxy.yaml |92 +
.../generic/k8s-1-9-and-above/zookeeper.yaml | 149 +
deployment/kubernetes/generic/original/admin.yaml |44 +
.../kubernetes/generic/{ => original}/bookie.yaml | 0
.../kubernetes/generic/{ => original}/broker.yaml | 0
.../generic/{ => original}/cluster-metadata.yaml | 0
.../generic/{ => original}/monitoring.yaml | 0
.../kubernetes/generic/{ => original}/proxy.yaml | 0
.../generic/{ => original}/zookeeper.yaml | 0
.../pulsar/templates/autorecovery-deployment.yaml | 2 +-
distribution/io/src/assemble/io.xml| 3 +
distribution/offloaders/pom.xml| 2 +-
distribution/server/src/assemble/LICENSE.bin.txt |16 +-
docker/pulsar-standalone/django/collector.py | 2 +-
docker/pulsar/Dockerfile | 7 +-
docker/pulsar/pom.xml |28 +
docker/pulsar/scripts/gen-yml-from-env.py | 2 +-
managed-ledger-shaded/pom.xml | 295 -
managed-ledger/pom.xml | 2 +-
.../apache/bookkeeper/mledger/AsyncCallbacks.java | 3 +-
.../apache/bookkeeper/mledger/ManagedCursor.java |21 +
.../apache/bookkeeper/mledger/ManagedLedger.java | 5 +
.../bookkeeper/mledger/impl/ManagedCursorImpl.java |77 +-
.../bookkeeper/mledger/impl/ManagedLedgerImpl.java |64 +-
.../apache/bookkeeper/mledger/impl/OpAddEntry.java | 2 +-
.../bookkeeper/mledger/impl/OpFindNewest.java | 5 +-
.../mledger/impl/ReadOnlyManagedLedgerImpl.java| 2 +-
.../mledger/impl/EntryCacheManagerTest.java| 9 +-
.../mledger/impl/ManagedCursorConcurrencyTest.java |14 +-
.../mledger/impl/ManagedCursorContainerTest.java |30 +-
.../bookkeeper/mledger/impl/ManagedCursorTest.java | 123 +-
.../mledger/impl/ManagedLedgerErrorsTest.java |56 +-
.../impl/ManagedLedgerSingleBookieTest.java| 6 +-
.../mledger/impl/ManagedLedgerTerminationTest.java |10 +-
.../bookkeeper/mledger/impl/ManagedLedgerTest.java | 229 +-
.../mledger/impl/NonDurableCursorTest.java |25 +-
.../mledger/impl/OffloadLedgerDeleteTest.java | 3 +-
.../mledger/impl/OffloadPrefixReadTest.java| 1 +
.../bookkeeper/mledger/impl/OffloadPrefixTest.java | 245 +-
.../bookkeeper/mledger/impl/PositionTest.java | 8 +-
.../mledger/impl/ReadOnlyCursorTest.java |10 +-
.../bookkeeper/mledger/util/CallbackMutexTest.java |15 +-
.../bookkeeper/mledger/util/RangeCacheTest.java|21 +-
.../org/apache/bookkeeper/test/PortManager.java| 8 +-
.../java/org/apache/zookeeper/MockZooKeeper.java |87 +-
pom.xml|12 +-
pulsar-broker-auth-athenz/pom.xml
[GitHub] [pulsar] wolfstudy commented on issue #5338: [Docs] Add putstate cli docs for functions
wolfstudy commented on issue #5338: [Docs] Add putstate cli docs for functions URL: https://github.com/apache/pulsar/pull/5338#issuecomment-540302324 @merlimat @sijie @jiazhai @codelipenghui PTAL, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540297977 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5353: [Doc] Test do not merge
aahmed-se commented on issue #5353: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5353#issuecomment-540297672 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5353: [Doc] Test do not merge
aahmed-se commented on issue #5353: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5353#issuecomment-540296679 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5353: [Doc] Test do not merge
aahmed-se commented on issue #5353: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5353#issuecomment-540294329 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5353: [Doc] Test do not merge
aahmed-se commented on issue #5353: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5353#issuecomment-540292190 run java8 tests run integration tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #5353: [Doc] Test do not merge
aahmed-se opened a new pull request #5353: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5353 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #5352: [Doc] Test do not merge
aahmed-se opened a new pull request #5352: [Doc] Test do not merge URL: https://github.com/apache/pulsar/pull/5352 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #5351: [Doc] Test Do not merge
aahmed-se closed pull request #5351: [Doc] Test Do not merge URL: https://github.com/apache/pulsar/pull/5351 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5351: [Doc] Test Do not merge
aahmed-se commented on issue #5351: [Doc] Test Do not merge URL: https://github.com/apache/pulsar/pull/5351#issuecomment-540275650 rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540272996 rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5351: [Doc] Test Do not merge
aahmed-se commented on issue #5351: [Doc] Test Do not merge URL: https://github.com/apache/pulsar/pull/5351#issuecomment-540272368 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540270435 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540266442 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5351: [Doc] Test Do not merge
aahmed-se commented on issue #5351: [Doc] Test Do not merge URL: https://github.com/apache/pulsar/pull/5351#issuecomment-540261207 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5351: [Doc] Test Do not merge
aahmed-se commented on issue #5351: [Doc] Test Do not merge URL: https://github.com/apache/pulsar/pull/5351#issuecomment-540259399 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5351: [Doc] Test Do not merge
aahmed-se commented on issue #5351: [Doc] Test Do not merge URL: https://github.com/apache/pulsar/pull/5351#issuecomment-540256806 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5351: [Doc] Test Do not merge
aahmed-se commented on issue #5351: [Doc] Test Do not merge URL: https://github.com/apache/pulsar/pull/5351#issuecomment-540226499 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #5351: [Doc] Test
aahmed-se opened a new pull request #5351: [Doc] Test URL: https://github.com/apache/pulsar/pull/5351 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540205367 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #5075: [pulsar-client] support set read-position based on timestamp
rdhabalia commented on issue #5075: [pulsar-client] support set read-position based on timestamp URL: https://github.com/apache/pulsar/pull/5075#issuecomment-540201804 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540198073 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540163923 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5350: Pulsar can't load the customized SerDe
sijie commented on issue #5350: Pulsar can't load the customized SerDe URL: https://github.com/apache/pulsar/issues/5350#issuecomment-540154470 The problem is in PulsarSink.java. We don't use the right class loader to load the SerDe class. https://github.com/apache/pulsar/blob/6ff1bbae0fe230eb72d62574d7685a745f884416/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/sink/PulsarSink.java#L330 The fix should be straightforward to use `functionClassLoader` to load the SerDe class. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie opened a new issue #5350: Pulsar can't load the customized SerDe
sijie opened a new issue #5350: Pulsar can't load the customized SerDe URL: https://github.com/apache/pulsar/issues/5350 **Describe the bug** Pulsar couldn't find deployed SerDe class. But it exists in my fat jar ``` 12:59:00.903 [t1/n1/JournalBatchParseFunction-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [t1/n1/JournalBatchParseFunction:0] Uncaught exception in Java Instance java.lang.RuntimeException: User class must be in class path at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:67) ~[org.apache.pulsar-pulsar-functions-utils-2.4.1.jar:2.4.1] at org.apache.pulsar.functions.instance.InstanceUtils.createInstance(InstanceUtils.java:84) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?] at org.apache.pulsar.functions.instance.InstanceUtils.initializeSerDe(InstanceUtils.java:43) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?] at org.apache.pulsar.functions.source.TopicSchema.newSchemaInstance(TopicSchema.java:197) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:2.4.1] at org.apache.pulsar.functions.source.TopicSchema.lambda$getSchema$0(TopicSchema.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:2.4.1] at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_212] at org.apache.pulsar.functions.source.TopicSchema.getSchema(TopicSchema.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:2.4.1] at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:330) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:2.4.1] at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:255) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:2.4.1] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:769) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupJavaInstance(JavaInstanceRunnable.java:203) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:234) [org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212] Caused by: java.lang.ClassNotFoundException: ru.alfabank.api.utils.JournalBatchSerde at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_212] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_212] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_212] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_212] at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_212] at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:65) ~[org.apache.pulsar-pulsar-functions-utils-2.4.1.jar:2.4.1] ... 12 more ``` The command used to submit the function ``` bin/pulsar-admin --admin-url http://corpint4:6680 functions create --retain-ordering --jar http://binary/artifactory/libs-release-local/ru/alfabank/api/utils/pulsar-util-function-api-pack/0.10.2/pulsar-util-function-api-pack-0.10.2-all.jar --classname ru.alfabank.api.utils.JournalBatchParseFunction --inputs persistent://t1/n1/queue_1 --output persistent://t1/n1/queue_journal --output-serde-classname ru.alfabank.api.utils.JournalBatchSerde --tenant t1 --namespace n1 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rocketraman opened a new issue #5349: UnAckedMessageTracker: 3600 messages have timed-out... but they haven't?
rocketraman opened a new issue #5349: UnAckedMessageTracker: 3600 messages have
timed-out... but they haven't?
URL: https://github.com/apache/pulsar/issues/5349
**Describe the bug**
I've deployed a topic into Pulsar 2.4.1 with default configuration for
message TTL, which as I understand it should mean that unacked messages should
be retained forever.
However, my consumers periodically report:
```
org.apa.pul.cli.imp.UnAckedMessageTracker :
[ConsumerBase{subscription='my-sub', consumerName='my-consumer',
topic='persistent://dev/my-ns/q'}] 3600 messages have timed-out
```
I've seen this on 3 separate environments, starting the topic and associated
subscriptions from scratch. Subscriptions are Shared, and configured to start
at position "earliest".
The value is always 3600. In all 3 environments, which have vastly different
numbers of messages in the topic, the number is always 3600. In some
environments I don't believe 3600 messages were even placed into the topic.
This doesn't actually appear to cause any problems -- everything seems to be
working normally. Its just a scary and annoying message to see.
**To Reproduce**
Steps to reproduce the behavior:
1. Create a topic with default settings, change retention size to -1 and
retention period to 1 hour (I can't see how these settings would be relevant,
but that's what my topics have).
2. Produce and consume some messages from the topic.
3. Wait until the warning is reported.
**Expected behavior**
No warning.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image
aahmed-se commented on issue #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335#issuecomment-540138549 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar.wiki] branch master updated: Updated PIP 45: Pluggable metadata interface (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 88dafa2 Updated PIP 45: Pluggable metadata interface (markdown) 88dafa2 is described below commit 88dafa2318cb04de6fabab224e7d1ee33451c978 Author: Matteo Merli AuthorDate: Wed Oct 9 11:34:46 2019 -0700 Updated PIP 45: Pluggable metadata interface (markdown) --- PIP-45:-Pluggable-metadata-interface.md | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/PIP-45:-Pluggable-metadata-interface.md b/PIP-45:-Pluggable-metadata-interface.md index c8404d9..1301dd5 100644 --- a/PIP-45:-Pluggable-metadata-interface.md +++ b/PIP-45:-Pluggable-metadata-interface.md @@ -1,5 +1,10 @@ -# PIP-45 - Pluggable metadata interface +* **Status**: Proposal +* **Author**: Matteo Merli +* **Pull Request**: +* **Mailing List discussion**: +* **Release**: + ## Goals
[pulsar.wiki] branch master updated: Updated PIP 33: Replicated subscriptions (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 843bd77 Updated PIP 33: Replicated subscriptions (markdown) 843bd77 is described below commit 843bd778e85fe7b09c7734beeb12916ed169f24e Author: Matteo Merli AuthorDate: Wed Oct 9 11:34:29 2019 -0700 Updated PIP 33: Replicated subscriptions (markdown) --- PIP-33:-Replicated-subscriptions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/PIP-33:-Replicated-subscriptions.md b/PIP-33:-Replicated-subscriptions.md index 42468c8..9654227 100644 --- a/PIP-33:-Replicated-subscriptions.md +++ b/PIP-33:-Replicated-subscriptions.md @@ -1,9 +1,9 @@ -* **Status**: Proposal +* **Status**: Merged * **Author**: Ivan Kelly, Matteo Merli * **Pull Request**: * **Mailing List discussion**: -* **Release**: +* **Release**: 2.4.0 ## Goal
[pulsar.wiki] branch master updated: Updated Home (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 6fdd4b0 Updated Home (markdown) 6fdd4b0 is described below commit 6fdd4b0e8ee1c9d5a5c8ca01ba5a89030dab15f0 Author: Matteo Merli AuthorDate: Wed Oct 9 11:33:26 2019 -0700 Updated Home (markdown) --- Home.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Home.md b/Home.md index ab6ada5..c66894d 100644 --- a/Home.md +++ b/Home.md @@ -17,11 +17,11 @@ We encourage to document any big change or feature or any addition to public use * [[PIP 38: Batch Receiving Messages]] * [[PIP 37: Large message size handling in Pulsar]] * [[PIP 35: Improve topic lookup for topics that have high number of partitions]] -* [[PIP 33: Replicated subscriptions]] * [[PIP 31: Transaction Support]] * [[PIP 21: Pulsar Edge Component]] ### Accepted +* [[PIP 33: Replicated subscriptions]] * [[PIP 36: Max Message Size]] * [[PIP 34: Add new subscribe type Key_shared]] * [[PIP 32: Go Function API, Instance and LocalRun]]
[pulsar.wiki] branch master updated: Updated Home (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new d5586b8 Updated Home (markdown) d5586b8 is described below commit d5586b86768a1dc36075d35d05f17ac82986a2c2 Author: Matteo Merli AuthorDate: Wed Oct 9 11:32:50 2019 -0700 Updated Home (markdown) --- Home.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Home.md b/Home.md index f2d8524..ab6ada5 100644 --- a/Home.md +++ b/Home.md @@ -7,6 +7,7 @@ We encourage to document any big change or feature or any addition to public use ### Proposed +* [[PIP 45: Pluggable metadata interface]] * [[PIP 44: Separate schema compatibility checker for producer and consumer]] * [[PIP 43: producer send message with different schema]] * [[PIP 42: KoP - Kafka on Pulsar]]
[pulsar.wiki] branch master updated: Created PIP-45: Pluggable metadata interface (markdown)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git
The following commit(s) were added to refs/heads/master by this push:
new 4d31863 Created PIP-45: Pluggable metadata interface (markdown)
4d31863 is described below
commit 4d31863cb233e12fc1d74d5097facfd5869b2d3b
Author: Matteo Merli
AuthorDate: Wed Oct 9 11:32:28 2019 -0700
Created PIP-45: Pluggable metadata interface (markdown)
---
PIP-45:-Pluggable-metadata-interface.md | 317
1 file changed, 317 insertions(+)
diff --git a/PIP-45:-Pluggable-metadata-interface.md
b/PIP-45:-Pluggable-metadata-interface.md
new file mode 100644
index 000..c8404d9
--- /dev/null
+++ b/PIP-45:-Pluggable-metadata-interface.md
@@ -0,0 +1,317 @@
+
+# PIP-45 - Pluggable metadata interface
+
+## Goals
+
+Provide a unified pluggable interface that can abstract all the Pulsar metadata
+interactions.
+
+After the refactoring, the default implementation will still be based on
ZooKeeper and it will
+be 100% compatible with the existing metadata. The metadata will be kept in
the same location
+and in the same exact format.
+
+Once we have the interface defined we could have multiple backend
implementations:
+ * ZooKeeper
+ * Etcd
+ * In memory - for unit tests purposes
+ * On local disk - for usage in Pulsar standalone
+
+## Context
+
+Pulsar is currently using ZooKeeper for metadata and coordination purposes.
These accesses are
+being done from Pulsar brokers and some administrative CLI tools. BookKeeper
already supports
+a pluggable metadata store.
+
+Additionally, ZooKeeper client API is being accessed from several places in
the codebase, so we
+first need to consolidate all these accesses through a single generic
`MetadataStore` interface.
+
+This interface is based on the needs that Pulsar has in interacting with
metadata and with the
+semantics offered by existing metadata stores (eg. ZooKeeper, Etcd and
others). The API will be
+considered as "Beta" (meaning it could be evolved in breaking way) until we
have at least few
+concrete implementations. Therefore, at least initially, this will be an
internal Pulsar API and
+it will not be open to user plugins.
+
+## Refactoring steps
+
+### 1. Define metadata store API
+
+The metadata store is modeled after a basic Key-Value interface with
`compareAndSet()` updates
+based on the version of a particular value.
+
+```java
+public interface MetadataStore extends AutoCloseable {
+
+/**
+ * Read the value of one key, identified by the path
+ *
+ * The async call will return a future that yields a {@link GetResult}
that will contain the value and the
+ * associated {@link Stat} object.
+ *
+ * If the value is not found, the future will yield an empty {@link
Optional}.
+ *
+ * @param path
+ *the path of the key to get from the store
+ * @return a future to track the async request
+ */
+CompletableFuture> get(String path);
+
+/**
+ * Return all the nodes (lexicographically sorted) that are children to
the specific path.
+ *
+ * If the path itself does not exist, it will return an empty list.
+ *
+ * @param path
+ *the path of the key to check on the store
+ * @return a future to track the async request
+ */
+CompletableFuture> getChildren(String path);
+
+/**
+ * Read whether a specific path exists.
+ *
+ * Note: In case of keys with multiple levels (eg: '/a/b/c'), checking the
existence of a parent (eg. '/a') might
+ * not necessarily return true, unless the key had been explicitly created.
+ *
+ * @param path
+ *the path of the key to check on the store
+ * @return a future to track the async request
+ */
+CompletableFuture exists(String path);
+
+/**
+ * Put a new value for a given key.
+ *
+ * The caller can specify an expected version to be atomically checked
against the current version of the stored
+ * data.
+ *
+ * The future will return the {@link Stat} object associated with the
newly inserted value.
+ *
+ *
+ * @param path
+ *the path of the key to delete from the store
+ * @param value
+ *the value to
+ * @param expectedVersion
+ *if present, the version will have to match with the
currently stored value for the operation to
+ *succeed. Use -1 to enforce a non-existing value.
+ * @throws BadVersionException
+ * if the expected version doesn't match the actual version of
the data
+ * @return a future to track the async request
+ */
+CompletableFuture put(String path, byte[] value, Optional
expectedVersion);
+
+/**
+ *
+ * @param path
+ *the path of the key to delete from the store
+ * @param
[pulsar] branch master updated (db15d32 -> 5df6488)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from db15d32 Added basic metadata store API and initial ZK implementation (#5330) add 5df6488 Use Buffered channels to avoid blocking on callback (#5336) No new revisions were added by this update. Summary of changes: pulsar-client-go/pulsar/c_client.go | 14 +++--- pulsar-client-go/pulsar/c_consumer.go | 6 +++--- pulsar-client-go/pulsar/c_producer.go | 6 +++--- pulsar-client-go/pulsar/c_reader.go | 2 +- 4 files changed, 14 insertions(+), 14 deletions(-)
[GitHub] [pulsar] merlimat merged pull request #5336: [Issue 5324][go client] Use Buffered channels to avoid blocking on callback
merlimat merged pull request #5336: [Issue 5324][go client] Use Buffered channels to avoid blocking on callback URL: https://github.com/apache/pulsar/pull/5336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5348: Load tls file fail on Windows
merlimat commented on issue #5348: Load tls file fail on Windows
URL: https://github.com/apache/pulsar/issues/5348#issuecomment-540086629
One workaround is to use the preferred way to add the TLS auth, like:
```java
PulsarClient client = PulsarClient.builder()
.tlsTrustCertsFilePath(".")
.authentication(AuthentiationFactory.TLS(
"H:\\pulsar\\ssl\\1.11\\admin.cert.pem",
"H:\\pulsar\\ssl\\1.11\\admin.key-pk8.pem")
.serviceUrl(localClusterUrl)
.build();
```
(The docs should be updated to suggest this form by default)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] liangyuanpeng opened a new issue #5348: Load tls file fail on Windows
liangyuanpeng opened a new issue #5348: Load tls file fail on Windows
URL: https://github.com/apache/pulsar/issues/5348
**Describe the bug**
```
PulsarClient client = PulsarClient.builder()
.tlsTrustCertsFilePath("H:\\pulsar\\ssl\\1.11\\ca.cert.pem")
.authentication("org.apache.pulsar.client.impl.auth.AuthenticationTls","tlsCertFile:H:\\pulsar\\ssl\\1.11\\admin.cert.pem,tlsKeyFile:H:\\pulsar\\ssl\\1.11\\admin.key-pk8.pem")
.serviceUrl(localClusterUrl)
.build();
```
I try to load tls file on windows,but i just got the error:
```
org.apache.pulsar.client.api.PulsarClientException:
java.lang.IllegalArgumentException: certFilePath must not be null
```
I have debug the client code and i found that
``org.apache.pulsar.client.api#AuthenticationUtil.configureFromPulsar1AuthParamString``
In Windows OS,kv.length = 3, so authParams put fail,
If i missed something, tell me please!
**To Reproduce**
Steps to reproduce the behavior:
1. client load tls on windows
**Expected behavior**
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: Windows
**Additional context**
Add any other context about the problem here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] tuteng opened a new pull request #5347: [Docs]Fix website io connector link 404
tuteng opened a new pull request #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347 ### Motivation Currently, the old version of the connector document is incorrect. The new version of the connector document is not compatible with the old version, resulting in an HTTP 404 response. ### Modifications * Documentation for older versions http link of connectors remains unchanged. * Add document of old versions. ### Verifying this change Local Test Pass This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ccronemberger opened a new issue #5346: bin/pulsar standalone --zookeeper-port 2183 does not work with version 2.4.1
ccronemberger opened a new issue #5346: bin/pulsar standalone --zookeeper-port 2183 does not work with version 2.4.1 URL: https://github.com/apache/pulsar/issues/5346 I need to run Zookeeper in a different port number, so I specify "--zookeeper-port 2183" in the command line, but it is still using default port 2181 in the BookieServer configuration. As you can see in the attached log. **To Reproduce** Steps to reproduce the behavior: 1. Go to Pulsar installation directory 2. Run the command "bin/pulsar standalone --zookeeper-port 2183" 3. Search for 2181 in the logs 4. You will find it in the zookeeperServers property in the log generate for org.apache.bookkeeper.proto.BookieServer. **Expected behavior** It should use the specified port in all places. I even tried to change the configuration in the bookkeeper.conf file, but it didn't help. **Desktop :** - OS: uname -a Linux GFT_BTB 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 30 14:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux [output.log](https://github.com/apache/pulsar/files/3707871/output.log) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet opened a new issue #5345: [Doc] Fix doc issues related to BookKeeper, Python Client, and so on
Anonymitaet opened a new issue #5345: [Doc] Fix doc issues related to BookKeeper, Python Client, and so on URL: https://github.com/apache/pulsar/issues/5345 Fix some issues as shown below:  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] xujianhai666 commented on issue #69: [ISSUE #68][feat]add Option for new producer (#68)
xujianhai666 commented on issue #69: [ISSUE #68][feat]add Option for new producer (#68) URL: https://github.com/apache/pulsar-client-go/pull/69#issuecomment-539981117 > I think this is a good proposal to refactor the API interface. @xujianhai666 Can you fix this for `ClientOption` and `ConsumerOption` @wolfstudy done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332913435 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. + +### broker +Delayed Message Delivery is enabled by default on broker conf as below +``` +# Whether to enable the delayed delivery for messages. +# If disabled, messages will be immediately delivered and there will +# be no tracking overhead. +delayedDeliveryEnabled=true + +# Control the tick time for when retrying on delayed delivery, +# affecting the accuracy of the delivery time compared to the scheduled time. +# Default is 1 second. +delayedDeliveryTickTimeMillis=1000 +``` + +### producer +Here's an example for Delayed message producer in java: Review comment: @Anonymitaet done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332912987 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. + +### broker +Delayed Message Delivery is enabled by default on broker conf as below +``` +# Whether to enable the delayed delivery for messages. +# If disabled, messages will be immediately delivered and there will +# be no tracking overhead. +delayedDeliveryEnabled=true + +# Control the tick time for when retrying on delayed delivery, +# affecting the accuracy of the delivery time compared to the scheduled time. +# Default is 1 second. +delayedDeliveryTickTimeMillis=1000 +``` + +### producer Review comment: done @Anonymitaet This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch asf-site updated: Updated site at revision db15d32
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7a6b4d3 Updated site at revision db15d32
7a6b4d3 is described below
commit 7a6b4d35624097a75d65b1a05cf272ebf26a
Author: jenkins
AuthorDate: Wed Oct 9 09:24:54 2019 +
Updated site at revision db15d32
---
content/docs/en/next/client-libraries-java.html| 4 +-
.../docs/en/next/client-libraries-java/index.html | 4 +-
content/docs/fr/next/client-libraries-java.html| 14 +++---
.../docs/fr/next/client-libraries-java/index.html | 14 +++---
content/docs/ja/next/client-libraries-java.html| 4 +-
.../docs/ja/next/client-libraries-java/index.html | 4 +-
content/docs/zh-CN/next/client-libraries-java.html | 14 +++---
.../zh-CN/next/client-libraries-java/index.html| 14 +++---
content/swagger/2.5.0-SNAPSHOT/swagger.json| 56 +++---
.../swagger/2.5.0-SNAPSHOT/swaggerfunctions.json | 26 +-
content/swagger/master/swagger.json| 56 +++---
content/swagger/master/swaggerfunctions.json | 26 +-
12 files changed, 118 insertions(+), 118 deletions(-)
diff --git a/content/docs/en/next/client-libraries-java.html
b/content/docs/en/next/client-libraries-java.html
index 27f236a..3f09803 100644
--- a/content/docs/en/next/client-libraries-java.html
+++ b/content/docs/en/next/client-libraries-java.html
@@ -196,10 +196,10 @@ client.close();
Close operations can also be asynchronous:
producer.closeAsync()
- .thenRun(() - System.out.println("Producer
closed"));
+ .thenRun(() - System.out.println("Producer
closed"))
.exceptionally((ex) - {
System.err.println("Failed to close producer:
" + ex);
- return ex;
+ return null;
});
diff --git a/content/docs/en/next/client-libraries-java/index.html
b/content/docs/en/next/client-libraries-java/index.html
index 27f236a..3f09803 100644
--- a/content/docs/en/next/client-libraries-java/index.html
+++ b/content/docs/en/next/client-libraries-java/index.html
@@ -196,10 +196,10 @@ client.close();
Close operations can also be asynchronous:
producer.closeAsync()
- .thenRun(() - System.out.println("Producer
closed"));
+ .thenRun(() - System.out.println("Producer
closed"))
.exceptionally((ex) - {
System.err.println("Failed to close producer:
" + ex);
- return ex;
+ return null;
});
diff --git a/content/docs/fr/next/client-libraries-java.html
b/content/docs/fr/next/client-libraries-java.html
index 5887e35..ca3c59a 100644
--- a/content/docs/fr/next/client-libraries-java.html
+++ b/content/docs/fr/next/client-libraries-java.html
@@ -180,13 +180,13 @@ client.close();
Les opérations de fermeture peuvent également être asynchrones :
-```java
-producer.closeAsync()
- .thenRun(() - System.out.println("Producer closed"));
- .exceptionally((ex) - {
- System.err.println("Failed to close producer: " + ex);
- return ex;
- });
+```java
+producer.closeAsync()
+ .thenRun(()
- System.out.println("Producer
closed"))
+ .exceptionally((ex) - {
+ System.err.println("Failed to close producer:
" + ex);
+ return null;
+ });
Les opérations de fermeture peuvent également être asynchrones :
-```java
-producer.closeAsync()
- .thenRun(() - System.out.println("Producer closed"));
- .exceptionally((ex) - {
- System.err.println("Failed to close producer: " + ex);
- return ex;
- });
+```java
+producer.closeAsync()
+ .thenRun(()
- System.out.println("Producer
closed"))
+ .exceptionally((ex) - {
+ System.err.println("Failed to close producer:
" + ex);
+ return null;
+ });
java
producer.closeAsync()
- .thenRun(() - System.out.println("Producer closed"));
+ .thenRun(() - System.out.println("Producer closed"))
.exceptionally((ex) - {
System.err.println("Failed to close producer: " + ex);
- return ex;
+ return null;
});
java
producer.closeAsync()
- .thenRun(() - System.out.println("Producer closed"));
+ .thenRun(() - System.out.println("Producer closed"))
.exceptionally((ex) - {
System.err.println("Failed to close producer: " + ex);
- return ex;
+ return null;
});
关闭操作也可以是异步的：
-```java
-producer.closeAsync()
- .thenRun(() - System.out.println("Producer closed"));
- .exceptionally((ex) - {
- System.err.println("Failed to close producer: " + ex);
- return ex;
- });
+```java
+producer.closeAsync()
+ .thenRun(()
- System.out.println("Producer
closed"))
+ .exceptionally((ex) - {
+ System.err.println("Failed to close producer:
" + ex);
+ return null;
+ });
关闭操作也可以是异步的：
-```java
-producer.closeAsync()
- .thenRun(() - System.out.println("Producer closed"));
- .exceptionally((ex) -
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332912201 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. + +### broker +Delayed Message Delivery is enabled by default on broker conf as below +``` +# Whether to enable the delayed delivery for messages. +# If disabled, messages will be immediately delivered and there will +# be no tracking overhead. +delayedDeliveryEnabled=true + +# Control the tick time for when retrying on delayed delivery, Review comment: @Anonymitaet done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332911640 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. + +### broker +Delayed Message Delivery is enabled by default on broker conf as below +``` +# Whether to enable the delayed delivery for messages. +# If disabled, messages will be immediately delivered and there will +# be no tracking overhead. Review comment: done @Anonymitaet This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332911387 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. + +### broker +Delayed Message Delivery is enabled by default on broker conf as below +``` +# Whether to enable the delayed delivery for messages. +# If disabled, messages will be immediately delivered and there will Review comment: @Anonymitaet done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332910988 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. + +### broker +Delayed Message Delivery is enabled by default on broker conf as below Review comment: @Anonymitaet done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332910309 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. + +### broker Review comment: done @Anonymitaet This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332910041 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: Review comment: done @Anonymitaet This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332905715 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. +The diagram below illustrates concept: + + + +broker save message without any check. when consumer consume message, if message is delayed type, message will add to DelayedDeliveryTracker. subscription will check and get timeout messages from DelayedDeliveryTracker. Review comment: @Anonymitaet done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332904033 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. Review comment: @Anonymitaet done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on a change in pull request #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#discussion_r332902612 ## File path: site2/docs/concepts-messaging.md ## @@ -411,4 +411,32 @@ Message deduplication makes Pulsar an ideal messaging system to be used in conju > More in-depth information can be found in [this > post](https://streaml.io/blog/pulsar-effectively-once/) on the [Streamlio > blog](https://streaml.io/blog) +## Delayed Message Delivery +Delayed Message Delivery enables you to consume new message later, instead of immediately as normal. In this mechanism, messages is stored in memory after publish to broker, after specific delayed time passed, the message will be delivered to consumer. Review comment: @codelipenghui done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5232: Upgrade dependencies for security fixes
aahmed-se commented on issue #5232: Upgrade dependencies for security fixes URL: https://github.com/apache/pulsar/pull/5232#issuecomment-539901113 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ChangWinde closed issue #5283: JAAS login in broker failed
ChangWinde closed issue #5283: JAAS login in broker failed URL: https://github.com/apache/pulsar/issues/5283 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] boncheff closed pull request #5247: Update schema-get-started.md
boncheff closed pull request #5247: Update schema-get-started.md URL: https://github.com/apache/pulsar/pull/5247 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se merged pull request #5330: Added basic metadata store API and initial ZK implementation
aahmed-se merged pull request #5330: Added basic metadata store API and initial ZK implementation URL: https://github.com/apache/pulsar/pull/5330 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (136048c -> db15d32)
This is an automated email from the ASF dual-hosted git repository.
aahmed pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 136048c If cursor is not durable, close dispatcher when all consumers
are removed from subscription (#5340)
add db15d32 Added basic metadata store API and initial ZK implementation
(#5330)
No new revisions were added by this update.
Summary of changes:
pom.xml| 1 +
.../api-java => pulsar-metadata}/pom.xml | 31 ++-
.../org/apache/pulsar/metadata/api/GetResult.java | 16 +-
.../apache/pulsar/metadata/api/MetadataStore.java | 111 +
.../pulsar/metadata/api/MetadataStoreConfig.java | 23 +-
.../metadata/api/MetadataStoreException.java | 36 +--
.../pulsar/metadata/api/MetadataStoreFactory.java | 33 +--
.../java/org/apache/pulsar/metadata/api/Stat.java | 22 +-
.../metadata/impl/zookeeper/ZKMetadataStore.java | 251 +
.../apache/pulsar/metadata/MetadataStoreTest.java | 250
.../org/apache/pulsar/metadata/TestZKServer.java | 119 ++
11 files changed, 815 insertions(+), 78 deletions(-)
copy {pulsar-functions/api-java => pulsar-metadata}/pom.xml (67%)
copy
pulsar-transaction/coordinator/src/main/java/org/apache/pulsar/transaction/coordinator/TransactionCoordinatorID.java
=> pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/GetResult.java
(73%)
create mode 100644
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/MetadataStore.java
copy
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/DeadLetterPolicy.java
=>
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/MetadataStoreConfig.java
(63%)
copy
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/PulsarServerException.java
=>
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/MetadataStoreException.java
(62%)
copy pulsar-client/src/main/java/org/apache/pulsar/client/util/MathUtils.java
=>
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/MetadataStoreFactory.java
(53%)
copy
pulsar-broker/src/main/java/org/apache/pulsar/broker/protocol/ProtocolHandlerMetadata.java
=> pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/Stat.java (71%)
create mode 100644
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/zookeeper/ZKMetadataStore.java
create mode 100644
pulsar-metadata/src/test/java/org/apache/pulsar/metadata/MetadataStoreTest.java
create mode 100644
pulsar-metadata/src/test/java/org/apache/pulsar/metadata/TestZKServer.java
[GitHub] [pulsar] jiazhai merged pull request #5340: [pulsar-broker] If cursor is not durable, close dispatcher when all consumers are removed from subscription
jiazhai merged pull request #5340: [pulsar-broker] If cursor is not durable, close dispatcher when all consumers are removed from subscription URL: https://github.com/apache/pulsar/pull/5340 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (5426c5a -> 136048c)
This is an automated email from the ASF dual-hosted git repository. zhaijia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 5426c5a Avoid exiting on OOM in uni tests (#5343) add 136048c If cursor is not durable, close dispatcher when all consumers are removed from subscription (#5340) No new revisions were added by this update. Summary of changes: .../service/persistent/PersistentSubscription.java | 17 ++- .../pulsar/broker/service/PersistentTopicTest.java | 24 ++ 2 files changed, 40 insertions(+), 1 deletion(-)
[GitHub] [pulsar] jiazhai commented on issue #5342: Use simple GC settings when running integration tests
jiazhai commented on issue #5342: Use simple GC settings when running integration tests URL: https://github.com/apache/pulsar/pull/5342#issuecomment-539881290 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5279: Support for python native logging from python wrapper
jiazhai commented on issue #5279: Support for python native logging from python wrapper URL: https://github.com/apache/pulsar/pull/5279#issuecomment-539880484 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-539876820 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
