[GitHub] [pulsar] sijie commented on issue #5761: [pulsar-client] Switch partition when a batch is full (in RoundRobinMessageRouter)
sijie commented on issue #5761: [pulsar-client] Switch partition when a batch is full (in RoundRobinMessageRouter) URL: https://github.com/apache/pulsar/pull/5761#issuecomment-559383342 I think to achieve "sticky" partitioner. we might have to move the buffering logic to PartitionedProducerImpl, otherwise it is reallly hard to align buffering logic with switching partition logic. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] nkurihar commented on issue #5758: Bump jackson libraries to 2.10.1
nkurihar commented on issue #5758: Bump jackson libraries to 2.10.1 URL: https://github.com/apache/pulsar/pull/5758#issuecomment-559393594 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] yjshen closed pull request #5761: [pulsar-client] Switch partition when a batch is full (in RoundRobinMessageRouter)
yjshen closed pull request #5761: [pulsar-client] Switch partition when a batch is full (in RoundRobinMessageRouter) URL: https://github.com/apache/pulsar/pull/5761 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] reugn opened a new pull request #5765: add Murmur3_32Hash private constructor
reugn opened a new pull request #5765: add Murmur3_32Hash private constructor URL: https://github.com/apache/pulsar/pull/5765 ### Motivation Add private constructor to Murmur3_32Hash Singleton. ### Modifications Private constructor added. ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client.
codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client. URL: https://github.com/apache/pulsar/pull/5680#issuecomment-559413827 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5748: Fix NPE and release already released ByteBuf in batch message container
codelipenghui commented on issue #5748: Fix NPE and release already released ByteBuf in batch message container URL: https://github.com/apache/pulsar/pull/5748#issuecomment-559414382 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5745: Add document for sticky consumer
codelipenghui commented on issue #5745: Add document for sticky consumer URL: https://github.com/apache/pulsar/pull/5745#issuecomment-559417694 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5765: add Murmur3_32Hash private constructor
codelipenghui commented on issue #5765: add Murmur3_32Hash private constructor URL: https://github.com/apache/pulsar/pull/5765#issuecomment-559421160 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5764: [#5753][Doc] Fix the dead link for TLS Authentication on Proxies
codelipenghui commented on issue #5764: [#5753][Doc] Fix the dead link for TLS Authentication on Proxies URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559421624 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5758: Bump jackson libraries to 2.10.1
codelipenghui commented on issue #5758: Bump jackson libraries to 2.10.1 URL: https://github.com/apache/pulsar/pull/5758#issuecomment-559421991 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rcollina commented on issue #2307: Provide a netcore client
rcollina commented on issue #2307: Provide a netcore client URL: https://github.com/apache/pulsar/issues/2307#issuecomment-559423071 Without an official, supported .NET Core client it becomes hard to suggest Pulsar versus e.g. Kafka. Would you please reconsider your stance on this matter? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5755: UnAckedMessages admin information not correct
jiazhai commented on issue #5755: UnAckedMessages admin information not correct URL: https://github.com/apache/pulsar/issues/5755#issuecomment-559429518 @Fomal-haut Thanks for reporting this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5762: Add the filesystem offload doc
codelipenghui commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r351687199 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -30,6 +33,14 @@ Pulsar uses multi-part objects to upload the segment data. It is possible that a We recommend you add a life cycle rule your bucket to expire incomplete multi-part upload after a day or two to avoid getting charged for incomplete uploads. +## The Filesystem offloading mechanism + +A topic data was stored with a lot of ledgers. Offload these ledgers into filesystem, we need a path for it. +The implementation of the Filesystem offload to generate the path for **tenant/namespace/topic/ledgerId**. Review comment: The path of the data is `basePath` + /tenant/namespace/domain/topic/ + ledgerId-UUID right? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client.
codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client. URL: https://github.com/apache/pulsar/pull/5680#issuecomment-559443759 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5719: Remove cursor while remove non-durable subscription
jiazhai commented on issue #5719: Remove cursor while remove non-durable subscription URL: https://github.com/apache/pulsar/pull/5719#issuecomment-559447387 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] annotated tag v2.4.0-candidate-3 updated (811dd45 -> 8d8b2fb)
This is an automated email from the ASF dual-hosted git repository. rxl pushed a change to annotated tag v2.4.0-candidate-3 in repository https://gitbox.apache.org/repos/asf/pulsar.git. *** WARNING: tag v2.4.0-candidate-3 was modified! *** from 811dd45 (commit) to 8d8b2fb (tag) tagging 811dd45e87600d7163c8f74b40d377831f64bbfb (commit) replaces v2.4.2-candidate-1 by xiaolong.ran on Thu Nov 28 19:09:27 2019 +0800 - Log - Release v2.4.0-candidate-3 -BEGIN PGP SIGNATURE- iQJDBAABCAAtFiEEHqpd6S06DEdJ6YRttJ34KK/JpEIFAl3fqucPHHJ4bEBhcGFj aGUub3JnAAoJELSd+CivyaRCyloP/RXpD4gUhhVLBn3pUV/XrsMj8/ju/MgunZUJ itQn+MKPUZN0D2yTeAbogJKuDIYjbiNYihBAHC8QtOtc5m1FdgQk2n4x+Bg23Wd+ 6cnYe8nz4C8OMLmpxjU3B2vmh/Y5jbLtCsL2x9jDiTJvATPRYJrloa2vouA+bQvc TWJYULsCXq5RnotYOvxnGV6QvNMfS3SZpcMqzo3ZEKmKHs/zOBRSBSX7upgIKq1K YgctXa5vY+XDrl6bcpvBHseiqB6h6io0IiraZPEYJL/QV2Rr3z2oF665sPjk92Rl nzUS8kkqSSv6pq7HJy3hAfP3291oyYC1blOoUbx0KzJID05zt518pW2aZD63PAlK AGfI6SLXnYFgq5bmYv2xDcZNYdBgUdW5ZXWVHHEofzlvloRDmJiayqEr9mAat5Zp vg5FZtkxCyDW0WK2qrhpfAkVV5Mo6tRv+aau45kcDtDNOejCzAFTmx3lPE73G1/q Mc/j1vhFObWpWlvAp1USINcq0ChwsJ80ChLeL3zbKBAPjhs0Oectxm+ijinzI/Mw dyyB8dhlIMx5FgHIuG+EDYyIcjukQnpI7KO58QUgL8qDYJ6jIr8iYbcRcSwuxYSj sT4iSkMGaFldbrSg4pZG+9Nx9I6HyJirtUlOnH7DRMeaTSvoKgzQNMghwma/1I8H ZeXM0NvR =cXG3 -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[pulsar] annotated tag v2.4.2-candidate-3 updated (811dd45 -> c917a7f)
This is an automated email from the ASF dual-hosted git repository. rxl pushed a change to annotated tag v2.4.2-candidate-3 in repository https://gitbox.apache.org/repos/asf/pulsar.git. *** WARNING: tag v2.4.2-candidate-3 was modified! *** from 811dd45 (commit) to c917a7f (tag) tagging 811dd45e87600d7163c8f74b40d377831f64bbfb (commit) replaces v2.4.2-candidate-1 by xiaolong.ran on Thu Nov 28 19:12:23 2019 +0800 - Log - Release v2.4.2-candidate-3 -BEGIN PGP SIGNATURE- iQJDBAABCAAtFiEEHqpd6S06DEdJ6YRttJ34KK/JpEIFAl3fq5cPHHJ4bEBhcGFj aGUub3JnAAoJELSd+CivyaRCXd4QAK5q2q0cQfdSV1Z+pWD20ugfc3bklZccl7gg LLhtoazws3DMtk83W9ZIgNgDAOSQgaEvysZJrKDEPxBMSI7D5++PSi0wMmNn3hgN 71hQh088Hk2xzjqZGydy62HB1d4noslXbaGfWNN1zGJwqmOw25knvHj9sD39fRTQ 3rqqUwchcw4AGJBReBCjFMDgm3zu2sWiY2Pb9W9Ux/ohDzhuR65J8vT/sfkNXXxj 80uigY90RB6fiPeYj5liM6HyLaE1e5U5uH+ErJ+PkrK2FLv5EonMA+IOlYyOc+O1 bXiYbqf/x8dNvJQ7dJtx+iZQ8Ifd3YaSMpzJiv3iGt86UJmcsft09TjBvXZSdoEl cvAXNCL3gqoJmtJZ+kf3Vj8S2IVKEZEcSvFLF9qPEa7A9nhcQJA4GuRo90MEXs4o fkd70qtObJqZ372igT07dLXAgc4dUUKeDtpRSC6PK3BZAC9E3z6qudEsp5wVh8bp yCT3aB8Dmv4lt4bKr5UsMHYdabLLKR06ejhZfWtNf84LAGi3vCIYKwD3TB1gbMnU s0O3+PCw4qW3i45DI6RZvubpmzydo+6LqGaAAfO1Nj6NGvWeplTA72S4RkTsdhOr u02ReIE3ztZ65T0neuzqSzIz8rnRXUQRj/gJWaGBj6MIKHWHxIQTtTt8FchXxXVU YEdqVDIX =7dwf -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[GitHub] [pulsar] reugn commented on a change in pull request #5638: Bump jackson-databind from 2.9.9.3 to 2.10.1
reugn commented on a change in pull request #5638: Bump jackson-databind from 2.9.9.3 to 2.10.1 URL: https://github.com/apache/pulsar/pull/5638#discussion_r351743315 ## File path: pom.xml ## @@ -164,7 +164,7 @@ flexible messaging model and an intuitive client API. 2.10.0 1.60 2.9.9 -2.9.9.3 +2.10.1 Review comment: duplicate #5758 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5762: Add the filesystem offload doc
congbobo184 commented on issue #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#issuecomment-559472289 I think clients should know the filesystem implementation use MapFile, then they can configure the MapFile's some configuration so I add the filesystem offload mechanism. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on a change in pull request #5762: Add the filesystem offload doc
congbobo184 commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r351749722 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -30,6 +33,14 @@ Pulsar uses multi-part objects to upload the segment data. It is possible that a We recommend you add a life cycle rule your bucket to expire incomplete multi-part upload after a day or two to avoid getting charged for incomplete uploads. +## The Filesystem offloading mechanism + +A topic data was stored with a lot of ledgers. Offload these ledgers into filesystem, we need a path for it. +The implementation of the Filesystem offload to generate the path for **tenant/namespace/topic/ledgerId**. Review comment: yes, it should be in this way. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] mickdelaney commented on issue #2307: Provide a netcore client
mickdelaney commented on issue #2307: Provide a netcore client URL: https://github.com/apache/pulsar/issues/2307#issuecomment-559489093 @rcollina this might also be viable: https://github.com/apache/pulsar/issues/271 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
svn commit: r36971 - /dev/pulsar/pulsar-2.4.2-candidate-1/DEB/
Author: rxl Date: Thu Nov 28 13:25:13 2019 New Revision: 36971 Log: Delete bin package of pulsar-2.4.2-candidate-1 Removed: dev/pulsar/pulsar-2.4.2-candidate-1/DEB/
svn commit: r36972 - /dev/pulsar/pulsar-2.4.2-candidate-1/
Author: rxl Date: Thu Nov 28 13:25:49 2019 New Revision: 36972 Log: Delete pulsar-2.4.2-candidate-1 Removed: dev/pulsar/pulsar-2.4.2-candidate-1/
svn commit: r36974 - /dev/pulsar/KEYS
Author: rxl Date: Thu Nov 28 13:26:53 2019 New Revision: 36974 Log: Delete pulsar-2.4.2-candidate-2 Removed: dev/pulsar/KEYS
svn commit: r36973 - /dev/pulsar/pulsar-2.4.2-candidate-2/
Author: rxl Date: Thu Nov 28 13:26:11 2019 New Revision: 36973 Log: Delete pulsar-2.4.2-candidate-2 Removed: dev/pulsar/pulsar-2.4.2-candidate-2/
[GitHub] [pulsar] jiazhai commented on issue #5719: Remove cursor while remove non-durable subscription
jiazhai commented on issue #5719: Remove cursor while remove non-durable subscription URL: https://github.com/apache/pulsar/pull/5719#issuecomment-559494294 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
svn commit: r36975 - /dev/pulsar/pulsar-manager/
Author: rxl Date: Thu Nov 28 13:27:33 2019 New Revision: 36975 Log: Delete pulsar-manager-0.1.0 Removed: dev/pulsar/pulsar-manager/
[GitHub] [pulsar] Jennifer88huang commented on issue #4752: [docs] Add Go SerDe option
Jennifer88huang commented on issue #4752: [docs] Add Go SerDe option URL: https://github.com/apache/pulsar/issues/4752#issuecomment-559496240 @sijie Got it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang closed issue #5614: Deprecate `brokerDeleteInactiveTopicsFrequencySeconds` or `brokerServicePurgeInactiveFrequencyInSeconds`
Jennifer88huang closed issue #5614: Deprecate `brokerDeleteInactiveTopicsFrequencySeconds` or `brokerServicePurgeInactiveFrequencyInSeconds` URL: https://github.com/apache/pulsar/issues/5614 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang merged pull request #5632: [Issue 5614] Deprecate brokerServicePurgeInactiveFrequencyInSeconds
Jennifer88huang merged pull request #5632: [Issue 5614] Deprecate brokerServicePurgeInactiveFrequencyInSeconds URL: https://github.com/apache/pulsar/pull/5632 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (f9929d7 -> da80606)
This is an automated email from the ASF dual-hosted git repository. hjf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from f9929d7 Bump athenz libraries to 1.8.38 (#5756) add da80606 [Issue 5614] Deprecate brokerServicePurgeInactiveFrequencyInSeconds (#5632) No new revisions were added by this update. Summary of changes: conf/broker.conf| 6 +++--- conf/standalone.conf| 5 ++--- deployment/terraform-ansible/templates/broker.conf | 6 +++--- .../org/apache/pulsar/broker/ServiceConfiguration.java | 5 - .../org/apache/pulsar/broker/service/BrokerService.java | 2 +- .../pulsar/broker/service/ReplicatorTestBase.java | 6 +++--- .../pulsar/client/api/NonPersistentTopicTest.java | 6 +++--- pulsar-client-cpp/test-conf/standalone-ssl.conf | 16 +--- pulsar-client-cpp/test-conf/standalone.conf | 17 +++-- pulsar-client-cpp/tests/authentication.conf | 10 ++ pulsar-client-cpp/tests/standalone.conf | 14 -- site2/docs/reference-configuration.md | 2 -- .../version-2.4.0/reference-configuration.md| 2 -- .../version-2.4.1/reference-configuration.md| 2 -- 14 files changed, 53 insertions(+), 46 deletions(-)
[GitHub] [pulsar-client-go] reugn opened a new pull request #108: SetWriteDeadline on connection
reugn opened a new pull request #108: SetWriteDeadline on connection URL: https://github.com/apache/pulsar-client-go/pull/108 Issue #107 ### Modifications SetWriteDeadline on connection. ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / GoDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] reugn commented on issue #88: in go mod 1.12 compile error
reugn commented on issue #88: in go mod 1.12 compile error URL: https://github.com/apache/pulsar-client-go/issues/88#issuecomment-559511151 @sunvooker try `export GO111MODULE=on` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5753: [Doc] link for TLS Authentication on Proxies is dead
Jennifer88huang commented on issue #5753: [Doc] link for TLS Authentication on Proxies is dead URL: https://github.com/apache/pulsar/issues/5753#issuecomment-559522611 @belinda-wong The following image will help you to resolve the issue.  You need to update it as "see ["Enable TLS Authentication on Proxies"](security-tls-authentication.md#enable-tls-authentication-on-proxies)". After you update, you can run website locally and verify the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang edited a comment on issue #5753: [Doc] link for TLS Authentication on Proxies is dead
Jennifer88huang edited a comment on issue #5753: [Doc] link for TLS Authentication on Proxies is dead URL: https://github.com/apache/pulsar/issues/5753#issuecomment-559522611 @belinda-wong The following image will help you to resolve the issue.  You need to update it as `see ["Enable TLS Authentication on Proxies"](security-tls-authentication.md#enable-tls-authentication-on-proxies)`. After you update, you can [run website locally](https://github.com/apache/pulsar/tree/master/site2#run-the-site-locally) and verify the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rcollina commented on issue #2307: Provide a netcore client
rcollina commented on issue #2307: Provide a netcore client URL: https://github.com/apache/pulsar/issues/2307#issuecomment-559528102 @mickdelaney thanks for the link This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5719: Remove cursor while remove non-durable subscription
jiazhai commented on issue #5719: Remove cursor while remove non-durable subscription URL: https://github.com/apache/pulsar/pull/5719#issuecomment-559530188 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5755: UnAckedMessages admin information not correct
jiazhai commented on issue #5755: UnAckedMessages admin information not correct URL: https://github.com/apache/pulsar/issues/5755#issuecomment-559532239 This issue contains a detailed re-produce steps. mark as help wanted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5759: Inactive non-persistent subscriptions not purged
jiazhai commented on issue #5759: Inactive non-persistent subscriptions not purged URL: https://github.com/apache/pulsar/issues/5759#issuecomment-559533780 Thanks for the detailed reports @wmccarley . This is a feature that need to fix for non-persistent topics. This should be a direct fix that need following #1352 have done, mark this as help wanted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-559534496 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client.
codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client. URL: https://github.com/apache/pulsar/pull/5680#issuecomment-559535486 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client.
codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client. URL: https://github.com/apache/pulsar/pull/5680#issuecomment-559535639 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5765: add Murmur3_32Hash private constructor
codelipenghui commented on issue #5765: add Murmur3_32Hash private constructor URL: https://github.com/apache/pulsar/pull/5765#issuecomment-559536503 run java tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5765: add Murmur3_32Hash private constructor
codelipenghui commented on issue #5765: add Murmur3_32Hash private constructor URL: https://github.com/apache/pulsar/pull/5765#issuecomment-559536758 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5763: Broker graceful shutdown when znode of loadbalance disappeared
jiazhai commented on issue #5763: Broker graceful shutdown when znode of loadbalance disappeared URL: https://github.com/apache/pulsar/issues/5763#issuecomment-559539489 Hi @codelipenghui Thanks for the reporting. Seems the fix is direct, need to "Catch the no node exception and create the disappeared znode" , right? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5760: IndexOutOfBoundsException in PerformanceProducer
jiazhai commented on issue #5760: IndexOutOfBoundsException in PerformanceProducer URL: https://github.com/apache/pulsar/issues/5760#issuecomment-559548151 hi @codelipenghui I could reproduce this in a short way. 1. start a standalone: `bin/pulsar standalone` 2. run pulsar-perf to produce message: `bin/pulsar-perf produce persistent://public/default/test-topic --num-producers 1 -r 2000` 3. kill pulsar standalone, and no touch pulsar-perf(let it still running) 4. after a while (5-10 minutes) restart standalone, then will find this issue in pulsar-perf. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai edited a comment on issue #5760: IndexOutOfBoundsException in PerformanceProducer
jiazhai edited a comment on issue #5760: IndexOutOfBoundsException in PerformanceProducer URL: https://github.com/apache/pulsar/issues/5760#issuecomment-559548151 hi @codelipenghui I could reproduce this in a short timee way. 1. start a standalone: `bin/pulsar standalone` 2. run pulsar-perf to produce message: `bin/pulsar-perf produce persistent://public/default/test-topic --num-producers 1 -r 2000` 3. kill pulsar standalone, and no touch pulsar-perf(let it still running) 4. after a while (5-10 minutes) restart standalone, then will find this issue in pulsar-perf. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai edited a comment on issue #5760: IndexOutOfBoundsException in PerformanceProducer
jiazhai edited a comment on issue #5760: IndexOutOfBoundsException in PerformanceProducer URL: https://github.com/apache/pulsar/issues/5760#issuecomment-559548151 hi @codelipenghui I could reproduce this in a short time way. 1. start a standalone: `bin/pulsar standalone` 2. run pulsar-perf to produce message: `bin/pulsar-perf produce persistent://public/default/test-topic --num-producers 1 -r 2000` 3. kill pulsar standalone, and no touch pulsar-perf(let it still running) 4. after a while (5-10 minutes) restart standalone, then will find this issue in pulsar-perf. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #271: Consider using gRPC as an externally exposed API
sijie commented on issue #271: Consider using gRPC as an externally exposed API URL: https://github.com/apache/pulsar/issues/271#issuecomment-559559488 @mickdelaney @TC-oKozlov thank you for your input. just to understand a bit more about the requirements, are you expecting a gRPC based proxy or pulsar broker protocol exposed in gPRC? This would lead into two different approaches. A gRPC based proxy means providing a much simpler protocol than the current broker protocol. So it is easy to have different language gRPC clients. But it will has its own limitations and drawbacks, such as another network hop, and some of the features might be hard to support and etc. Exposing pulsar broker protocol in gPRC can solve the problem in handling wire-level request & response encoding and decoding. However the challenge of implementing a Pulsar client is not about handling wire-level encoding and decoding. It is more about the logic within a Pulsar client, such as flow-control, topic lookup, error handling and etc. So we will still be facing the same challenges that current Pulsar client is facing. It is probably even worse than implementing language client wrapper using Pulsar c/c++ client, because implementing a language client wrapper is much simpler and less error prone than re-implementing flow-control, topic lookup and error handling in different languages. I would like to collect more requirements of gRPC to understand what is the right approach for solving the problem here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #2307: Provide a netcore client
sijie commented on issue #2307: Provide a netcore client URL: https://github.com/apache/pulsar/issues/2307#issuecomment-559564195 @rcollina Sure. I hear your concerns. Just to understand it more, if I understand correctly, Kafka doesn't provide an _official_ dotnet client in the apache repo. The dotnet client was maintained by confluent outside of the ASF. What is your thought about this? Is this model okay to you? The model that Pulsar is using is very similar as Kafka. The Pulsar core repo maintains two core implementations, one is Java implementation and the other one is c++ implementation (with a c library exposed). So the community can develop other language wrappers based on the c/c++ library. In this way, Pulsar core team can focus on the core features and ensure it implemented correctly and reliably, and encourage more people in the community helping with developing different language clients (or wrappers). Although I guess people doesn't like implementing wrappers over a c/c++ library, people want *native* clients, which ends up with implementing the protocols in different languages (go, dotnet, rust and etc). If the requirement is to have a language-native client for every different language, it is really hard for Pulsar team to maintain and it also limits the scalability of the development. Thus we encourage developing language-native clients in the community. There are a couple of options we can probably explore on providing _official_ clients. 1) implementing a dotnet wrapper over c/c++ library if it is a trivial enough to maintain. 2) setting up a *pulsar-ecosystem* repo and move some of the community implementations to be under its umbrella. similar as what https://flink.apache.org/ecosystem.html and https://flink-packages.org/ let me know what you think. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] cbornet commented on issue #271: Consider using gRPC as an externally exposed API
cbornet commented on issue #271: Consider using gRPC as an externally exposed API URL: https://github.com/apache/pulsar/issues/271#issuecomment-559566128 I think moving to gRPC for the Pulsar clients would have some benefits. For instance it already handles flow control and bi-directional streaming. Another interesting alternative could be RSocket which has some very nice features such as session resumption and message-level backpressure. In JAVA, it would be possible to have a fully reactive-streams API using these protocols. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] cbornet edited a comment on issue #271: Consider using gRPC as an externally exposed API
cbornet edited a comment on issue #271: Consider using gRPC as an externally exposed API URL: https://github.com/apache/pulsar/issues/271#issuecomment-559566128 I think moving to gRPC for the Pulsar clients would have some benefits. For instance it already handles flow control and bi-directional streaming. For those who want to write native clients, that's a layer less to develop. Another interesting alternative could be RSocket which has some very nice features such as session resumption and message-level backpressure. In JAVA, it would be possible to have a fully reactive-streams API using these protocols. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] mickdelaney commented on issue #271: Consider using gRPC as an externally exposed API
mickdelaney commented on issue #271: Consider using gRPC as an externally exposed API URL: https://github.com/apache/pulsar/issues/271#issuecomment-559566538 @sijie thanks for the detailed feedback. i was thinking of the former, my thinking being that it would atleast remove some of the concerns in maintaining the various language level clients. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5147: Deprecate getstatus subcommand for source command
sijie commented on issue #5147: Deprecate getstatus subcommand for source command URL: https://github.com/apache/pulsar/pull/5147#issuecomment-559606703 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5762: Add the filesystem offload doc
sijie commented on issue #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#issuecomment-559607110 while @congbobo184 and @codelipenghui are settling down the technical details, @Jennifer88huang and @Anonymitaet we should consider reorganizing the documentation for tiered storage. we should consider adding a section called "Storage" or "Tiered Storage" at the sidebar rather than putting everything into a cookbook. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5279: Support for python native logging from python wrapper
sijie commented on issue #5279: Support for python native logging from python wrapper URL: https://github.com/apache/pulsar/pull/5279#issuecomment-559610266 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (da80606 -> b23c732)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from da80606 [Issue 5614] Deprecate brokerServicePurgeInactiveFrequencyInSeconds (#5632) add b23c732 [Issue 3009][docs] Add presto sql REST APIs doc (#5376) No new revisions were added by this update. Summary of changes: site2/docs/sql-rest-api.md | 185 site2/website/sidebars.json | 3 +- 2 files changed, 187 insertions(+), 1 deletion(-) create mode 100644 site2/docs/sql-rest-api.md
[GitHub] [pulsar] sijie closed issue #3009: documentation request: document presto http endpoint
sijie closed issue #3009: documentation request: document presto http endpoint URL: https://github.com/apache/pulsar/issues/3009 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #5376: [Issue 3009][docs] Add presto sql REST APIs doc
sijie merged pull request #5376: [Issue 3009][docs] Add presto sql REST APIs doc URL: https://github.com/apache/pulsar/pull/5376 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5745: Add document for sticky consumer
sijie commented on issue #5745: Add document for sticky consumer URL: https://github.com/apache/pulsar/pull/5745#issuecomment-559610469 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Remove cursor while remove non-durable subscription (#5719)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2e30c08 Remove cursor while remove non-durable subscription (#5719)
2e30c08 is described below
commit 2e30c086b2461531c62164d09ea148928a0e3ae6
Author: lipenghui
AuthorDate: Fri Nov 29 06:12:49 2019 +0800
Remove cursor while remove non-durable subscription (#5719)
### Motivation
Remove cursor from cursors of managed ledger while remove non-durable
subscription. The data deletion is depends the mark delete position of all
cursors, if left a unused cursor in the cursors of managed ledger, data can't
be delete as expected.
### Modifications
Remove cursor while remove non-durable subscription
---
.../AbstractDispatcherSingleActiveConsumer.java| 4 +-
.../org/apache/pulsar/broker/service/Consumer.java | 12 +-
.../apache/pulsar/broker/service/Dispatcher.java | 6 ++-
.../apache/pulsar/broker/service/Subscription.java | 6 ++-
.../nonpersistent/NonPersistentDispatcher.java | 4 +-
.../NonPersistentDispatcherMultipleConsumers.java | 2 +-
.../nonpersistent/NonPersistentSubscription.java | 2 +-
.../PersistentDispatcherMultipleConsumers.java | 4 +-
.../service/persistent/PersistentSubscription.java | 14 ++-
.../org/apache/pulsar/client/impl/ReaderTest.java | 44 +++---
10 files changed, 79 insertions(+), 19 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
index 02fcea1..509c3e3 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
@@ -219,11 +219,11 @@ public abstract class
AbstractDispatcherSingleActiveConsumer extends AbstractBas
*
* @return
*/
-public synchronized CompletableFuture disconnectAllConsumers() {
+public synchronized CompletableFuture disconnectAllConsumers(boolean
isResetCursor) {
closeFuture = new CompletableFuture<>();
if (!consumers.isEmpty()) {
-consumers.forEach(Consumer::disconnect);
+consumers.forEach(consumer -> consumer.disconnect(isResetCursor));
cancelPendingRead();
} else {
// no consumer connected, complete disconnect immediately
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
index fa45eb4..8c0e1d5 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
@@ -293,15 +293,23 @@ public class Consumer {
* pending message acks
*/
public void close() throws BrokerServiceException {
-subscription.removeConsumer(this);
+close(false);
+}
+
+public void close(boolean isResetCursor) throws BrokerServiceException {
+subscription.removeConsumer(this, isResetCursor);
cnx.removedConsumer(this);
}
public void disconnect() {
+disconnect(false);
+}
+
+public void disconnect(boolean isResetCursor) {
log.info("Disconnecting consumer: {}", this);
cnx.closeConsumer(this);
try {
-close();
+close(isResetCursor);
} catch (BrokerServiceException e) {
log.warn("Consumer {} was already closed: {}", this,
e.getMessage(), e);
}
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Dispatcher.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Dispatcher.java
index 24118ae..cffa024 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Dispatcher.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Dispatcher.java
@@ -60,7 +60,11 @@ public interface Dispatcher {
*
* @return
*/
-CompletableFuture disconnectAllConsumers();
+CompletableFuture disconnectAllConsumers(boolean isResetCursor);
+
+default CompletableFuture disconnectAllConsumers() {
+return disconnectAllConsumers(false);
+}
void resetCloseFuture();
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Subscription.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Subscription.java
index c044fa5..18c7c49 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Subscription.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Subscription.java
@@ -37,7
[GitHub] [pulsar] sijie merged pull request #5719: Remove cursor while remove non-durable subscription
sijie merged pull request #5719: Remove cursor while remove non-durable subscription URL: https://github.com/apache/pulsar/pull/5719 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rcollina commented on issue #2307: Provide a netcore client
rcollina commented on issue #2307: Provide a netcore client URL: https://github.com/apache/pulsar/issues/2307#issuecomment-559612954 @sijie thank you for your thorough reply, much appreciated. Confluent is an official enough a name when it comes to evaluating Kafka’s pros and cons. I can’t honestly comment on which scenario would work best, whether to follow Kafka’s model regarding project ownership. What I know, given my current understanding of Pulsar’s featureset, it’s that it is closer to my requirements than Kafka’s. However, the project’s site does not even mention .NET Core. This unfortunately can turn potential users away. I’m aware offering a proper client is no easy undertaking, especially considering the introduction of “memory friendly” constructs like Span in the latest .NET iterations, but it’s essential to increase adoption. My humble opinion, of course. That being said, a gRPC endpoint + client package could be a viable solution, something to start with. Thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #5746: NPE occur when send an over size message.
sijie closed issue #5746: NPE occur when send an over size message. URL: https://github.com/apache/pulsar/issues/5746 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #5748: Fix NPE and release already released ByteBuf in batch message container
sijie merged pull request #5748: Fix NPE and release already released ByteBuf in batch message container URL: https://github.com/apache/pulsar/pull/5748 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Fix NPE when send a large message and don't release batchedMessageMetadataAndPayload when discard in batch message container. (#5748)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new ecf7792 Fix NPE when send a large message and don't release batchedMessageMetadataAndPayload when discard in batch message container. (#5748) ecf7792 is described below commit ecf779210c31d1051f9cd2c823a1a7d078cdfd75 Author: lipenghui AuthorDate: Fri Nov 29 06:38:23 2019 +0800 Fix NPE when send a large message and don't release batchedMessageMetadataAndPayload when discard in batch message container. (#5748) Fixes #5746 #5747 ### Motivation Fix NPE and release an already released ByteBuf when publish an oversize message. Here is error log: ``` io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1 at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-common-4.1.43.Final.jar:4.1.43.Final] at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[netty-common-4.1.43.Final.jar:4.1.43.Final] at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) ~[netty-buffer-4.1.43.Final.jar:4.1.43.Final] at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88) ~[netty-common-4.1.43.Final.jar:4.1.43.Final] at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:113) [netty-common-4.1.43.Final.jar:4.1.43.Final] at org.apache.pulsar.client.impl.BatchMessageKeyBasedContainer$KeyedBatch.discard(BatchMessageKeyBasedContainer.java:244) [classes/:?] at org.apache.pulsar.client.impl.BatchMessageKeyBasedContainer.createOpSendMsg(BatchMessageKeyBasedContainer.java:125) [classes/:?] at org.apache.pulsar.client.impl.BatchMessageKeyBasedContainer.createOpSendMsgs(BatchMessageKeyBasedContainer.java:145) [classes/:?] at org.apache.pulsar.client.impl.ProducerImpl.batchMessageAndSend(ProducerImpl.java:1426) [classes/:?] at org.apache.pulsar.client.impl.ProducerImpl.triggerFlush(ProducerImpl.java:1411) [classes/:?] at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:112) [classes/:?] at org.apache.pulsar.client.impl.TypedMessageBuilderImpl.send(TypedMessageBuilderImpl.java:89) [classes/:?] at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:63) [classes/:?] at org.apache.pulsar.broker.service.BatchMessageTest.testSendOverSizeMessage(BatchMessageTest.java:875) [test-classes/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201] at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) [testng-6.14.3.jar:?] at org.testng.internal.Invoker.invokeMethod(Invoker.java:583) [testng-6.14.3.jar:?] at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719) [testng-6.14.3.jar:?] at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989) [testng-6.14.3.jar:?] at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) [testng-6.14.3.jar:?] at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) [testng-6.14.3.jar:?] at org.testng.TestRunner.privateRun(TestRunner.java:648) [testng-6.14.3.jar:?] at org.testng.TestRunner.run(TestRunner.java:505) [testng-6.14.3.jar:?] at org.testng.SuiteRunner.runTest(SuiteRunner.java:455) [testng-6.14.3.jar:?] at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450) [testng-6.14.3.jar:?] at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415) [testng-6.14.3.jar:?] at org.testng.SuiteRunner.run(SuiteRunner.java:364) [testng-6.14.3.jar:?] at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) [testng-6.14.3.jar:?] at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84) [testng-6.14.3.jar:?] at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208) [testng-6.14.3.jar:?] at org.testng.TestNG.runSuitesLocally(TestNG.java:1137) [testng-6.14.3.jar:?] at org.testng.TestNG.runSuites(TestNG.java:1049) [testng-6.14.3.jar:?] at org.testng.TestNG.run(TestNG.java:1017) [testng-6.14.3.jar:?] at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73) [testng-plugin.jar:?] at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123) [testng-plugin.jar:?]
[GitHub] [pulsar] sijie opened a new pull request #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky
sijie opened a new pull request #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky URL: https://github.com/apache/pulsar/pull/5766 Master Issue: # *Motivation* ManagedLedgerFactoryChangeLedgerPathTest sets the zk session timeout to 10 ms. 10ms is too low for a zookeeper client to connect to the zookeeper server. *Modifications* Remove zkSessionTimeout and use the default This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zhaohaidao opened a new pull request #5767: Support batch authorization of partitioned topic
zhaohaidao opened a new pull request #5767: Support batch authorization of partitioned topic URL: https://github.com/apache/pulsar/pull/5767 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Add unit tests for this change and make sure that the change passes the CI checks *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5758: Bump jackson libraries to 2.10.1
massakam commented on issue #5758: Bump jackson libraries to 2.10.1 URL: https://github.com/apache/pulsar/pull/5758#issuecomment-559627897 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5762: Add the filesystem offload doc
Jennifer88huang commented on issue #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#issuecomment-559631006 @sijie Got it. Thanks for your reminder. We should consider reorganizing it. Currently, there are two "tiered storage" sections in sidebar under "Concepts and architecture" and "Cookbooks". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5741: [website]Add content of pulsar-manager in website(include release notes, download link)
tuteng commented on issue #5741: [website]Add content of pulsar-manager in website(include release notes, download link) URL: https://github.com/apache/pulsar/pull/5741#issuecomment-559632396 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie opened a new pull request #5768: [CLI] Add a broker tool for operations of a specific broker
sijie opened a new pull request #5768: [CLI] Add a broker tool for operations of a specific broker URL: https://github.com/apache/pulsar/pull/5768 *Motivation* Add a tool to collect load report on a specific broker. This is used for troubleshooting problems in a production cluster. *Modifications* Add a broker tool for operations of a specific broker. ``` broker-tool is used for operations on a specific broker Usage: broker-tool [flags] [commands] Commands: load-report Collect the load report of a specific broker helpDisplay help information about it Flags: -h, --help Display help information Use "broker-tool [command] --help" or "broker-tool help [command]" for more information about a command ``` ``` Collect the load report of a specific broker Usage: load-report [flags] Flags: -i, --interval-ms Interval to collect load report, in milliseconds -h, --help Display help information ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky
sijie commented on issue #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky URL: https://github.com/apache/pulsar/pull/5766#issuecomment-559632656 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5767: Support batch authorization of partitioned topic
sijie commented on issue #5767: Support batch authorization of partitioned topic URL: https://github.com/apache/pulsar/pull/5767#issuecomment-559632996 @zhaohaidao can you update the description of the pull request? so the reviewers can understand what is the motivation of this pull request? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5279: Support for python native logging from python wrapper
sijie commented on issue #5279: Support for python native logging from python wrapper URL: https://github.com/apache/pulsar/pull/5279#issuecomment-559633449 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5745: Add document for sticky consumer
codelipenghui commented on issue #5745: Add document for sticky consumer URL: https://github.com/apache/pulsar/pull/5745#issuecomment-559633716 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5660: [docs]Add old version api document
tuteng commented on issue #5660: [docs]Add old version api document URL: https://github.com/apache/pulsar/pull/5660#issuecomment-559634878 It seems to be my local language environment problem. I will regenerate these documents. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5738: [Pulsar Client Tools]Support generate documentation of pulsar admin cli automatically
tuteng commented on issue #5738: [Pulsar Client Tools]Support generate documentation of pulsar admin cli automatically URL: https://github.com/apache/pulsar/pull/5738#issuecomment-559637809 Thank you for your suggestion. The situation you mentioned is my initial implementation plan. However, when I tested it, I found that different command objects `jcommand` was used in the PulsarAdminTool and CmdBase. PulsarAdminTool: https://github.com/apache/pulsar/blob/master/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/PulsarAdminTool.java#L87 CmdBase: https://github.com/apache/pulsar/blob/master/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdBase.java#L38 All the child commands were inherited from the class `CmdBase`. In subclasses such as `CmdTopics`, we cannot get the `commandMap` object. Therefore, it may be necessary to pass down the `commandmap` object and the `jcommand` object of `PulsarAdminTool`. therefore, it is necessary to add some logic to the `PulsarAdmintool`. I am not sure if my understanding is correct. I hope to get your reply, but I will try this implementation as much as possible. @sijie This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker
tuteng commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker URL: https://github.com/apache/pulsar/pull/5768#issuecomment-559639959 I saw a `broker-stats` sub-command under the command `pulsar-admin` and a `load-report` sub-command under the command `broker-stats`. I also saw this sub-command here. is there any difference between the two, why not consider adding it under pulsar-admin? @sijie This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5741: [website]Add content of pulsar-manager in website(include release notes, download link)
tuteng commented on issue #5741: [website]Add content of pulsar-manager in website(include release notes, download link) URL: https://github.com/apache/pulsar/pull/5741#issuecomment-559640081 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5745: Add document for sticky consumer
codelipenghui commented on issue #5745: Add document for sticky consumer URL: https://github.com/apache/pulsar/pull/5745#issuecomment-559640226 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky
tuteng commented on issue #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky URL: https://github.com/apache/pulsar/pull/5766#issuecomment-559640236 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351963820 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -626,6 +660,44 @@ None In some data formats, for example, Avro, you can define fields with default values. Consequently, adding or removing a field with a default value is a fully compatible change. +### UNDEFINED + + + + + + + +Compatibility check strategy + + + + + +Note + + + + + + + + + +`UNDEFINED` + + + + + +The schema compatibility check strategy in order to Upgrade to 2.5.0 or above for compatibility with pulsar version below 2.5.0. + + + + + + + Review comment: Add schema verification here. I've re-written some sentences as well. ```suggestion ## Schema verification When a producer or a consumer tries to connect to a topic, a broker performs some checks to verify a schema. ### Producer When a producer tries to connect to a topic (suppose ignore the schema auto creation), a broker does the following checks: * Check if the schema carried by the producer exists in the schema registry or not. * If the schema is already registered, then the producer is connected to a broker and produce messages with that schema. * If the schema is not registered, then Pulsar verifies if the schema is allowed to be registered based on the configured compatibility check strategy. ### Consumer When a consumer tries to connect to a topic, a broker checks if a carried schema is compatible with a registered schema based on the configured schema compatibility check strategy. Compatibility check strategy Check logic `ALWAYS_COMPATIBLE` All pass `ALWAYS_INCOMPATIBLE` No pass `BACKWARD` Can read the last schema `BACKWARD_TRANSITIVE` Can read all schemas `FORWARD` Can read the last schema `FORWARD_TRANSITIVE` Can read the last schema `FULL` Can read the last schema `FULL_TRANSITIVE` Can read all schemas ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351958691 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -366,6 +366,40 @@ Any order + + + + +`UNDEFINED` + + + + + +The schema compatibility check strategy in order to Upgrade to 2.5.0 or above for compatibility with pulsar version below 2.5.0. + + + + + +NONE + + + + + +NONE + + Review comment: ```suggestion It depends ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351959083 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -626,6 +660,44 @@ None In some data formats, for example, Avro, you can define fields with default values. Consequently, adding or removing a field with a default value is a fully compatible change. +### UNDEFINED + + + + + + + +Compatibility check strategy + + + + + +Note Review comment: ```suggestion Description ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351963750 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -781,3 +853,168 @@ Consequently, you can upgrade the producers and consumers in **any order**. + +## Schema verification for producers Review comment: I've moved this part before `order of upgrading clients`, please delete lines 857-1017. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351958714 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -366,6 +366,40 @@ Any order + + + + +`UNDEFINED` + + + + + +The schema compatibility check strategy in order to Upgrade to 2.5.0 or above for compatibility with pulsar version below 2.5.0. + + + + + +NONE + + + + + +NONE + + + + + +NONE Review comment: ```suggestion It depends ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351958676 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -366,6 +366,40 @@ Any order + + + + +`UNDEFINED` + + + + + +The schema compatibility check strategy in order to Upgrade to 2.5.0 or above for compatibility with pulsar version below 2.5.0. + + + + + +NONE Review comment: ```suggestion It depends ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351958437 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -366,6 +366,40 @@ Any order + + + + +`UNDEFINED` + + + + + +The schema compatibility check strategy in order to Upgrade to 2.5.0 or above for compatibility with pulsar version below 2.5.0. Review comment: ```suggestion * If you upgrade a lower version to Pulsar 2.5.0, then it uses the `UNDEFINED` strategy, at this time, it actually uses the compatibility strategy of the topic. * If you use Pulsar 2.5.0 and a higher version, then it uses the `UNDEFINED` strategy, at this time, it actually uses the `FULL` compatibility strategy. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r351959172 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -626,6 +660,44 @@ None In some data formats, for example, Avro, you can define fields with default values. Consequently, adding or removing a field with a default value is a fully compatible change. +### UNDEFINED + + + + + + + +Compatibility check strategy + + + + + +Note + + + + + + + + + +`UNDEFINED` + + + + + +The schema compatibility check strategy in order to Upgrade to 2.5.0 or above for compatibility with pulsar version below 2.5.0. Review comment: ```suggestion * If you upgrade a lower version to Pulsar 2.5.0, then it uses the `UNDEFINED` strategy, at this time, it actually uses the compatibility strategy of the topic. * If you use Pulsar 2.5.0 and a higher version, then it uses the `UNDEFINED` strategy, at this time, it actually uses the `FULL` compatibility strategy. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker
sijie commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker URL: https://github.com/apache/pulsar/pull/5768#issuecomment-559643791 This is the tool runs on broker. It is used for troubleshooting why broker can’t collect right load report. The command in Pulsar-admin is used for querying the data reported to zookeeper. That’s the difference. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5738: [Pulsar Client Tools]Support generate documentation of pulsar admin cli automatically
sijie commented on issue #5738: [Pulsar Client Tools]Support generate documentation of pulsar admin cli automatically URL: https://github.com/apache/pulsar/pull/5738#issuecomment-559645674 > I found that different command objects jcommand was used in the PulsarAdminTool and CmdBase. The jcommand in PulsarrAdminTool is the root command.The jcommand in CmdBase is for a command group. > All the child commands were inherited from the class CmdBase. In subclasses such as CmdTopics, we cannot get the commandMap object. You can get the command groups from the commandMap of PulsarAdminTool. Then you can iterate the command group. For each command group, you can get its sub commandMap. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
svn commit: r36981 [2/2] - in /dev/pulsar/pulsar-2.4.2-candidate-3: ./ DEB/ RPMS/ RPMS/repodata/ connectors/
Added: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-jdbc-2.4.2.nar.asc == --- dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-jdbc-2.4.2.nar.asc (added) +++ dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-jdbc-2.4.2.nar.asc Fri Nov 29 03:18:59 2019 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEEHqpd6S06DEdJ6YRttJ34KK/JpEIFAl3f6CIACgkQtJ34KK/J +pEKZ1Q//atFldb2O0a/5RYM4nhxITBghgarwUFta4lIy6k5Ub8PAiOH4nRI2dJTj +W7EG/iFcHhAYD+IHOVD4H9DimYG4+41hpeZqvxR2R1mktG3hm2nHRdzoMJD3NVV0 +GRSLdkLy6VL22sjmiO0wU7lo8JM17PoheoFC6rLBZ0/7JwRKiGS3hspYlk+6uW2e +McCCjU+79mzye/99BhqhB4jS4R1FM2O7l9UGlf1XFknpcEg1aZsYckR7bqrgder+ +i7KgDn9Jm2V5emWy9lV9q+Z2lyR8i+2hHdmQbIcbbSVtqo3t3GKlQIrWiuHb9lvN +zY4Y1N/3LFMlYVUMw2SrUAbDiVI3Sa6jkJvdfDKt8JPB8TjdI6xuW07lOa8mCGG9 +k7UMFqD1w4rPm+jjg1GCgLm6wP+2NvUb3HMzuH1l/omZyFnQoRb8uX0KopoYhAeX +4B69Gx1A25jfNqS84vsohPW4Y6J3xFVSmYK3WXrZS1I6qRzBL+8m3zakg2XM119C +b/gqStP2EywrvyAZHqMqS9mJ1MzeJbvQjMnAvCWn6buC9hAcjR2xOoHYRVKyyqiw +teah6/JxXJiLwvHGQkm4hesFtJdmzKAdbIFoUPE/AZ5VlEbCaosyTX288PEnuBFr +epxXHUVAxMmA/xrGfhlbqJF5uiQy0sL8bEIz6W/FBDquHtog8h0= +=HFW5 +-END PGP SIGNATURE- Added: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-jdbc-2.4.2.nar.sha512 == --- dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-jdbc-2.4.2.nar.sha512 (added) +++ dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-jdbc-2.4.2.nar.sha512 Fri Nov 29 03:18:59 2019 @@ -0,0 +1 @@ +fb1ae2a42e4674bef84a5227b10c87e31871c1d8187dcee931419f4bffeb8b393e28f9ffa9e2751ba53c9445d4a8efecdba78cb233d19b78b0b070ec1b41a3fd ./connectors/pulsar-io-jdbc-2.4.2.nar Added: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar == Binary file - no diff available. Propchange: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar.asc == --- dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar.asc (added) +++ dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar.asc Fri Nov 29 03:18:59 2019 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEEHqpd6S06DEdJ6YRttJ34KK/JpEIFAl3f6CMACgkQtJ34KK/J +pEKboBAA4+Izop88SMvAVDlGux2a2imxlvawJ+/f0WOzsnmyHrBum8Z0fgcqrLUA +DNszhKfA2Cp3pZxm3WiSZZ1Iz7BSqAj5jisWf/HZ0w55LrbVcYlonkpUQHuLO/z+ +8b5I5weP8w1WiilZ0MFV6PeR2VRbnLTMg/qenCajc4Q8Vse7R1voOUJCd4aRvwbp +vV3Xwa73Uuk3eg/uei/5YsAKxwuF4ejBnPG0l9b+B6DURBllCtozysOsenU0LbcV +cJqtB7XKHu+CE1ljJ6sxB+RJCUn0dT/Fsos+1UtOthnWvy0DgksGmtdR2uuQRPap +s6o6Ozlghrzm2ASyM8T/iZj7YhamRPHTLgmCZtKXDAdh1i09V+5zcdXS02jaNWTG +IYZdMiVpF6DXfrz/Ib2gwyImLnJQ6GxczKD2hvLOS6akupqUffS4kII56T1HBvbY +0LTtUK/aMnfzEZk+vU/HOL0L5sp2tq2v2wRpDz+guRBjqPz4Te9KhDj6vNiQlq4Y +w+QE1R6xP29x7kWQTjORBZOjEfqoBnH/0UQBOJxs+U5Ckr4QgLjDHphTm3zW1kDP +zyx3yT7txYo2qv2+zkRuasonC1tyewikbaaTXY38bx8ezESdY4HfDYQM2JDeVYcf +epIJlvc/8wJ0qASIBNtZyAdbfrbL/R/Vc8RS+IZ9hrd1GupOlsk= +=LfUf +-END PGP SIGNATURE- Added: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar.sha512 == --- dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar.sha512 (added) +++ dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-2.4.2.nar.sha512 Fri Nov 29 03:18:59 2019 @@ -0,0 +1 @@ +bc1939ec51fcec38359a1473c258cbc7dc75f3d14b710f53b6faeb12b89dffb2af513cd3c21b77d052b8a0a88dc4051a0aa57a97b6be5fbf7ad5c455aabcc508 ./connectors/pulsar-io-kafka-2.4.2.nar Added: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-connect-adaptor-2.4.2.nar == Binary file - no diff available. Propchange: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-connect-adaptor-2.4.2.nar -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-connect-adaptor-2.4.2.nar.asc == --- dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-connect-adaptor-2.4.2.nar.asc (added) +++ dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-kafka-connect-adaptor-2.4.2.nar.asc Fri Nov 29 03:18:59 2019 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEEHqpd6S06DEdJ6YRttJ34KK/JpEIFAl3f6CsACgkQtJ34KK/J +pEI2nw//R5HFeOjrLDQQ96lrCdUCEOj+EhZF++zNo1L4xMkc529lPgoHujdiogW
svn commit: r36981 [1/2] - in /dev/pulsar/pulsar-2.4.2-candidate-3: ./ DEB/ RPMS/ RPMS/repodata/ connectors/
Author: rxl Date: Fri Nov 29 03:18:59 2019 New Revision: 36981 Log: Staging artifacts and signature for Pulsar release 2.4.2 Added: dev/pulsar/pulsar-2.4.2-candidate-3/ dev/pulsar/pulsar-2.4.2-candidate-3/DEB/ dev/pulsar/pulsar-2.4.2-candidate-3/DEB/Packages.gz (with props) dev/pulsar/pulsar-2.4.2-candidate-3/DEB/Packages.gz.asc dev/pulsar/pulsar-2.4.2-candidate-3/DEB/Packages.gz.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/ dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/apache-pulsar-client-debuginfo-2.4.2-1.x86_64.rpm (with props) dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/apache-pulsar-client-debuginfo-2.4.2-1.x86_64.rpm.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/apache-pulsar-client-debuginfo-2.4.2-1.x86_64.rpm.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/ dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/b63720768aaddd3a20fbdd6f629e19762d370943e1f55cd0dea30630e04b999b-primary.xml.gz (with props) dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/b63720768aaddd3a20fbdd6f629e19762d370943e1f55cd0dea30630e04b999b-primary.xml.gz.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/b63720768aaddd3a20fbdd6f629e19762d370943e1f55cd0dea30630e04b999b-primary.xml.gz.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/d2e33d37010272101e891b9c3cfcb8f145420ec7cd7996dc42d6b7d3f380bf0e-filelists.xml.gz (with props) dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/d2e33d37010272101e891b9c3cfcb8f145420ec7cd7996dc42d6b7d3f380bf0e-filelists.xml.gz.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/d2e33d37010272101e891b9c3cfcb8f145420ec7cd7996dc42d6b7d3f380bf0e-filelists.xml.gz.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/dc09fd722436618912dc98dd8a436c3dc230da405c08bf6d74515c5285c587c6-other.sqlite.bz2 (with props) dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/dc09fd722436618912dc98dd8a436c3dc230da405c08bf6d74515c5285c587c6-other.sqlite.bz2.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/dc09fd722436618912dc98dd8a436c3dc230da405c08bf6d74515c5285c587c6-other.sqlite.bz2.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/deff38ee75d0897f0efa30804c3cf25c88e4dbcedf27283702946aee91cef83d-filelists.sqlite.bz2 (with props) dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/deff38ee75d0897f0efa30804c3cf25c88e4dbcedf27283702946aee91cef83d-filelists.sqlite.bz2.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/deff38ee75d0897f0efa30804c3cf25c88e4dbcedf27283702946aee91cef83d-filelists.sqlite.bz2.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/e03a84e132004b733f0391396d58daa2897e0ed0c0249b7729e501abaa2691c1-primary.sqlite.bz2 (with props) dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/e03a84e132004b733f0391396d58daa2897e0ed0c0249b7729e501abaa2691c1-primary.sqlite.bz2.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/e03a84e132004b733f0391396d58daa2897e0ed0c0249b7729e501abaa2691c1-primary.sqlite.bz2.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/fd16df1037d96ed64bc9483e11d5427f29de2879697713291740a6eeb2c9f886-other.xml.gz (with props) dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/fd16df1037d96ed64bc9483e11d5427f29de2879697713291740a6eeb2c9f886-other.xml.gz.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/fd16df1037d96ed64bc9483e11d5427f29de2879697713291740a6eeb2c9f886-other.xml.gz.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/repomd.xml dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/repomd.xml.asc dev/pulsar/pulsar-2.4.2-candidate-3/RPMS/repodata/repomd.xml.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-2.4.2-bin.tar.gz (with props) dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-2.4.2-bin.tar.gz.asc dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-2.4.2-bin.tar.gz.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-2.4.2-src.tar.gz (with props) dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-2.4.2-src.tar.gz.asc dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-2.4.2-src.tar.gz.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-offloaders-2.4.2-bin.tar.gz (with props) dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-offloaders-2.4.2-bin.tar.gz.asc dev/pulsar/pulsar-2.4.2-candidate-3/apache-pulsar-offloaders-2.4.2-bin.tar.gz.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/connectors/ dev/pulsar/pulsar-2.4.2-candidate-3/connectors/LICENSE dev/pulsar/pulsar-2.4.2-candidate-3/connectors/README dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-aerospike-2.4.2.nar (with props) dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-aerospike-2.4.2.nar.asc dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-aerospike-2.4.2.nar.sha512 dev/pulsar/pulsar-2.4.2-candidate-3/connectors/pulsar-io-canal-2.4.2.nar (with props) dev/pulsar/pulsar-2.4.2-candidate-3/conn
[GitHub] [pulsar] sijie commented on issue #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky
sijie commented on issue #5766: [tests] ManagedLedgerFactoryChangeLedgerPathTest is flaky URL: https://github.com/apache/pulsar/pull/5766#issuecomment-559646482 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc
congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-559647039 @sijie ``` bin/pulsar-admin namespaces set-schema-autoupdate-strategy --compatibility tenant/namespace bin/pulsar-admin namespaces get-schema-autoupdate-strategy --compatibility tenant/namespace ``` I think this two command should not be provided to users. It's easy to confuse users. ``` bin/pulsar-admin namespaces set-schema-compatibility-strategy --compatibility tenant/namespace bin/pulsar-admin namespaces get-schema-compatibility-strategy --compatibility tenant/namespace bin/pulsar-admin namespaces get-is-allow-auto-update-schema --disable tenant/namespace ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 edited a comment on issue #5757: [doc]Modify schema compatibility check doc
congbobo184 edited a comment on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-559647039 @sijie ``` bin/pulsar-admin namespaces set-schema-autoupdate-strategy --compatibility tenant/namespace bin/pulsar-admin namespaces get-schema-autoupdate-strategy --compatibility tenant/namespace ``` I think this two command should not be provided to users. It's easy to confuse users following commands : ``` bin/pulsar-admin namespaces set-schema-compatibility-strategy --compatibility tenant/namespace bin/pulsar-admin namespaces get-schema-compatibility-strategy --compatibility tenant/namespace bin/pulsar-admin namespaces get-is-allow-auto-update-schema --disable tenant/namespace ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5757: [doc]Modify schema compatibility check doc
sijie commented on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-559648008 @congbobo184 if those two commands are deprecated, you can add `hidden` flag to hide them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc
congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-559649018 This seems like a good way This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 edited a comment on issue #5757: [doc]Modify schema compatibility check doc
congbobo184 edited a comment on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-559649018 This seems like a good way. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
