[GitHub] [pulsar] Anonymitaet commented on issue #6044: [doc] Add documentation about running Pulsar Functions using LocalRunner
Anonymitaet commented on issue #6044: URL: https://github.com/apache/pulsar/issues/6044#issuecomment-659892269 Confirmed w/ @wolfstudy, creating a function can not be achieved via the `LocalRunner` class. Doc is available at [here](https://streamnative.io/docs/latest/process/pulsar-functions/deploy/manage). Will copy to Pulsar later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Anonymitaet closed issue #6044: [doc] Add documentation about running Pulsar Functions using LocalRunner
Anonymitaet closed issue #6044: URL: https://github.com/apache/pulsar/issues/6044 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] jiazhai opened a new pull request #7575: fix website build for format error in sidebars.json
jiazhai opened a new pull request #7575: URL: https://github.com/apache/pulsar/pull/7575 Fixes #7536 ### Motivation website build error happens: https://builds.apache.org/job/pulsar-website-build/1070/console ``` $ docusaurus-write-translations internal/modules/cjs/loader.js:800 throw err; ^ SyntaxError: /pulsar/site2/website/sidebars.json: Unexpected string in JSON at position 2167 at JSON.parse () at Object.Module._extensions..json (internal/modules/cjs/loader.js:797:27) at Module.load (internal/modules/cjs/loader.js:653:32) at tryModuleLoad (internal/modules/cjs/loader.js:593:12) at Function.Module._load (internal/modules/cjs/loader.js:585:3) at Module.require (internal/modules/cjs/loader.js:692:17) at require (internal/modules/cjs/helpers.js:25:18) at Object. (/pulsar/site2/website/node_modules/docusaurus/lib/server/readMetadata.js:44:17) at Module._compile (internal/modules/cjs/loader.js:778:30) at Module._compile (/pulsar/site2/website/node_modules/pirates/lib/index.js:99:24) error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. ``` this is caused by the format error in sidebars.json brings in #7462 ### Modifications Fix the error. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se commented on pull request #7570: Add maven profile to skip broker unit tests
aahmed-se commented on pull request #7570: URL: https://github.com/apache/pulsar/pull/7570#issuecomment-659879325 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] wenjun8888 opened a new issue #7574: Set jvm options to each components
wenjun opened a new issue #7574:
URL: https://github.com/apache/pulsar/issues/7574
sir, i have an questions to ask.
when i setting jvm options in pulsar_env.sh
for example:
```
# Extra options to be passed to the jvm
PULSAR_MEM=${PULSAR_MEM:-"-Xms4g -Xmx4g -XX:MaxDirectMemorySize=4g"}
```
Is that means 4g memory allocated for each components.
In my case, a node with 8G memory and there are zookeeper\bookie\broker on
it.
So i think 8G is not enough...
Please tell me if i have the wrong idea, or can i set jvm options for singel
components.
Thank you very much!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] aahmed-se commented on pull request #7570: Add maven profile to skip broker unit tests
aahmed-se commented on pull request #7570: URL: https://github.com/apache/pulsar/pull/7570#issuecomment-659872542 //pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se closed pull request #5461: (WIP) Add Unified native clients build step
aahmed-se closed pull request #5461: URL: https://github.com/apache/pulsar/pull/5461 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se closed pull request #5931: Add github workflow to generate python wheel artifacts
aahmed-se closed pull request #5931: URL: https://github.com/apache/pulsar/pull/5931 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se closed pull request #6180: [Do not merge]Alpine test github actions
aahmed-se closed pull request #6180: URL: https://github.com/apache/pulsar/pull/6180 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se closed pull request #6289: Add script to build alpine pulsar client image
aahmed-se closed pull request #6289: URL: https://github.com/apache/pulsar/pull/6289 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] srkukarni opened a new pull request #7573: Allow null consume in BatchPushSource
srkukarni opened a new pull request #7573: URL: https://github.com/apache/pulsar/pull/7573 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation BatchSource records allow sources to return a null record to indicate that the batch is done. For BatchPushSource, since we are using LinkedBlockingQueue, user's cannot simply pass a null value. Thus we need a special mechanism to indicate the end of a batch. ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se closed pull request #6196: Add docker build image for alpine python client
aahmed-se closed pull request #6196: URL: https://github.com/apache/pulsar/pull/6196 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se closed pull request #5863: (WIP) Test 1 Do no merge
aahmed-se closed pull request #5863: URL: https://github.com/apache/pulsar/pull/5863 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se closed pull request #7549: Reduce modules build in core module profile
aahmed-se closed pull request #7549: URL: https://github.com/apache/pulsar/pull/7549 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #7572: Kevin Wilson changes to fix segment crashes in pulsar
merlimat commented on a change in pull request #7572:
URL: https://github.com/apache/pulsar/pull/7572#discussion_r456227158
##
File path: pulsar-client-cpp/lib/ClientConnection.cc
##
@@ -289,13 +286,14 @@ void
ClientConnection::startConsumerStatsTimer(std::vector consumerSta
consumerStatsRequests.push_back(it->first);
}
-DeadlineTimerPtr timer = consumerStatsRequestTimer_;
-if (timer) {
-timer->expires_from_now(operationsTimeout_);
-
timer->async_wait(std::bind(&ClientConnection::handleConsumerStatsTimeout,
shared_from_this(),
-std::placeholders::_1,
consumerStatsRequests));
+// If the close operation has reset the consumerStatsRequestTimer_ then
the use_count will be zero
+// Check if we have a timer still before we set the request timer to pop
again.
+if (consumerStatsRequestTimer_.use_count() > 0) {
Review comment:
```suggestion
if (consumerStatsRequestTimer_) {
```
##
File path: pulsar-client-cpp/lib/ClientConnection.cc
##
@@ -1344,8 +1324,12 @@ void ClientConnection::handleKeepAliveTimeout() {
havePendingPingRequest_ = true;
sendCommand(Commands::newPing());
-
keepAliveTimer_->expires_from_now(boost::posix_time::seconds(KeepAliveIntervalInSeconds));
-
keepAliveTimer_->async_wait(std::bind(&ClientConnection::handleKeepAliveTimeout,
shared_from_this()));
+// If the close operation has already called the
keepAliveTimer_.reset() then the use_count will be zero And we do not attempt
+// to dereference the pointer.
+if (keepAliveTimer_.use_count() > 0) {
Review comment:
```suggestion
if (keepAliveTimer_) {
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #7558: [Issue 7557][pulsar-zookeeper-utils] Replace the use of Paths.get(...).getParent() for a zookeeper path in…
merlimat merged pull request #7558: URL: https://github.com/apache/pulsar/pull/7558 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (2a0cb69 -> 3db03be)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 2a0cb69 Add more logging to the auth operations on failure (#7567) add 3db03be Replace the use of Paths.get(...).getParent() for a zookeeper path in ZooKeeperCache (#7558) No new revisions were added by this update. Summary of changes: .../java/org/apache/pulsar/zookeeper/ZkUtils.java | 42 .../apache/pulsar/zookeeper/ZooKeeperCache.java| 3 +- .../pulsar/zookeeper/ZkUtilsParentPathTest.java| 112 + 3 files changed, 155 insertions(+), 2 deletions(-) create mode 100644 pulsar-zookeeper-utils/src/test/java/org/apache/pulsar/zookeeper/ZkUtilsParentPathTest.java
[GitHub] [pulsar] merlimat closed issue #7557: InvalidPathException on shutting down pulsar when running embedded on windows
merlimat closed issue #7557: URL: https://github.com/apache/pulsar/issues/7557 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] irairdon opened a new pull request #7572: Kevin Wilson changes to fix segment crashes in pulsar
irairdon opened a new pull request #7572: URL: https://github.com/apache/pulsar/pull/7572 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on issue #683: ReflectionFactory is internal proprietary API and may be removed
vcottagiri commented on issue #683: URL: https://github.com/apache/pulsar/issues/683#issuecomment-659854825 @sijie - Let me look into this one This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang opened a new issue #7571: Does puslar-admin
aloyszhang opened a new issue #7571: URL: https://github.com/apache/pulsar/issues/7571 **Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] **Describe the solution you'd like** A clear and concise description of what you want to happen. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** Add any other context or screenshots about the feature request here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se opened a new pull request #7570: Add profile to skip broker unit tests
aahmed-se opened a new pull request #7570: URL: https://github.com/apache/pulsar/pull/7570 Allow a profile to skip pulsar-broker tests specifically. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-go] LvBay opened a new pull request #326: Fix producer block
LvBay opened a new pull request #326: URL: https://github.com/apache/pulsar-client-go/pull/326 When the client creates a producer and a consumer with the same id, then the consumer processing handleCloseConsumer() will cause the producer with the same id to be unable to handle handleSendReceipt() and handleCloseProducer() in the future. The most direct impact is the Send() method Will always block. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] bleacha870721 commented on issue #7569: Failed to build bookie server
bleacha870721 commented on issue #7569: URL: https://github.com/apache/pulsar/issues/7569#issuecomment-659833611 pulsar version: 2.5.2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] bleacha870721 opened a new issue #7569: Failed to build bookie server
bleacha870721 opened a new issue #7569: URL: https://github.com/apache/pulsar/issues/7569 **Describe the bug** Bookie pod CrashLoopBackOff and cannot restart **To Reproduce** Steps to reproduce the behavior:  04:04:26.014 [main] ERROR org.apache.bookkeeper.server.Main - Failed to build bookie server org.apache.bookkeeper.bookie.BookieException$InvalidCookieException: Cookie [4 bookieHost: "10.233.106.106:3181" journalDir: "data/bookkeeper/journal" ledgerDirs: "1\tdata/bookkeeper/ledgers" instanceId: "7940c93b-5da8-4fa7-941e-d254d678fb1c" ] is not matching with [4 bookieHost: "10.233.107.121:3181" journalDir: "data/bookkeeper/journal" ledgerDirs: "1\tdata/bookkeeper/ledgers" instanceId: "7940c93b-5da8-4fa7-941e-d254d678fb1c" ] at org.apache.bookkeeper.bookie.Cookie.verifyInternal(Cookie.java:136) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.bookie.Cookie.verify(Cookie.java:147) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.bookie.Bookie.verifyAndGetMissingDirs(Bookie.java:369) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.bookie.Bookie.checkEnvironmentWithStorageExpansion(Bookie.java:432) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.bookie.Bookie.checkEnvironment(Bookie.java:250) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.bookie.Bookie.(Bookie.java:688) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:136) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.proto.BookieServer.(BookieServer.java:105) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.server.service.BookieService.(BookieService.java:41) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:301) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.server.Main.doMain(Main.java:221) [org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] at org.apache.bookkeeper.server.Main.main(Main.java:203) [org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] **Expected behavior** A clear and concise description of what you expected to happen. **Desktop (please complete the following information):** - OS: No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04 LTS Release: 18.04 Codename:bionic This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7563: 2.6 AuthorizationProvider breaks compatibly with earlier versions
sijie commented on issue #7563: URL: https://github.com/apache/pulsar/issues/7563#issuecomment-659831829 @cckellogg I chatted with @srkukarni. I think the default implementation was pushed down the zk implementation. We can put some of the implementations at the interface level. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #7453: Remove bouncy castle shaded module to avoid bring error of verifySingleJar
sijie commented on pull request #7453: URL: https://github.com/apache/pulsar/pull/7453#issuecomment-659830917 ping @jiazhai This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #7516: Add support for partitioned tables
sijie commented on pull request #7516: URL: https://github.com/apache/pulsar/pull/7516#issuecomment-659830427 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #7555: Modify the log level of functions and move the java_instance_log4j2.xml file from the jar package to the conf directory
sijie commented on pull request #7555: URL: https://github.com/apache/pulsar/pull/7555#issuecomment-659829114 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie merged pull request #7567: Add more logging to the auth operations on failure
sijie merged pull request #7567: URL: https://github.com/apache/pulsar/pull/7567 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] cckellogg commented on issue #7563: 2.6 AuthorizationProvider breaks compatibly with earlier versions
cckellogg commented on issue #7563: URL: https://github.com/apache/pulsar/issues/7563#issuecomment-659827828 @sijie I'm not sure if the builtin AuthorizationProvider is broken in 2.6 I have not done much testing with that one. Custom AuthorizationProviders built against version less than 2.6 are broken when running with version 2.6. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (d2b866b -> 2a0cb69)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from d2b866b Update Jersey to 2.31 (#7515) add 2a0cb69 Add more logging to the auth operations on failure (#7567) No new revisions were added by this update. Summary of changes: .../broker/authorization/AuthorizationProvider.java | 15 --- 1 file changed, 12 insertions(+), 3 deletions(-)
[GitHub] [pulsar] Jennifer88huang commented on issue #5399: Create documentation for KubernetesSecretsTokenAuthProvider
Jennifer88huang commented on issue #5399: URL: https://github.com/apache/pulsar/issues/5399#issuecomment-659825872 @addisonj Have you met any difficulties in fixing the issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7568: I can't access websocket(embedded) via pulsar-proxy
sijie commented on issue #7568: URL: https://github.com/apache/pulsar/issues/7568#issuecomment-659816202 @darrenx999 pulsar proxy doesn't support WebSocket protocol. Ideally, you can just add WebSocket as a separate container in the proxy pod and expose the port through proxy loadbalancer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] darrenx999 commented on issue #7568: I can't access websocket(embedded) via pulsar-proxy
darrenx999 commented on issue #7568: URL: https://github.com/apache/pulsar/issues/7568#issuecomment-659813777 @sijie thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] darrenx999 closed issue #7568: I can't access websocket(embedded) via pulsar-proxy
darrenx999 closed issue #7568: URL: https://github.com/apache/pulsar/issues/7568 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7455: Key_Shared consumers with different subscriptions get messages out-of-order in 2.6.0.
sijie commented on issue #7455: URL: https://github.com/apache/pulsar/issues/7455#issuecomment-659812829 @feeblefakie we realized the importance of this issue. @codelipenghui is working on this. please give us some time on fixing this issue since we also have other tasks ongoing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] srkukarni merged pull request #7515: Update Jersey to 2.31
srkukarni merged pull request #7515: URL: https://github.com/apache/pulsar/pull/7515 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (f8b2a23 -> d2b866b)
This is an automated email from the ASF dual-hosted git repository. sanjeevrk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from f8b2a23 Handle NotAllowed Exception at the client side. (#7430) add d2b866b Update Jersey to 2.31 (#7515) No new revisions were added by this update. Summary of changes: distribution/server/licenses/LICENSE-EPL-2.0.txt | 277 +++ distribution/server/src/assemble/LICENSE.bin.txt | 44 ++-- pom.xml | 4 +- pulsar-client-admin-shaded/pom.xml | 2 +- pulsar-client-all/pom.xml| 2 +- pulsar-sql/presto-distribution/LICENSE | 6 +- 6 files changed, 310 insertions(+), 25 deletions(-) create mode 100644 distribution/server/licenses/LICENSE-EPL-2.0.txt
[GitHub] [pulsar] darrenx999 commented on issue #7568: I can't access websocket(embedded) via pulsar-proxy
darrenx999 commented on issue #7568: URL: https://github.com/apache/pulsar/issues/7568#issuecomment-659811657 > @darrenx999 Current helm chart doesn't support WebSocket. Can you create an issue at https://github.com/apache/pulsar-helm-chart? I see,so I modify the broker config to support ws, but i don't know the workflow of the proxy,why can't access ws via proxy. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7477: [python-client 2.6.0] ContextualVersionConflict grpcio dependency conflict
sijie commented on issue #7477: URL: https://github.com/apache/pulsar/issues/7477#issuecomment-659811336 Gotcha. Can you also raise an issue at https://github.com/apache/bookkeeper? So we can relax the requirement in bookkeeper side. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7568: I can't access websocket(embedded) via pulsar-proxy
sijie commented on issue #7568: URL: https://github.com/apache/pulsar/issues/7568#issuecomment-659810685 @darrenx999 Current helm chart doesn't support WebSocket. Can you create an issue at https://github.com/apache/pulsar-helm-chart? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] darrenx999 opened a new issue #7568: I can't access websocket(embedded) via pulsar-proxy
darrenx999 opened a new issue #7568:
URL: https://github.com/apache/pulsar/issues/7568
Expected behavior
Tell us what should happen
Actual behavior
1.I setup a k8s pulsar cluster with values-minikube.yaml.
2.set webSocketServiceEnabled=true on broker, after restart pod , I can
access websocket in pod.
3. I try to access websocket via pulsar-proxy, failed.
error info:
websocket._exceptions.WebSocketBadStatusException: Handshake status 404 Not
Found
Steps to reproduce
1. helm install --values example/values-minikube.yaml
2. TOPIC =
'ws://localhost:8080/ws/v2/producer/persistent/public/default/message'
connect ws on broker pod,successful.
3. TOPIC = 'ws://localhost/ws/v2/producer/persistent/public/default/message'
connect ws on proxy,failed
System configuration
**Pulsar version**: 2.6.0
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] wolfstudy commented on issue #5608: [Doc] Document that --py can be a zipfile
wolfstudy commented on issue #5608: URL: https://github.com/apache/pulsar/issues/5608#issuecomment-659805940 > I don't see any mention of zip files at those links - is this awaiting docs to be republished? I can't tell, because there's no link to the relevant git commit either. Hello @candlerb When we want to submit a python function, we can use --py to specify that we are about to submit a python function. --py specifies the user python function, which is an executable file, not a zip file, so what do you mean by zip file? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7559: Health Check Topic growth
sijie commented on issue #7559: URL: https://github.com/apache/pulsar/issues/7559#issuecomment-659785942 @irairdon Can you also give us the broker log and heap dump if possible? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7556: Possibility to break a state in MultiTopicsConsumerImpl class during subscribe new topics
sijie commented on issue #7556: URL: https://github.com/apache/pulsar/issues/7556#issuecomment-659784025 @git-enzo Are you interested in sending a pull request for this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7554: Messages that have already been acked are redelivered when upgrading Pulsar version
sijie commented on issue #7554: URL: https://github.com/apache/pulsar/issues/7554#issuecomment-659783624 @codelipenghui Can you check this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7547: Queue declaration parameters for rabbitMQ source connector configurable
sijie commented on issue #7547: URL: https://github.com/apache/pulsar/issues/7547#issuecomment-659778546 @joofio Are you interested in contributing an enhancement to this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7546: Create non partition topic err V2.6.0
sijie commented on issue #7546: URL: https://github.com/apache/pulsar/issues/7546#issuecomment-659778011 @lhui1991 Did you see any errors at the broker side? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7545: A swallowed exception in exception handling during subscribe new topic
sijie commented on issue #7545: URL: https://github.com/apache/pulsar/issues/7545#issuecomment-659776802 @git-enzo Are you interested in contributing a bug fix? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7543: Dead code in MultiTopicsConsumerImpl class - occurs if topic name validation failed during subscribing
sijie commented on issue #7543: URL: https://github.com/apache/pulsar/issues/7543#issuecomment-659776378 @git-enzo Are you interested in contributing a bug fix? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] wolfstudy commented on pull request #6752: Support different docker images in Kubernetes runtime of Pulsar Functions
wolfstudy commented on pull request #6752: URL: https://github.com/apache/pulsar/pull/6752#issuecomment-659774728 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7538: [Question] How to generate sequenceId for commit marker in partition topic
sijie commented on issue #7538: URL: https://github.com/apache/pulsar/issues/7538#issuecomment-659773048 Since the commit is initiated by TC and done by TB. TB first writes a `committing` record to the transaction buffer. You can use the message-id of the `committing` record to create a long number. This can be used as the sequence id to write the commit marker to the data partition. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #7430: Handle NotAllowed Exception at the client side.
codelipenghui merged pull request #7430: URL: https://github.com/apache/pulsar/pull/7430 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (73f3632 -> f8b2a23)
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 73f3632 Add modules to core modules (#7566) add f8b2a23 Handle NotAllowed Exception at the client side. (#7430) No new revisions were added by this update. Summary of changes: .../broker/service/BrokerServiceException.java | 2 ++ .../client/api/KeySharedSubscriptionTest.java | 2 +- .../pulsar/client/api/PulsarClientException.java | 22 ++ pulsar-client-cpp/include/pulsar/Result.h | 1 + pulsar-client-cpp/lib/ClientConnection.cc | 3 +++ pulsar-client-cpp/lib/Result.cc| 3 +++ .../org/apache/pulsar/client/impl/ClientCnx.java | 2 ++ .../apache/pulsar/common/api/proto/PulsarApi.java | 3 +++ pulsar-common/src/main/proto/PulsarApi.proto | 1 + 9 files changed, 38 insertions(+), 1 deletion(-)
[GitHub] [pulsar] Huanli-Meng commented on issue #7415: [Document & Website][Pulsar 2.5.0]--schema registry should be removed from the concept section
Huanli-Meng commented on issue #7415: URL: https://github.com/apache/pulsar/issues/7415#issuecomment-659765002 @Jennifer88huang , how about this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7563: 2.6 AuthorizationProvider breaks compatibly with earlier versions
sijie commented on issue #7563: URL: https://github.com/apache/pulsar/issues/7563#issuecomment-659742552 @cckellogg just to clarify this question first - does this change break the existing authorization implementation (model)? This would help me understand the severity of this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] srkukarni opened a new pull request #7567: Add more logging to the auth operations on failure
srkukarni opened a new pull request #7567: URL: https://github.com/apache/pulsar/pull/7567 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] merlimat commented on issue #7565: PulsarClient-cpp segv after close() method is called.
merlimat commented on issue #7565: URL: https://github.com/apache/pulsar/issues/7565#issuecomment-659741695 @klwilson227 Do you have a simple way to reproduce this (even if it involves putting `sleep()` calls in the `close()`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7533: [Issue 7492][pulsar-broker] Cleanup configuration process when using PulsarStandaloneBuilder
vcottagiri commented on a change in pull request #7533:
URL: https://github.com/apache/pulsar/pull/7533#discussion_r456137308
##
File path:
pulsar-broker/src/test/java/org/apache/pulsar/PulsarStandaloneBuilderTest.java
##
@@ -0,0 +1,62 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.OutputStreamWriter;
+import java.io.PrintWriter;
+
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.assertEquals;
+import static org.testng.Assert.assertNotNull;
+
+public class PulsarStandaloneBuilderTest {
+@Test
+public void testBuildCreatesConfigObjects() throws FileNotFoundException {
+File testConfigFile = new File("tmp." + System.currentTimeMillis() +
".properties");
+if (testConfigFile.exists()) {
+testConfigFile.delete();
+}
+PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(new
FileOutputStream(testConfigFile)));
+printWriter.println("managedLedgerDefaultEnsembleSize=1");
+printWriter.println("managedLedgerDefaultWriteQuorum=1");
+printWriter.println("managedLedgerDefaultAckQuorum=1");
+printWriter.println("journalMaxSizeMB=1024");
+printWriter.println("journalPreAllocSizeMB=8");
+printWriter.println("journalWriteBufferSizeKB=32");
+printWriter.close();
+testConfigFile.deleteOnExit();
+
+final PulsarStandalone pulsarStandalone =
PulsarStandaloneBuilder.instance()
+.withConfigFile(testConfigFile.getAbsolutePath())
+.build();
Review comment:
use try-with-resources block here so that pulsarStandalone.close() is
called for cleanup.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7533: [Issue 7492][pulsar-broker] Cleanup configuration process when using PulsarStandaloneBuilder
vcottagiri commented on a change in pull request #7533:
URL: https://github.com/apache/pulsar/pull/7533#discussion_r456135678
##
File path:
pulsar-broker/src/test/java/org/apache/pulsar/PulsarStandaloneBuilderTest.java
##
@@ -0,0 +1,62 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.OutputStreamWriter;
+import java.io.PrintWriter;
+
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.assertEquals;
+import static org.testng.Assert.assertNotNull;
+
+public class PulsarStandaloneBuilderTest {
+@Test
+public void testBuildCreatesConfigObjects() throws FileNotFoundException {
+File testConfigFile = new File("tmp." + System.currentTimeMillis() +
".properties");
+if (testConfigFile.exists()) {
+testConfigFile.delete();
+}
+PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(new
FileOutputStream(testConfigFile)));
Review comment:
This line could use the [PrintWriter constructor with the File
argument](https://docs.oracle.com/javase/8/docs/api/java/io/PrintWriter.html#PrintWriter-java.io.File-)
, avoiding the decorator boilerplate.
`PrintWriter printWriter = new PrintWriter(testConfigFile)`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7533: [Issue 7492][pulsar-broker] Cleanup configuration process when using PulsarStandaloneBuilder
vcottagiri commented on a change in pull request #7533:
URL: https://github.com/apache/pulsar/pull/7533#discussion_r456132432
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneBuilder.java
##
@@ -97,10 +108,31 @@ public PulsarStandaloneBuilder
withAdvertisedAddress(String advertisedAddress) {
return this;
}
-public PulsarStandalone build() {
-ServiceConfiguration config = new ServiceConfiguration();
-config.setClusterName("standalone");
-pulsarStandalone.setConfig(config);
+public PulsarStandalone build()
+{
+// Change IOException and ConfigurationException into a
RuntimeException, because if the
+// config file isn't readable, there is nothing a caller can do, so
don't bother with
+// a checked exception that needs to be catched
+try {
+// By reading the configuration file here, the user can modify the
configurations before
+// calling PulsarStandalone.start()
+ServerConfiguration bkServerConf = new ServerConfiguration();
+bkServerConf.loadConf(new
File(pulsarStandalone.getConfigFile()).toURI().toURL());
+pulsarStandalone.setBkServerConfig(bkServerConf);
+
+pulsarStandalone.setConfig(PulsarConfigurationLoader.create(
+new FileInputStream(pulsarStandalone.getConfigFile()),
ServiceConfiguration.class));
+}
+catch (IOException | ConfigurationException e) {
Review comment:
Move catch to previous line after close braces to follow formatting
conventions
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7533: [Issue 7492][pulsar-broker] Cleanup configuration process when using PulsarStandaloneBuilder
vcottagiri commented on a change in pull request #7533:
URL: https://github.com/apache/pulsar/pull/7533#discussion_r456132195
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneBuilder.java
##
@@ -97,10 +108,31 @@ public PulsarStandaloneBuilder
withAdvertisedAddress(String advertisedAddress) {
return this;
}
-public PulsarStandalone build() {
-ServiceConfiguration config = new ServiceConfiguration();
-config.setClusterName("standalone");
-pulsarStandalone.setConfig(config);
+public PulsarStandalone build()
+{
Review comment:
Move opening braces to previous line to follow formatting conventions
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7430: Handle NotAllowed Exception at the client side.
codelipenghui commented on pull request #7430: URL: https://github.com/apache/pulsar/pull/7430#issuecomment-659729134 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #7489: Every function/connector exception results in a new metric
devinbost commented on issue #7489: URL: https://github.com/apache/pulsar/issues/7489#issuecomment-659728513 @vzhikserg Looks great! We might want to take a look at the Python implementation too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se merged pull request #7566: Add modules to core modules
aahmed-se merged pull request #7566: URL: https://github.com/apache/pulsar/pull/7566 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (19d27ec -> 73f3632)
This is an automated email from the ASF dual-hosted git repository. aahmed pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 19d27ec Fixed flaky test AdminApiTest2.testBacklogNoDelayed (#7562) add 73f3632 Add modules to core modules (#7566) No new revisions were added by this update. Summary of changes: pom.xml | 5 + 1 file changed, 5 insertions(+)
[GitHub] [pulsar] devinbost commented on pull request #7539: [Issue 7489] Remove timestamp from metrics
devinbost commented on pull request #7539: URL: https://github.com/apache/pulsar/pull/7539#issuecomment-659727667 @vzhikserg This is great! Thanks for working on this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aahmed-se opened a new pull request #7566: Add modules to core modules
aahmed-se opened a new pull request #7566: URL: https://github.com/apache/pulsar/pull/7566 Add additional modules to core modules This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] klwilson227 opened a new issue #7565: PulsarClient-cpp segv after close() method is called.
klwilson227 opened a new issue #7565: URL: https://github.com/apache/pulsar/issues/7565 **Describe the bug** After the execution of the close method, a segmentation violation can be seen from time to time when either the keepAliveTimer or the consumerStatsTimer pops during or soon after the close operation. The segv shows the following stack trace: Signal: [0x000b] PID: [0x00048988] PC: [0x7f02e615c1b6] FP: [0x7f02c40d0180] SIGSEGV: SEGV_MAPERR SI_ADDR : [0x] (_ZN6Basics9Backtrace11DoBacktraceEiiPvS1_+0x8f1) [0x4074ca1] (_ZN6Basics20GlobalSignalHandlers14logFatalSignalEiPvS1_+0xd2) [0x40c91b2] (_ZN6BasicsL26sigHandler_withinATryCatchEiP7siginfoPv+0x2c2) [0x40ca6b2] (_ZN6BasicsL10sigHandlerEiP7siginfoPv+0xe) [0x40ca77e] (_L_unlock_13+0x34) [0x7f03dc81e5d0] (_ZN5boost4asio20basic_deadline_timerINS_10posix_time5ptimeENS0_11time_traitsIS3_EENS0_22deadline_timer_serviceIS3_S5_EEE16expires_from_nowERKNS2_13time_durationE+0x16) [0x7f02e615c1b6] (UNKNOWN) [0x7f02e61d3768] (UNKNOWN) [0x7f02e61d3de3] (_ZN5boost4asio3ssl7contextD1Ev+0xdc1) [0x7f02e61e57b1] (_ZN5boost4asio6detail15task_io_service3runERNS_6system10error_codeE+0x311) [0x7f02e6121331] (_ZNSt14_Function_base13_Base_managerISt5_BindIFSt7_Mem_fnIMN6pulsar12ConsumerImplEFvNS3_6ResultENS3_9MessageIdESt8functionIFvS5_S6_St10shared_ptrIS4_ESt12_PlaceholderILi1EESF_ILi2EES9_EEE10_M_managerERSt9_Any_dataRKSL_St18_Manager_operation+0xaf6) [0x7f02e611e2c6] (_ZNSt6thread5_ImplISt12_Bind_simpleIFSt5_BindIFSt7_Mem_fnIMN6pulsar15ExecutorServiceEFvSt10shared_ptrIN5boost4asio10io_servicePS5_SA_EEvEEE6_M_runEv+0x52) [0x7f02e6122772] (_ZNSt11this_thread11__sleep_forENSt6chrono8durationIlSt5ratioILl1ELl1NS1_IlS2_ILl1ELl10+0x1c0) [0x7f03dcde2070] (start_thread+0xc5) [0x7f03dc816dd5] (clone+0x6d) [0x7f03dc13402d] Steps to reproduce the behavior: This problem is difficult to reproduce without tweaking the source base. As this exception is based on timing. To stress the environment add a short sleep to the ClientConnection::close() just after the mutex has been granted. This will give a long window for all of the timers to pop and align on the mutex if necessary. **Expected behavior** Even with the sleep in place the timers should pop and be handled. So that they do not cause a segmentation violation and bring down the application for which the client is embedded with. **Screenshots** N/A **Desktop (please complete the following information):** - OS: Centos **Additional context** Add any other context about the problem here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] fmiguelez opened a new issue #7564: Schema properties are not updated if schema definition does not change
fmiguelez opened a new issue #7564:
URL: https://github.com/apache/pulsar/issues/7564
**Describe the bug**
The only available operation to update properties of a schema (`SchemaInfo`)
is to use `createSchema()` method of `PulsarAdmin`. The problem with that is
that it will not update the provided properties if schema definition does not
change.
The only way the admin API provides to update the properties associated with
an schema is to delete it first and then create it from scratch. This can be
dangerous because if schema creation fails aftewards we will be left with an
unreadable topic (missing schema) and also different consumers using different
schema versions may also face issues (when we delete an schema all versions are
deleted).
**To Reproduce**
```
PulsarAdmin client = createClient();
String topicUri = getTopicUri();
SchemaInfo info = client.schemas().getSchemaInfo(topicUri);
/*
* We add a new property
*/
info.getProperties().put("newProp", "newValue");
PostSchemaPayload sp = new PostSchemaPayload();
sp.setType(info.getType().toString());
sp.setSchema(info.getSchemaDefinition());
sp.setProperties(info.getProperties());
/*
* This should update schema properties
*/
client.schemas().createSchema(topicUri, sp);
/*
* This object does not contain "newProp"
*/
info = client.schemas().getSchemaInfo(topicUri);
```
**Expected behavior**
Method to create/update schema should update properties even if schema
definition does not change.
**Screenshots**
**Desktop (please complete the following information):**
- Windows with Docker Desktop
**Additional context**
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] seeday closed pull request #6679: [][io-rabbitmq] Add `passive` to io-rabbitmq config options
seeday closed pull request #6679: URL: https://github.com/apache/pulsar/pull/6679 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (eb2ca72 -> 19d27ec)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from eb2ca72 Fix typo in ci-cpp.yaml (#7561) add 19d27ec Fixed flaky test AdminApiTest2.testBacklogNoDelayed (#7562) No new revisions were added by this update. Summary of changes: .../java/org/apache/pulsar/broker/admin/AdminApiTest2.java | 13 - 1 file changed, 12 insertions(+), 1 deletion(-)
[GitHub] [pulsar] merlimat merged pull request #7562: Fixed flaky test AdminApiTest2.testBacklogNoDelayed
merlimat merged pull request #7562: URL: https://github.com/apache/pulsar/pull/7562 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] cckellogg opened a new issue #7563: 2.6 AuthorizationProvider breaks compatibly with earlier versions
cckellogg opened a new issue #7563:
URL: https://github.com/apache/pulsar/issues/7563
**Describe the bug**
The 2.6 AuthorizationProvider breaks compatibility and behavior with custom
Authorization plugins written against versions lower than 2.6.
**Expected behavior**
For example:
The Authorization for most of the Namespace operations called this method in
AuthorizationProvider ```isTenantAdmin```
Now the Authorization for the Namespace operations call ends up calling this
method ```allowNamespaceOperationAsync``` and the default implementation is:
```
return FutureUtil.failedFuture(
new IllegalStateException("NamespaceOperation is not supported
by the Authorization provider you are using."));
```
This completely breaks any custom authorization plugin written against 2.5
if a user upgrades to version 2.6. When upgrading to 2.6 there should be no
change in behavior for any custom authorization plugin written against an
earlier version.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #7561: [ci] Fix typo in ci-cpp.yaml
merlimat merged pull request #7561: URL: https://github.com/apache/pulsar/pull/7561 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (5fd3b0a -> eb2ca72)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 5fd3b0a [docs] Improve configuration details of functions worker (#7534) add eb2ca72 Fix typo in ci-cpp.yaml (#7561) No new revisions were added by this update. Summary of changes: .github/workflows/ci-cpp.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [pulsar] merlimat opened a new pull request #7562: Fixed flaky test AdminApiTest2.testBacklogNoDelayed
merlimat opened a new pull request #7562: URL: https://github.com/apache/pulsar/pull/7562 ### Motivation The `AdminApiTest2.testBacklogNoDelayed` is checking the non-delayed messages backlog immediately after publish, but that stats is only updated when the messages are inserted in the delay tracker. This, though, happens when messages are being dispatched so there's no guarantee that the stats is already updated when the `send()` returns. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7550: Allow to filter packages and groups in tests more easily
vcottagiri commented on a change in pull request #7550: URL: https://github.com/apache/pulsar/pull/7550#discussion_r455952809 ## File path: README.md ## @@ -113,6 +113,13 @@ $ cd module-name (e.g: pulsar-client) $ mvn test -Dtest=unit-test-name (e.g: ConsumerBuilderImplTest) ``` +Run Selected Test packages: + +```bash +$ cd module-name (e.g: pulsar-broker) +$ mvn test -pl module-name -Dinclude=org/apache/pulsar/**/*.java Review comment: @aahmed-se - In the docs, since there is a cd into 'module-name' do we still need the -pl option? If we are using the -pl option, using the -am ([--also-make](https://books.sonatype.com/mvnref-book/reference/_using_advanced_reactor_options.html#_specifying_a_subset_of_projects)) would be advisable This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] vcottagiri opened a new pull request #7561: [ci] Fix typo in ci-cpp.yaml
vcottagiri opened a new pull request #7561: URL: https://github.com/apache/pulsar/pull/7561 Fixed 'pacakge' typo This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7548: [Issue 5922] - Cleanup poms to use ${project.groupId} and ${project.version} consistently
vcottagiri commented on a change in pull request #7548:
URL: https://github.com/apache/pulsar/pull/7548#discussion_r455932914
##
File path: examples/flink/pom.xml
##
@@ -74,6 +72,12 @@
${project.version}
+
Review comment:
fixed by #7560
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] vcottagiri opened a new pull request #7560: [Issue #5922] fix - remove buildtools dependency in flink pom as its already inherited
vcottagiri opened a new pull request #7560: URL: https://github.com/apache/pulsar/pull/7560 Addresses [comment in PR for #5922](https://github.com/apache/pulsar/pull/7548/files#r455897202) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7548: [Issue 5922] - Cleanup poms to use ${project.groupId} and ${project.version} consistently
vcottagiri commented on a change in pull request #7548:
URL: https://github.com/apache/pulsar/pull/7548#discussion_r455924931
##
File path: examples/flink/pom.xml
##
@@ -74,6 +72,12 @@
${project.version}
+
Review comment:
Yes, my bad
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] irairdon opened a new issue #7559: Health Check Topic growth
irairdon opened a new issue #7559:
URL: https://github.com/apache/pulsar/issues/7559
**Describe the bug**
If a node restart occurs the health check topic backlog will start to grow
and appears to never stop growing
**To Reproduce**
Steps to reproduce the behavior:
1. Install pulsar 2.5.2 latest with Node restart fix
2. Start producer and consumer on any number of topics
3. restart a node in the kubernetes cluster
4. Observe backlog growth of one of the broker health check topics
**Expected behavior**
Health check is internal topics so I would expect pulsar to manage the
backlog of internal topic without extra configuration
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
**Additional context**
Here is the stats and stats-internal for the topic growing.
pulsar@itomdipulsar-bastion-7587f6cf5c-klwhx:/pulsar> bin/pulsar-admin
topics stats persistent://pulsar/itomdipulsar/172.16.89.64:8080/healthcheck
{
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"msgRateOut" : 0.2056635827046,
"msgThroughputOut" : 20.200057202185317,
"bytesInCounter" : 4045630,
"msgInCounter" : 40148,
"bytesOutCounter" : 748410,
"msgOutCounter" : 7410,
"averageMsgSize" : 0.0,
"storageSize" : 751137,
"backlogSize" : 748410,
"publishers" : [ ],
"subscriptions" : {
"reader-f7da745e7f" : {
"msgRateOut" : 0.2056635827046,
"msgThroughputOut" : 20.200057202185317,
"bytesOutCounter" : 748410,
"msgOutCounter" : 7410,
"msgRateRedeliver" : 0.0,
"msgBacklog" : 7410,
"msgBacklogNoDelayed" : 7410,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"unackedMessages" : 0,
"type" : "Exclusive",
"activeConsumerName" : "9d046",
"msgRateExpired" : 0.0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 1594914513257,
"lastConsumedTimestamp" : 1594916563294,
"lastAckedTimestamp" : 0,
"consumers" : [ {
"msgRateOut" : 0.2056635827046,
"msgThroughputOut" : 20.200057202185317,
"bytesOutCounter" : 748410,
"msgOutCounter" : 7410,
"msgRateRedeliver" : 0.0,
"consumerName" : "9d046",
"availablePermits" : 590,
"unackedMessages" : 0,
"blockedConsumerOnUnackedMsgs" : false,
"lastAckedTimestamp" : 0,
"lastConsumedTimestamp" : 1594916563294,
"metadata" : { },
"clientVersion" : "2.5.2",
"connectedSince" : "2020-07-16T06:05:20.258Z",
"address" : "/172.16.89.64:38240"
} ],
"isReplicated" : false
}
},
"replication" : { },
"deduplicationStatus" : "Disabled"
}
pulsar@itomdipulsar-bastion-7587f6cf5c-klwhx:/pulsar> bin/pulsar-admin
topics stats-internal
persistent://pulsar/itomdipulsar/172.16.89.64:8080/healthcheck
{
"entriesAddedCounter" : 36163,
"numberOfEntries" : 7446,
"totalSize" : 752046,
"currentLedgerEntries" : 9,
"currentLedgerSize" : 909,
"lastLedgerCreatedTimestamp" : "2020-07-16T16:22:48.297Z",
"lastLedgerCreationFailureTimestamp" : "2020-07-16T06:05:20.045Z",
"waitingCursorsCount" : 2,
"pendingAddEntriesCount" : 0,
"lastConfirmedEntry" : "73889:8",
"state" : "LedgerOpened",
"ledgers" : [ {
"ledgerId" : 59169,
"entries" : 27,
"size" : 2727,
"offloaded" : false
}, {
"ledgerId" : 59211,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59293,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59346,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59395,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59447,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59500,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59549,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59602,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59652,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59700,
"entries" : 26,
"size" : 2626,
"offloaded" : false
}, {
"ledgerId" : 59756,
"entries" : 26,
"size" : 2626,
[GitHub] [pulsar] horsteff opened a new pull request #7558: [Issue 7557][pulsar-zookeeper-utils] Replace the use of Paths.get(...).getParent() for a zookeeper path in…
horsteff opened a new pull request #7558: URL: https://github.com/apache/pulsar/pull/7558 … ZooKeeperCache because it's system dependent and won't work when running Pulsar on Windows Fixes #7557 ### Motivation `org.apache.pulsar.zookeeper.ZooKeeperCache.process()` uses `java.nio.file.Paths` to evaluate the parent of a logical zookeeper path. The `Paths` class is system dependent while logical zookeeper paths are unix style, which leads to an `InvalidPathException` when running on windows. ### Modifications This change replaces the use of `Paths.get(...).getParent()` with a new method `getParentPath` which is placed in `org.apache.pulsar.zookeeper.ZkUtils` for common use. This method evaluates the parent of a unix style path and should return equal results as `UnixPath.getParent()` (which is used by `Paths` on unix like systems). ### Verifying this change This change adds test class `ZkUtilsParentPathTest` to check the correct function of the new `getParentPath` method. ### Documentation - Does this pull request introduce a new feature? yes (internal, `ZkUtils.getParentPath()`) - If yes, how is the feature documented? (JavaDocs / code comments) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] horsteff opened a new issue #7557: InvalidPathException on shutting down pulsar when running embedded on windows
horsteff opened a new issue #7557:
URL: https://github.com/apache/pulsar/issues/7557
**Describe the bug**
On shutting down an pulsar running embedded on windows it throws an
InvalidPathException:
```
java.nio.file.InvalidPathException: Illegal char <:> at index 29:
/loadbalance/brokers/hostname:8080
at
java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
at
java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
at
java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
at
java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
at java.base/java.nio.file.Path.of(Path.java:147)
at java.base/java.nio.file.Paths.get(Paths.java:69)
at
org.apache.pulsar.zookeeper.ZooKeeperCache.process(ZooKeeperCache.java:149)
at
org.apache.pulsar.zookeeper.LocalZooKeeperCache.process(LocalZooKeeperCache.java:60)
at
org.apache.pulsar.zookeeper.ZooKeeperDataCache.process(ZooKeeperDataCache.java:163)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:533)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:508)
```
This happens because the system dependent (Windows in this case)
java.nio.file.Paths is used to get the parent of a logical zookeeper path,
which isn't a filesystem path. Zookeeper uses unix like logical paths.
I encountered this when using an embedded pulsar for integration tests on
windows.
**To Reproduce**
Start and stop an embedded Pulsar. The exception will be logged:
```
val tempfile = Files.createTempFile(Paths.get(""), "standalone-",
".cfg").toFile()
tempfile.deleteOnExit()
val standalone = PulsarStandaloneBuilder.instance()
.withNoStreamStorage(true).build()
standalone.configFile = tempfile.absolutePath
standalone.config.managedLedgerDefaultEnsembleSize = 1
standalone.config.managedLedgerDefaultWriteQuorum = 1
standalone.config.managedLedgerDefaultAckQuorum = 1
standalone.start()
standalone.close()
```
**Expected behavior**
No InvalidPathException should be thrown on stopping embedded pulsar.
**Desktop (please complete the following information):**
- Windows
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #7548: [Issue 5922] - Cleanup poms to use ${project.groupId} and ${project.version} consistently
merlimat commented on a change in pull request #7548:
URL: https://github.com/apache/pulsar/pull/7548#discussion_r455903098
##
File path: examples/flink/pom.xml
##
@@ -74,6 +72,12 @@
${project.version}
+
Review comment:
Ok, the buildtools is anyway injected as a dependency for all the modules
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] vcottagiri commented on a change in pull request #7548: [Issue 5922] - Cleanup poms to use ${project.groupId} and ${project.version} consistently
vcottagiri commented on a change in pull request #7548:
URL: https://github.com/apache/pulsar/pull/7548#discussion_r455897202
##
File path: examples/flink/pom.xml
##
@@ -74,6 +72,12 @@
${project.version}
+
Review comment:
@merlimat
Sorry, this dependency is not needed as its inherited from the root pom.
I will create a PR fixing this asap
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7430: Handle NotAllowed Exception at the client side.
codelipenghui commented on pull request #7430: URL: https://github.com/apache/pulsar/pull/7430#issuecomment-659463151 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lbenc135 commented on issue #7477: [python-client 2.6.0] ContextualVersionConflict grpcio dependency conflict
lbenc135 commented on issue #7477: URL: https://github.com/apache/pulsar/issues/7477#issuecomment-659460179 @sijie It's a bit complicated because it involves having a python package which requires another python package. Basically it seems the problem is that in version 4.11.0 of `apache-bookkeeper-client` there was an explicit requirement added: `grpcio<1.26.0`, while in 4.10.0 this was only `>=1.8.2`. Since pulsar-client itself requires `grpcio`, but does not specify a version, pip installs the newest (maybe the solution is removing this dependency?). Setuptools verifies all the dependencies of my python package and throws this error. I temporarily solved it by explicitly adding `grpcio==1.25.0` in my requirements.txt. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7448: make AsyncHttpConnector.delayer thread daemon
codelipenghui commented on pull request #7448: URL: https://github.com/apache/pulsar/pull/7448#issuecomment-659446662 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] g0715158 commented on pull request #7555: Modify the log level of functions and move the java_instance_log4j2.xml file from the jar package to the conf directory
g0715158 commented on pull request #7555: URL: https://github.com/apache/pulsar/pull/7555#issuecomment-659430420 OK。think you 发自我的iPhone > 在 2020年7月16日，21:51，Jia Zhai 写道： > > > @g0715158 Would you please also put a copy of the file java_instance_log4j2.xml into conf directory? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or unsubscribe. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated: [docs] Improve configuration details of functions worker (#7534)
This is an automated email from the ASF dual-hosted git repository. zhaijia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 5fd3b0a [docs] Improve configuration details of functions worker (#7534) 5fd3b0a is described below commit 5fd3b0a886179f52b39257043e46ff222168df37 Author: Addison Higham AuthorDate: Thu Jul 16 07:54:00 2020 -0600 [docs] Improve configuration details of functions worker (#7534) Fixes #6086 ### Motivation The documentation for functions worker were lacking, as it didn't cover how to configure the functions worker in a variety of possible configurations of Pulsar, such as when the functions worker is run standalone or with TLS and/or authentication. ### Modifications This document expands the details for how to configure the functions worker based on a variety of scenarios. ### Verifying this change This is just docs, no test changes needed ### Does this pull request potentially affect one of the following parts: It doesn't effect anything other than docs --- site2/docs/functions-runtime.md | 191 ++-- 1 file changed, 162 insertions(+), 29 deletions(-) diff --git a/site2/docs/functions-runtime.md b/site2/docs/functions-runtime.md index 5134eb7..5734c2f 100644 --- a/site2/docs/functions-runtime.md +++ b/site2/docs/functions-runtime.md @@ -6,16 +6,16 @@ sidebar_label: "Setup: Configure Functions runtime" Pulsar Functions support the following methods to run functions. -- *Thread*: Invoke functions in threads in Functions Worker. -- *Process*: Invoke functions in processes forked by Functions Worker. -- *Kubernetes*: Submit functions as Kubernetes StatefulSets by Functions Worker. +- *Thread*: Invoke functions threads in functions worker. +- *Process*: Invoke functions in processes forked by functions worker. +- *Kubernetes*: Submit functions as Kubernetes StatefulSets by functions worker. Note > Pulsar supports adding labels to the Kubernetes StatefulSets and services > while launching functions, which facilitates selecting the target Kubernetes > objects. The differences of the thread and process modes are: -- Thread mode: when a function runs in thread mode, it runs on the same Java virtual machine (JVM) with Functions worker. -- Process mode: when a function runs in process mode, it runs on the same machine that Functions worker runs. +- Thread mode: when a function runs in thread mode, it runs on the same Java virtual machine (JVM) with functions worker. +- Process mode: when a function runs in process mode, it runs on the same machine that functions worker runs. ## Configure thread runtime It is easy to configure *Thread* runtime. In most cases, you do not need to configure anything. You can customize the thread group name with the following settings: @@ -46,6 +46,17 @@ processContainerFactory: ## Configure Kubernetes runtime + +### How it works + +The Kubernetes runtime works by having the functions worker generate and apply Kubernetes manifests. In the event that the functions worker is running on Kubernetes already, it can use the `serviceAccount` that is associated with the pod the functions worker is running in. Otherwise, it can be configured to communicate with a Kubernetes cluster. + +The manifests which the functions worker generates include a `StatefulSet`, a `Service` (which is used to communicate with the pods), and a `Secret` for auth credentials (when applicable). The `StatefulSet` manifest (by default) has a single pod, with the number of replicas determined by the "parallelism" of the function. On pod boot, the pod downloads the function payload (via the functions worker REST API). The pod's container image is configurable, but must have the functions runtime. + +The Kubernetes runtime also supports secrets, with the end user being able to create a Kubernetes secret and have it be exposed as an environment variable in the pod (described below). Additionally, the Kubernetes runtime fairly extensible, with the user being able to implement classes that customize the way Kubernetes manifests get generated, how auth data is passed to pods, and how secrets can be integrated. + +### Basic configuration + It is easy to configure Kubernetes runtime. You can just uncomment the settings of `kubernetesContainerFactory` in the `functions_worker.yaml` file. The following is an example. ```yaml @@ -78,10 +89,24 @@ kubernetesContainerFactory: percentMemoryPadding: 10 ``` -If you have already run a Pulsar cluster on Kubernetes, you can keep the settings unchanged at most of time. +As stated earlier, if you already run your functions worker embedded in a broker on Kubernetes, you can keep many of these settings as default. + +### Standalone functions worker on K
[GitHub] [pulsar] jiazhai closed issue #6086: [docs] Add doc on how to run Pulsar Functions as pod in Kubernetes
jiazhai closed issue #6086: URL: https://github.com/apache/pulsar/issues/6086 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] jiazhai merged pull request #7534: [docs] Improve configuration details of functions worker
jiazhai merged pull request #7534: URL: https://github.com/apache/pulsar/pull/7534 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] jiazhai commented on pull request #7539: [Issue 7489] Remove timestamp from metrics
jiazhai commented on pull request #7539: URL: https://github.com/apache/pulsar/pull/7539#issuecomment-659426383 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] jiazhai commented on pull request #7555: Modify the log level of functions and move the java_instance_log4j2.xml file from the jar package to the conf directory
jiazhai commented on pull request #7555: URL: https://github.com/apache/pulsar/pull/7555#issuecomment-659426096 @g0715158 Would you please also put a copy of the file `java_instance_log4j2.xml` into `conf` directory? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (4e358ef -> 14e3b7a)
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 4e358ef [PROTOBUF] Fix protobuf generation on handling repeated long number … (#7540) add 14e3b7a [Broker] Timeout opening managed ledger operation … (#7506) No new revisions were added by this update. Summary of changes: .../bookkeeper/mledger/impl/ManagedCursorImpl.java | 16 +++-- .../mledger/impl/ManagedLedgerFactoryImpl.java | 70 ++ 2 files changed, 72 insertions(+), 14 deletions(-)
[GitHub] [pulsar] codelipenghui merged pull request #7506: [Broker] Timeout opening managed ledger operation …
codelipenghui merged pull request #7506: URL: https://github.com/apache/pulsar/pull/7506 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] wolfstudy commented on pull request #6752: Support different docker images in Kubernetes runtime of Pulsar Functions
wolfstudy commented on pull request #6752: URL: https://github.com/apache/pulsar/pull/6752#issuecomment-659404814 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
