[GitHub] [pulsar] stanbog opened a new issue #10434: pulsar-broker not writing in log

2021-04-29 Thread GitBox
stanbog opened a new issue #10434: URL: https://github.com/apache/pulsar/issues/10434 Hello, We have a pulsar-broker cluster made from 3 nodes. We would like to forwards the logs into splunk so we are reading the file` /opt/pulsar/logs/pulsar-broker.log`. However, the pulsar-br

[GitHub] [pulsar] codelipenghui commented on a change in pull request #10319: Support listenerName for HttpLookupService

2021-04-29 Thread GitBox
codelipenghui commented on a change in pull request #10319: URL: https://github.com/apache/pulsar/pull/10319#discussion_r622786325 ## File path: pulsar-common/src/main/java/org/apache/pulsar/common/naming/TopicName.java ## @@ -331,11 +331,21 @@ public String getPersistenceNami

[GitHub] [pulsar] codelipenghui merged pull request #10422: [Transaction] Transaction log low water mark optimization.

2021-04-29 Thread GitBox
codelipenghui merged pull request #10422: URL: https://github.com/apache/pulsar/pull/10422 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[pulsar] branch master updated (5238822 -> bdb0990)

2021-04-29 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 5238822 [Transaction] Fix transaction buffer handler sync problem. (#10424) add bdb0990 [Transaction] Transa

[GitHub] [pulsar] RobertIndie commented on a change in pull request #10420: [Kinesis]Fix kinesis sink can not retry to send messages

2021-04-29 Thread GitBox
RobertIndie commented on a change in pull request #10420: URL: https://github.com/apache/pulsar/pull/10420#discussion_r622801176 ## File path: pulsar-io/kinesis/src/main/java/org/apache/pulsar/io/kinesis/KinesisSink.java ## @@ -121,10 +130,8 @@ public void write(Record record)

[GitHub] [pulsar] lhotari opened a new pull request #10435: [Broker] Make PersistentDispatcherMultipleConsumers.readMoreEntries synchronized

2021-04-29 Thread GitBox
lhotari opened a new pull request #10435: URL: https://github.com/apache/pulsar/pull/10435 - backports #10413 to branch-2.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[pulsar] branch branch-2.7 updated: [Broker] Fix issue in reusing EntryBatchIndexesAcks instances (#10400)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new 90c0c18 [Broker] Fix issue in reusin

[pulsar] branch branch-2.7 updated: fix standalone image error (#10359)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new 3be19ec fix standalone image error (

[GitHub] [pulsar] eolivelli commented on pull request #10337: Fix CPU 100% when deleting namespace

2021-04-29 Thread GitBox
eolivelli commented on pull request #10337: URL: https://github.com/apache/pulsar/pull/10337#issuecomment-829019260 @315157973 this patch does not apply to branch-2.7 can you please create a new PR for branch-2.7 ? -- This is an automated message from the Apache Git Service. To res

[GitHub] [pulsar] codelipenghui commented on issue #10433: NPE in broker: EntryImpl.getLength()

2021-04-29 Thread GitBox
codelipenghui commented on issue #10433: URL: https://github.com/apache/pulsar/issues/10433#issuecomment-829019325 @lhotari The pool here we are using is a thread-local pool, and always use the the same thread for a managed ledger to insert entry to the cache https://github.com/apache/puls

[GitHub] [pulsar] eolivelli commented on pull request #10317: [Doc] Add guides for end-to-end encryption in Node.js client

2021-04-29 Thread GitBox
eolivelli commented on pull request #10317: URL: https://github.com/apache/pulsar/pull/10317#issuecomment-829019934 I am removing 2.7.2 label, does are built from master, correct ? @Anonymitaet -- This is an automated message from the Apache Git Service. To respond to the message,

[pulsar] branch branch-2.7 updated: Support advertisedListeners for standalone (#10297)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new 4d9b94c Support advertisedListeners

[GitHub] [pulsar] eolivelli commented on pull request #10307: Fix C++ client cannot be built with Boost <=1.53

2021-04-29 Thread GitBox
eolivelli commented on pull request #10307: URL: https://github.com/apache/pulsar/pull/10307#issuecomment-829021127 This patch cannot be ported to branch-2.7, as it requires CI changes. CI changed a lot between 2.7 and 2.8 -- This is an automated message from the Apache Git Service. To

[GitHub] [pulsar] eolivelli commented on pull request #9440: [Issue 9438][pulsar-client] Fix inconsistent equals and hashCode for MessageIds

2021-04-29 Thread GitBox
eolivelli commented on pull request #9440: URL: https://github.com/apache/pulsar/pull/9440#issuecomment-829021833 This patch is too dangerous for 2.7.2 IMHO, I have removed the release/2.7.2 label @codelipenghui -- This is an automated message from the Apache Git Service. To respond

[GitHub] [pulsar] eolivelli commented on pull request #10305: [Issue 8214][pulsar-client] Added pausing of new topic consumers to MultiTopicsConsumerImpl.

2021-04-29 Thread GitBox
eolivelli commented on pull request #10305: URL: https://github.com/apache/pulsar/pull/10305#issuecomment-829022689 This patch does not apply to branch-2.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[pulsar] branch branch-2.7 updated: [Cursor] add return statement to exit asyncMarkDelete early on failure (#10272)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new d2138f7 [Cursor] add return statemen

[GitHub] [pulsar] eolivelli commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests

2021-04-29 Thread GitBox
eolivelli commented on pull request #9349: URL: https://github.com/apache/pulsar/pull/9349#issuecomment-829029259 this patch does not apply to branch-7.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [pulsar] eolivelli commented on pull request #10028: [Issue 10010][Client] fixed memory leak

2021-04-29 Thread GitBox
eolivelli commented on pull request #10028: URL: https://github.com/apache/pulsar/pull/10028#issuecomment-829030524 This patch does not apply to branch-7.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pulsar] eolivelli commented on pull request #10098: Fix IllegalStateException in PersistentReplicator

2021-04-29 Thread GitBox
eolivelli commented on pull request #10098: URL: https://github.com/apache/pulsar/pull/10098#issuecomment-829031452 LightProto is not present in branch-7.2, removing release/2.7.2 label -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[pulsar] branch branch-2.7 updated: [ISSUE 10153][Go Function] fix time unit ns -> ms (#10160)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new c088717 [ISSUE 10153][Go Function] f

[pulsar] branch branch-2.7 updated: [WebSocket Client] WebSocket url token param value optimization (#10187)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new 16387e2 [WebSocket Client] WebSocket

[GitHub] [pulsar] eolivelli commented on pull request #10220: [C++] Fix use-after-free undefined behavior due to object lifetime problem

2021-04-29 Thread GitBox
eolivelli commented on pull request #10220: URL: https://github.com/apache/pulsar/pull/10220#issuecomment-829032589 I am not sure it is safe to apply this patch to branch-2.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[pulsar] branch branch-2.7 updated: Suppress printing "skip Processing" rows in startup scripts (#10275)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new 2623241 Suppress printing "skip Proc

[GitHub] [pulsar] lhotari commented on issue #10433: NPE in broker: EntryImpl.getLength()

2021-04-29 Thread GitBox
lhotari commented on issue #10433: URL: https://github.com/apache/pulsar/issues/10433#issuecomment-829036198 @codelipenghui The pool is a thread-local pool, but the instance might be returned to a pool where it wasn't initially created, no? Thanks for pointing to the OpAddEntry code

[GitHub] [pulsar] lhotari edited a comment on issue #10433: NPE in broker: EntryImpl.getLength()

2021-04-29 Thread GitBox
lhotari edited a comment on issue #10433: URL: https://github.com/apache/pulsar/issues/10433#issuecomment-829036198 @codelipenghui Thanks for pointing to the OpAddEntry code location. I initially didn't realize that the code had both `entry` and `cachedEntry` and that it's the `entry.data`

[pulsar] branch branch-2.7 updated: [Issue 10221] Fix authorization error while using proxy and `Prefix` subscription authentication mode (#10226)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new fe7ee88 [Issue 10221] Fix authorizat

[pulsar] branch branch-2.7 updated: [Security] Upgrade commons-io to address CVE-2021-29425 (#10287)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new c941860 [Security] Upgrade commons-i

[GitHub] [pulsar] eolivelli commented on pull request #10121: Support set namespace and topic policy for non-persistent dispatcher

2021-04-29 Thread GitBox
eolivelli commented on pull request #10121: URL: https://github.com/apache/pulsar/pull/10121#issuecomment-829040007 @315157973 this patch does not apply to branch-7.2. Do you mind sending a PR against branch-7.2 ? this way we can deliver this fix in 2.7.2 -- This is an automated mes

[GitHub] [pulsar] lhotari commented on issue #10433: NPE in broker: EntryImpl.getLength()

2021-04-29 Thread GitBox
lhotari commented on issue #10433: URL: https://github.com/apache/pulsar/issues/10433#issuecomment-829047679 > The pool here we are using is a thread-local pool, and always use the the same thread for a managed ledger to insert entry to the cache @codelipenghui it seems that [Netty's

[GitHub] [pulsar] lhotari opened a new pull request #10436: [Java Client] Fix concurrency issue in incrementing epoch (#10278)

2021-04-29 Thread GitBox
lhotari opened a new pull request #10436: URL: https://github.com/apache/pulsar/pull/10436 - backport #10278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries ab

[GitHub] [pulsar] Anonymitaet commented on pull request #10317: [Doc] Add guides for end-to-end encryption in Node.js client

2021-04-29 Thread GitBox
Anonymitaet commented on pull request #10317: URL: https://github.com/apache/pulsar/pull/10317#issuecomment-829055022 @eolivelli I've added docs for both 2.7.2 and master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [pulsar] Anonymitaet edited a comment on pull request #10317: [Doc] Add guides for end-to-end encryption in Node.js client

2021-04-29 Thread GitBox
Anonymitaet edited a comment on pull request #10317: URL: https://github.com/apache/pulsar/pull/10317#issuecomment-829055022 @eolivelli I've added docs for both 2.7.2 and master, so users can read this content in 2.7.2 and master. -- This is an automated message from the Apache Git Servi

[GitHub] [pulsar] codelipenghui commented on pull request #10121: Support set namespace and topic policy for non-persistent dispatcher

2021-04-29 Thread GitBox
codelipenghui commented on pull request #10121: URL: https://github.com/apache/pulsar/pull/10121#issuecomment-829058345 This PR depends on https://github.com/apache/pulsar/pull/10061, we need to cherry-pick #10061 first. -- This is an automated message from the Apache Git Service. To res

[GitHub] [pulsar] AtticusLv opened a new issue #10437: pulsar java reactor based client support

2021-04-29 Thread GitBox
AtticusLv opened a new issue #10437: URL: https://github.com/apache/pulsar/issues/10437 Please support java and reactor client, reactive programming based client api. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [pulsar] lhotari commented on issue #10434: pulsar-broker not writing in log

2021-04-29 Thread GitBox
lhotari commented on issue #10434: URL: https://github.com/apache/pulsar/issues/10434#issuecomment-829068558 logging to the log file can be achieved by setting `PULSAR_ROUTING_APPENDER_DEFAULT=RollingFile` . The roll-over happens once per day or when the size of the log file reaches 1G

[pulsar] branch branch-2.7 updated (c941860 -> af6c2fa)

2021-04-29 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git. from c941860 [Security] Upgrade commons-io to address CVE-2021-29425 (#10287) new 26c2837 Fix schema ledger d

[pulsar] 02/02: Add underReplicate state in the topic internal stats (#10013)

2021-04-29 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git commit af6c2fa5659185193fa4af413aa5cea0007b81cd Author: lipenghui AuthorDate: Thu Apr 22 01:04:52 2021 +0800 Add unde

[pulsar] 01/02: Fix schema ledger deletion when deleting topic with delete schema. (#10383)

2021-04-29 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git commit 26c2837cb9e374942494e7f32f2d4d8f6df480f6 Author: lipenghui AuthorDate: Tue Apr 27 23:57:34 2021 +0800 Fix sche

[GitHub] [pulsar] vroyer opened a new issue #10438: Add a Message.getSchema() in the client API

2021-04-29 Thread GitBox
vroyer opened a new issue #10438: URL: https://github.com/apache/pulsar/issues/10438 When the schema is not known at compile time, a `Consumer` should be able to get the message schema. Currently, a `Consumer` can get the AVRO message and retreive its schema, but if the message is a

[GitHub] [pulsar] lhotari commented on pull request #10436: [Java Client] Fix concurrency issue in incrementing epoch (#10278)

2021-04-29 Thread GitBox
lhotari commented on pull request #10436: URL: https://github.com/apache/pulsar/pull/10436#issuecomment-829073111 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [pulsar] lhotari commented on issue #10437: pulsar java reactor based client support

2021-04-29 Thread GitBox
lhotari commented on issue #10437: URL: https://github.com/apache/pulsar/issues/10437#issuecomment-829081404 @AtticusLv Thanks for the feature request. I have some plans for this. The initial support would be an adapter library which adapts the existing Pulsar Java Client's Async API to Pr

[pulsar] branch branch-2.7 updated (af6c2fa -> 9a72fe4)

2021-04-29 Thread bogong
This is an automated email from the ASF dual-hosted git repository. bogong pushed a change to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git. from af6c2fa Add underReplicate state in the topic internal stats (#10013) new bf81fbf [Python] Fix nested Map

[pulsar] 01/02: [Python] Fix nested Map or Array in schema doesn't work (#9548)

2021-04-29 Thread bogong
This is an automated email from the ASF dual-hosted git repository. bogong pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git commit bf81fbf79e3222db01e6105b6330960c52ab42f2 Author: Yunze Xu AuthorDate: Fri Feb 12 02:10:11 2021 +0800 [Python] F

[pulsar] 02/02: [Python schema] Support python avro schema set default value. (#10265)

2021-04-29 Thread bogong
This is an automated email from the ASF dual-hosted git repository. bogong pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git commit 9a72fe4fa137e369aa27150e46c41b67041de022 Author: congbo <39078850+congbobo...@users.noreply.github.com> AuthorDate: M

[GitHub] [pulsar] eolivelli commented on issue #10438: Add a Message.getSchema() in the client API

2021-04-29 Thread GitBox
eolivelli commented on issue #10438: URL: https://github.com/apache/pulsar/issues/10438#issuecomment-829084318 @congbobo184 here https://github.com/apache/pulsar/pull/9970/files#diff-955419b4b0ad976e96f9f7595989e79c391109aeaa304bd286a80fc6eb9360c7R324 you are adding MessageImpl#getSchema()

[GitHub] [pulsar] eolivelli commented on a change in pull request #10428: [Issue #10427] Add AvroSchema UUID support fix

2021-04-29 Thread GitBox
eolivelli commented on a change in pull request #10428: URL: https://github.com/apache/pulsar/pull/10428#discussion_r622887818 ## File path: pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/AvroSchemaTest.java ## @@ -423,4 +425,16 @@ public void testAvroSchemaU

[GitHub] [pulsar] eolivelli commented on pull request #10121: Support set namespace and topic policy for non-persistent dispatcher

2021-04-29 Thread GitBox
eolivelli commented on pull request #10121: URL: https://github.com/apache/pulsar/pull/10121#issuecomment-829088145 I would like to not cherry pick #10061 it looks to me that it is better not to introduce now refactor, now that we are close to the release @codelipenghui -- This is an

[GitHub] [pulsar] eolivelli commented on pull request #7406: Improved in max-pending-bytes mechanism for broker

2021-04-29 Thread GitBox
eolivelli commented on pull request #7406: URL: https://github.com/apache/pulsar/pull/7406#issuecomment-829090882 @merlimat do you want to rebase this onto current master ? it looks like it is a very good improvement -- This is an automated message from the Apache Git Service. To respo

[GitHub] [pulsar] codelipenghui commented on pull request #10420: [Kinesis]Fix kinesis sink can not retry to send messages

2021-04-29 Thread GitBox
codelipenghui commented on pull request #10420: URL: https://github.com/apache/pulsar/pull/10420#issuecomment-829091168 @dlg99 @RobertIndie We can leverage the max unack message limitation to limit the memory of the connector when using `retainOrdering=false`, it needs users to use the Sha

[GitHub] [pulsar] zymap commented on issue #8989: Pulsar Admin API - allows any authenticated user to delete or make changes to any tenant/namespace/topic

2021-04-29 Thread GitBox
zymap commented on issue #8989: URL: https://github.com/apache/pulsar/issues/8989#issuecomment-829110658 Hi @rounak11, which pulsar version you are using? For the first issue, I can use a superuser to produce and consume messages with pulsar 2.7.1. For the second issue, I think y

[GitHub] [pulsar] zymap commented on issue #8989: Pulsar Admin API - allows any authenticated user to delete or make changes to any tenant/namespace/topic

2021-04-29 Thread GitBox
zymap commented on issue #8989: URL: https://github.com/apache/pulsar/issues/8989#issuecomment-829120675 @lmccay I think it's a document issue. You can find the rest API here: https://pulsar.incubator.apache.org/admin-rest-api/#operation/deleteTopic -- This is an automated message from t

[GitHub] [pulsar] lhotari opened a new issue #10439: [Broker & Bookie] Configure Bookkeeper client & server backpressure by default

2021-04-29 Thread GitBox
lhotari opened a new issue #10439: URL: https://github.com/apache/pulsar/issues/10439 **Is your enhancement request related to a problem? Please describe.** Pulsar doesn't take advantage of Bookkeeper's client-side and server-side backpressure features. https://github.com/apache/

[GitHub] [pulsar] eolivelli commented on issue #10439: [Broker & Bookie] Configure Bookkeeper client & server backpressure by default

2021-04-29 Thread GitBox
eolivelli commented on issue #10439: URL: https://github.com/apache/pulsar/issues/10439#issuecomment-829134966 @lhotari since 2.8 you can configure the bookkeeper client inside the broker just by adding bookkeeper_xxx in broker.conf check for instance the case of "bookkeeper_oppor

[GitHub] [pulsar] lhotari commented on issue #10439: [Broker & Bookie] Configure Bookkeeper client & server backpressure by default

2021-04-29 Thread GitBox
lhotari commented on issue #10439: URL: https://github.com/apache/pulsar/issues/10439#issuecomment-829147823 @eolivelli Would it be possible to cherry-pick #9232 to branch-2.7 so that it would be possible to configure `waitTimeoutOnBackpressureMs` on the broker side by using `bookkeeper_wa

[GitHub] [pulsar] lhotari commented on issue #10136: Memory leak

2021-04-29 Thread GitBox
lhotari commented on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829153804 Is this a duplicate of #9562 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [pulsar] galrose commented on issue #10136: Memory leak

2021-04-29 Thread GitBox
galrose commented on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829159281 No because the maxMessageBufferSizeInMb is just the buffer for messages. Even after there is no producers and the all the data is in the bookie then the memory stays the same, inste

[GitHub] [pulsar] lhotari commented on issue #10136: Memory leak

2021-04-29 Thread GitBox
lhotari commented on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829159713 For direct memory leaks, it's most likely about Netty pooled memory. For debugging the situation, it might be useful to check if there are log messages from the Netty leak detector.

[GitHub] [pulsar] galrose commented on issue #10136: Memory leak

2021-04-29 Thread GitBox
galrose commented on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829160476 Thanks I'll try to do that :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [pulsar] eolivelli commented on issue #10439: [Broker & Bookie] Configure Bookkeeper client & server backpressure by default

2021-04-29 Thread GitBox
eolivelli commented on issue #10439: URL: https://github.com/apache/pulsar/issues/10439#issuecomment-829160427 @lhotari yes I have picked it to branch-7.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[pulsar] branch branch-2.7 updated: Allow to configure BookKeeper Opportunistic Striping Feature (and all BK client features using bookkeeper_ prefix) (#9232)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new baddf9d Allow to configure BookKeepe

[GitHub] [pulsar] vroyer commented on a change in pull request #10428: [Issue #10427] Add AvroSchema UUID support fix

2021-04-29 Thread GitBox
vroyer commented on a change in pull request #10428: URL: https://github.com/apache/pulsar/pull/10428#discussion_r622975597 ## File path: pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/AvroSchemaTest.java ## @@ -423,4 +425,16 @@ public void testAvroSchemaUser

[GitHub] [pulsar] eolivelli merged pull request #10435: [Broker] Make PersistentDispatcherMultipleConsumers.readMoreEntries synchronized

2021-04-29 Thread GitBox
eolivelli merged pull request #10435: URL: https://github.com/apache/pulsar/pull/10435 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, ple

[pulsar] branch branch-2.7 updated: [Broker] Make Persistent*DispatcherMultipleConsumers.readMoreEntries synchronized (#10413) (#10435)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new 1f467a0 [Broker] Make Persistent*Di

[GitHub] [pulsar] eolivelli merged pull request #10436: [Java Client] Fix concurrency issue in incrementing epoch (#10278)

2021-04-29 Thread GitBox
eolivelli merged pull request #10436: URL: https://github.com/apache/pulsar/pull/10436 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, ple

[pulsar] branch branch-2.7 updated: [Java Client] Fix concurrency issue in incrementing epoch (#10278) (#10436)

2021-04-29 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new b68606a [Java Client] Fix concurrenc

[GitHub] [pulsar] lhotari edited a comment on issue #10136: Memory leak

2021-04-29 Thread GitBox
lhotari edited a comment on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829159713 For direct memory leaks, it's most likely about Netty pooled memory. For debugging the situation, it might be useful to check if there are log messages from the Netty leak de

[GitHub] [pulsar] lhotari edited a comment on issue #10136: Memory leak

2021-04-29 Thread GitBox
lhotari edited a comment on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829159713 For direct memory leaks, it's most likely about Netty pooled memory. For debugging the situation, it might be useful to check if there are log messages from the Netty leak de

[GitHub] [pulsar] lhotari edited a comment on issue #10136: Memory leak

2021-04-29 Thread GitBox
lhotari edited a comment on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829159713 For direct memory leaks, it's most likely about Netty pooled memory. For debugging the situation, it might be useful to check if there are log messages from the Netty leak de

[GitHub] [pulsar] BewareMyPower commented on pull request #10220: [C++] Fix use-after-free undefined behavior due to object lifetime problem

2021-04-29 Thread GitBox
BewareMyPower commented on pull request #10220: URL: https://github.com/apache/pulsar/pull/10220#issuecomment-829175188 @eolivelli I think it's safe to cherry-pick to branch-2.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [pulsar] lhotari commented on issue #10136: Memory leak

2021-04-29 Thread GitBox
lhotari commented on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829183581 For example, In the k8s Helm chart deployment, an environment variable can be set for the broker by adding it under `broker.configData` in the `values.yaml` for the deployment.

[GitHub] [pulsar] lhotari edited a comment on issue #10136: Memory leak

2021-04-29 Thread GitBox
lhotari edited a comment on issue #10136: URL: https://github.com/apache/pulsar/issues/10136#issuecomment-829183581 For example, In the k8s Helm chart deployment, an environment variable can be set for the broker by adding it under `broker.configData` in the `values.yaml` for the deploymen

[GitHub] [pulsar] MarvinCai commented on pull request #10401: [CLI] Add support for setting time based limit on backlog quota with CLI.

2021-04-29 Thread GitBox
MarvinCai commented on pull request #10401: URL: https://github.com/apache/pulsar/pull/10401#issuecomment-829200600 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pulsar] eolivelli commented on pull request #10220: [C++] Fix use-after-free undefined behavior due to object lifetime problem

2021-04-29 Thread GitBox
eolivelli commented on pull request #10220: URL: https://github.com/apache/pulsar/pull/10220#issuecomment-829207327 It does not apply cleanly to branch-2.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pulsar] josephlim75 commented on issue #572: Don't depend on ZooKeeper for coordination

2021-04-29 Thread GitBox
josephlim75 commented on issue #572: URL: https://github.com/apache/pulsar/issues/572#issuecomment-829216496 I hope Pulsar and BookKeeper will ditch Zookeeper and come out with their internal built-in cluster coordination like Raft, what new Kafka 2.8 has developed `KIP-500` and also how D

[GitHub] [pulsar] BewareMyPower commented on pull request #10220: [C++] Fix use-after-free undefined behavior due to object lifetime problem

2021-04-29 Thread GitBox
BewareMyPower commented on pull request #10220: URL: https://github.com/apache/pulsar/pull/10220#issuecomment-829220049 OK, I didn't notice it just now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [pulsar] josephlim75 edited a comment on issue #572: Don't depend on ZooKeeper for coordination

2021-04-29 Thread GitBox
josephlim75 edited a comment on issue #572: URL: https://github.com/apache/pulsar/issues/572#issuecomment-829216496 I hope Pulsar and BookKeeper will ditch Zookeeper and come out with their internal built-in cluster coordination like Raft, what new Kafka 2.8 has developed `KIP-500` and als

[GitHub] [pulsar] RobertIndie commented on pull request #10420: [Kinesis]Fix kinesis sink can not retry to send messages

2021-04-29 Thread GitBox
RobertIndie commented on pull request #10420: URL: https://github.com/apache/pulsar/pull/10420#issuecomment-829228354 Here I did some stress tests: * Start the sink connector with `retainOrdering` enabled. ![image](https://user-images.githubusercontent.com/16974619/116555693-40c2d700-

[GitHub] [pulsar] codelipenghui commented on pull request #10420: [Kinesis]Fix kinesis sink can not retry to send messages

2021-04-29 Thread GitBox
codelipenghui commented on pull request #10420: URL: https://github.com/apache/pulsar/pull/10420#issuecomment-829233360 @dlg99 Please help take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [pulsar] eolivelli commented on issue #10438: Add a Message.getSchema() in the client API

2021-04-29 Thread GitBox
eolivelli commented on issue #10438: URL: https://github.com/apache/pulsar/issues/10438#issuecomment-829234676 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries a

[GitHub] [pulsar] merlimat commented on pull request #7406: Improved in max-pending-bytes mechanism for broker

2021-04-29 Thread GitBox
merlimat commented on pull request #7406: URL: https://github.com/apache/pulsar/pull/7406#issuecomment-829235143 Yes, I’ll get this ready in the next few days. There are also some improvements I want to do to this PR. -- This is an automated message from the Apache Git Service. To respo

[GitHub] [pulsar] abhilashmandaliya opened a new issue #10440: Sink config parsing doesn't preserve the original data type

2021-04-29 Thread GitBox
abhilashmandaliya opened a new issue #10440: URL: https://github.com/apache/pulsar/issues/10440 **Describe the bug** Pulsar IO Sink config parsing is using Gson to parse the user-supplied config. Gson parses numeric values to double if we parse it to Java's Object class. This can be a p

[GitHub] [pulsar] abhilashmandaliya commented on pull request #10214: added a constructor to pass ExecutorProviders for the PulsarClient

2021-04-29 Thread GitBox
abhilashmandaliya commented on pull request #10214: URL: https://github.com/apache/pulsar/pull/10214#issuecomment-829261022 Closing due to some branching issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pulsar] abhilashmandaliya closed pull request #10214: added a constructor to pass ExecutorProviders for the PulsarClient

2021-04-29 Thread GitBox
abhilashmandaliya closed pull request #10214: URL: https://github.com/apache/pulsar/pull/10214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this serv

[GitHub] [pulsar] abhilashmandaliya opened a new pull request #10441: Using ObjectMapper instead of Gson to parse Source/SInk configuration

2021-04-29 Thread GitBox
abhilashmandaliya opened a new pull request #10441: URL: https://github.com/apache/pulsar/pull/10441 Fixes #10440 ### Motivation Data types should be preserved after deserializing the sink/source config. ### Modifications Using ObjectMapper instead of Gson to pars

[GitHub] [pulsar] abhilashmandaliya commented on a change in pull request #10441: Using ObjectMapper instead of Gson to parse Source/SInk configuration

2021-04-29 Thread GitBox
abhilashmandaliya commented on a change in pull request #10441: URL: https://github.com/apache/pulsar/pull/10441#discussion_r623086334 ## File path: pulsar-client/src/main/java/org/apache/pulsar/client/impl/PulsarClientImpl.java ## @@ -764,8 +763,18 @@ public void shutdown() t

[GitHub] [pulsar] abhilashmandaliya commented on pull request #10441: [pulsar-io] Using ObjectMapper instead of Gson to parse Source/SInk configuration

2021-04-29 Thread GitBox
abhilashmandaliya commented on pull request #10441: URL: https://github.com/apache/pulsar/pull/10441#issuecomment-829270164 @MarvinCai @eolivelli @315157973 @lhotari kindly review it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [pulsar] dinghram commented on issue #10433: NPE in broker: EntryImpl.getLength()

2021-04-29 Thread GitBox
dinghram commented on issue #10433: URL: https://github.com/apache/pulsar/issues/10433#issuecomment-829275756 Pulsar version: 2.7.0 + cherry-picked fixes Deployment: helm Java version: openjdk version "1.8.0_282" OpenJDK Runtime Environment (IcedTea 3.18.0) (build 1.8.0_282-

[GitHub] [pulsar] galrose commented on pull request #7406: Improved in max-pending-bytes mechanism for broker

2021-04-29 Thread GitBox
galrose commented on pull request #7406: URL: https://github.com/apache/pulsar/pull/7406#issuecomment-829277505 @merlimat thats great thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [pulsar] lhotari opened a new pull request #10442: [Docs] Add instructions for building custom docker images

2021-04-29 Thread GitBox
lhotari opened a new pull request #10442: URL: https://github.com/apache/pulsar/pull/10442 ### Motivation & Modifications Adds instructions for building custom docker images to README.md -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pulsar] lhotari commented on pull request #10442: [Docs] Add instructions for building custom docker images

2021-04-29 Thread GitBox
lhotari commented on pull request #10442: URL: https://github.com/apache/pulsar/pull/10442#issuecomment-829280896 @Anonymitaet @Huanli-Meng Please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [pulsar] wmccarley commented on issue #10406: Message loss in non-persistent geo-replication at high sending throughput

2021-04-29 Thread GitBox
wmccarley commented on issue #10406: URL: https://github.com/apache/pulsar/issues/10406#issuecomment-829291112 We use a lot of high-throughput non-persistent topics on our system for streaming telemetry, it's useful for use cases where the overhead of tracking message acknowledgement and r

[GitHub] [pulsar] eolivelli commented on a change in pull request #10428: [Issue #10427] Add AvroSchema UUID support fix

2021-04-29 Thread GitBox
eolivelli commented on a change in pull request #10428: URL: https://github.com/apache/pulsar/pull/10428#discussion_r623130527 ## File path: pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/AvroSchema.java ## @@ -127,7 +127,12 @@ public static void addLogicalTy

[GitHub] [pulsar] LiPengze97 commented on issue #10406: Message loss in non-persistent geo-replication at high sending throughput

2021-04-29 Thread GitBox
LiPengze97 commented on issue #10406: URL: https://github.com/apache/pulsar/issues/10406#issuecomment-829310879 > We use a lot of high-throughput non-persistent topics on our system for streaming telemetry, it's useful for use cases where the overhead of tracking message acknowledgement an

[GitHub] [pulsar] LiPengze97 commented on issue #10406: Message loss in non-persistent geo-replication at high sending throughput

2021-04-29 Thread GitBox
LiPengze97 commented on issue #10406: URL: https://github.com/apache/pulsar/issues/10406#issuecomment-829311445 > yes, non-persistent topic doesn't persist messages so, it doesn't give message delivery guarantee. it's used for usecases which only care about the latest data and don't care a

[GitHub] [pulsar] SleeperSmith commented on issue #572: Don't depend on ZooKeeper for coordination

2021-04-29 Thread GitBox
SleeperSmith commented on issue #572: URL: https://github.com/apache/pulsar/issues/572#issuecomment-829315109 Moving raft into Pulsar imo is like asking Wordpress to move mariadb into its own code base. Strongly opposed to the idea as this is just wasting Pulsar dev's time rewriting the ra

[GitHub] [pulsar] tomdavidson commented on issue #572: Don't depend on ZooKeeper for coordination

2021-04-29 Thread GitBox
tomdavidson commented on issue #572: URL: https://github.com/apache/pulsar/issues/572#issuecomment-829338113 > hope Pulsar devs don't make the same mistake Kafka did I thought thats what Pulsar is :D 100% in agreement with @SleeperSmith ectd/consul/zk as plugable options see

[GitHub] [pulsar] abhilashmandaliya commented on pull request #10441: [pulsar-io] Using ObjectMapper instead of Gson to parse Source/SInk configuration

2021-04-29 Thread GitBox
abhilashmandaliya commented on pull request #10441: URL: https://github.com/apache/pulsar/pull/10441#issuecomment-829341578 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [pulsar] codelipenghui merged pull request #10420: [Kinesis]Fix kinesis sink can not retry to send messages

2021-04-29 Thread GitBox
codelipenghui merged pull request #10420: URL: https://github.com/apache/pulsar/pull/10420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[pulsar] branch master updated (bdb0990 -> 345cd33)

2021-04-29 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from bdb0990 [Transaction] Transaction log low water mark optimization. (#10422) add 345cd33 [Kinesis]Fix kinesis

[GitHub] [pulsar] merlimat commented on a change in pull request #10441: [pulsar-io] Using ObjectMapper instead of Gson to parse Source/SInk configuration

2021-04-29 Thread GitBox
merlimat commented on a change in pull request #10441: URL: https://github.com/apache/pulsar/pull/10441#discussion_r623225368 ## File path: pulsar-client/src/main/java/org/apache/pulsar/client/impl/PulsarClientImpl.java ## @@ -197,6 +197,7 @@ private PulsarClientImpl(ClientCon

  1   2   >