[GitHub] [pulsar] michaeljmarshall commented on pull request #10276: [GitHub Actions] Fix logic that was incorrectly skipping tests

2021-04-19 Thread GitBox
michaeljmarshall commented on pull request #10276: URL: https://github.com/apache/pulsar/pull/10276#issuecomment-822993162 @lhotari, @sijie, @eolivelli - PTAL. This is an urgent fix that should be merged as soon as we're good with it. -- This is an automated message from the Apache Git

[GitHub] [pulsar] michaeljmarshall opened a new pull request #10276: [GitHub Actions] Fix logic that was incorrectly skipping tests

2021-04-19 Thread GitBox
michaeljmarshall opened a new pull request #10276: URL: https://github.com/apache/pulsar/pull/10276 ### Motivation Currently, when there are changes made to code and documentation in the same PR, many of the tests are skipped because the conditional logic is not being parsed as

[GitHub] [pulsar] lhotari commented on pull request #10275: [Broker] Suppress printing of "skip Processing" lines in startup scripts

2021-04-19 Thread GitBox
lhotari commented on pull request #10275: URL: https://github.com/apache/pulsar/pull/10275#issuecomment-822988622 @klwilson227 @codelipenghui Please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pulsar] lhotari opened a new pull request #10275: [Broker] Suppress printing of "skip Processing" lines in startup scripts

2021-04-19 Thread GitBox
lhotari opened a new pull request #10275: URL: https://github.com/apache/pulsar/pull/10275 ### Motivation The startup logging of the broker (and bookie) is very verbose after the changes made in #8709 . example of typical output: ``` [conf/broker.conf] skip Processing #

[pulsar] branch master updated: [Security] Upgrade Netty to 4.1.63.Final (#10266)

2021-04-19 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 0b71b13 [Security] Upgrade Netty to

[GitHub] [pulsar] eolivelli merged pull request #10266: [Security] Upgrade Netty to 4.1.63.Final to address CVE-2021-21409

2021-04-19 Thread GitBox
eolivelli merged pull request #10266: URL: https://github.com/apache/pulsar/pull/10266 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [pulsar] khiem111189 closed pull request #10274: Update security-authorization.md

2021-04-19 Thread GitBox
khiem89 closed pull request #10274: URL: https://github.com/apache/pulsar/pull/10274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [pulsar] khiem111189 opened a new pull request #10274: Update security-authorization.md

2021-04-19 Thread GitBox
khiem89 opened a new pull request #10274: URL: https://github.com/apache/pulsar/pull/10274 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)*

[pulsar] branch master updated: Add JsonRecordBuilder implementation (#10052)

2021-04-19 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 08cbdc7 Add JsonRecordBuilder implementation

[GitHub] [pulsar] codelipenghui merged pull request #10052: Add a JSON RecordBuilder to the GenericJsonSchema

2021-04-19 Thread GitBox
codelipenghui merged pull request #10052: URL: https://github.com/apache/pulsar/pull/10052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [pulsar] MarvinCai commented on issue #10263: Cannot delete non empty bundle

2021-04-19 Thread GitBox
MarvinCai commented on issue #10263: URL: https://github.com/apache/pulsar/issues/10263#issuecomment-822913298 We're relying on Zookeeper watch to invalidate cache entry and refetch, so that delay is inevitable. We might able to invalidate entry in cache before delete/update in

[GitHub] [pulsar] jerrypeng opened a new pull request #10273: Improve localrun performance by using JVM options more suited for it

2021-04-19 Thread GitBox
jerrypeng opened a new pull request #10273: URL: https://github.com/apache/pulsar/pull/10273 ### Motivation Currently, the JVM options used for localrun are the same as that used for the pulsar-admin tool. The JVM options used are suited for CLI tools that need a short startup

[GitHub] [pulsar] Anonymitaet commented on pull request #10267: Add documentation about Sink and Source

2021-04-19 Thread GitBox
Anonymitaet commented on pull request #10267: URL: https://github.com/apache/pulsar/pull/10267#issuecomment-822885355 @eolivelli many thanks for your contribution! I've left some comments, PTAL. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #10267: Add documentation about Sink and Source

2021-04-19 Thread GitBox
Anonymitaet commented on a change in pull request #10267: URL: https://github.com/apache/pulsar/pull/10267#discussion_r616261206 ## File path: site2/docs/io-develop.md ## @@ -85,6 +85,46 @@ interface, which means you need to implement the {@inject: github:`open`:/pulsar

[GitHub] [pulsar] Anonymitaet commented on pull request #10256: Issue 10236: [Doc] generate 2.7.2 docs

2021-04-19 Thread GitBox
Anonymitaet commented on pull request #10256: URL: https://github.com/apache/pulsar/pull/10256#issuecomment-822879935 @Jennifer88huang 1. Does [`releases.json`](https://github.com/apache/pulsar/blob/master/site2/website/releases.json) point to [this

[GitHub] [pulsar] david-streamlio commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616257932 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -269,9 +270,11 @@

[GitHub] [pulsar] david-streamlio commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616257832 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -324,7 +327,11 @@

[GitHub] [pulsar] david-streamlio commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616257540 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -337,16 +344,18 @@

[GitHub] [pulsar] david-streamlio commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616257284 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -337,16 +344,18 @@

[GitHub] [pulsar] david-streamlio commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616257032 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -337,16 +344,18 @@

[GitHub] [pulsar] david-streamlio commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616229860 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -322,6 +322,7 @@

[GitHub] [pulsar] michaeljmarshall commented on pull request #10254: [Broker] Remove possible error case for system topic name checking

2021-04-19 Thread GitBox
michaeljmarshall commented on pull request #10254: URL: https://github.com/apache/pulsar/pull/10254#issuecomment-822866380 > Why did the tests now actually run for this PR? There are code changes for this PR as well. Great question. I took a look and didn't see anything obviously

[GitHub] [pulsar] dlg99 commented on a change in pull request #10267: Add documentation about Sink and Source

2021-04-19 Thread GitBox
dlg99 commented on a change in pull request #10267: URL: https://github.com/apache/pulsar/pull/10267#discussion_r616243256 ## File path: site2/docs/io-develop.md ## @@ -123,6 +163,57 @@ Developing a sink connector **is similar to** developing a source connector, tha

[GitHub] [pulsar] michaeljmarshall opened a new pull request #10272: [Cursor] add return statement to exit asyncMarkDelete early on failure

2021-04-19 Thread GitBox
michaeljmarshall opened a new pull request #10272: URL: https://github.com/apache/pulsar/pull/10272 ### Motivation In reading through the message expiration code, I noticed that there is a case where we call `markDeleteFailed` on a callback, but don't subsequently return from the

[GitHub] [pulsar] david-streamlio commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616229860 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -322,6 +322,7 @@

[GitHub] [pulsar] jerrypeng commented on pull request #10208: Fix Pulsar Function localrun with multiple instances and metrics server is enabled

2021-04-19 Thread GitBox
jerrypeng commented on pull request #10208: URL: https://github.com/apache/pulsar/pull/10208#issuecomment-822838200 @michaeljmarshall sounds good to me. Can you volunteer to do the backport :) ? -- This is an automated message from the Apache Git Service. To respond to the message,

[pulsar] branch master updated: Fix Pulsar Function localrun with multiple instances and metrics server is enabled (#10208)

2021-04-19 Thread jerrypeng
This is an automated email from the ASF dual-hosted git repository. jerrypeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 7a89212 Fix Pulsar Function localrun with

[GitHub] [pulsar] jerrypeng merged pull request #10208: Fix Pulsar Function localrun with multiple instances and metrics server is enabled

2021-04-19 Thread GitBox
jerrypeng merged pull request #10208: URL: https://github.com/apache/pulsar/pull/10208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [pulsar] dlg99 opened a new issue #10271: pulsar-admin: Schema type BYTES is returned as Schema type NONE

2021-04-19 Thread GitBox
dlg99 opened a new issue #10271: URL: https://github.com/apache/pulsar/issues/10271 **Describe the bug** Schema type BYTES set for the topic is returned as Schema type NONE. I hit this issue while working on the sink that uses GenericObject / relies on AutoConsumeSchema

[GitHub] [pulsar] jerrypeng commented on a change in pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
jerrypeng commented on a change in pull request #10270: URL: https://github.com/apache/pulsar/pull/10270#discussion_r616207484 ## File path: pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java ## @@ -322,6 +322,7 @@ private

[GitHub] [pulsar-client-go] hunter2046 commented on pull request #507: Expose GetHashingFunction() method on producer

2021-04-19 Thread GitBox
hunter2046 commented on pull request #507: URL: https://github.com/apache/pulsar-client-go/pull/507#issuecomment-822802546 > The point is wrong, can MessageRouter not meet our functions? > > e.g: > > ``` >producer, err := client.CreateProducer(ProducerOptions{ >

[GitHub] [pulsar] david-streamlio opened a new pull request #10270: [Issue-10269] [pulsar-io] Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio opened a new pull request #10270: URL: https://github.com/apache/pulsar/pull/10270 Fixes #10269 ### Motivation When an Error or Exception occurs in write() method of the sink, the error is logged but no actions are performed. We need the exception to be

[GitHub] [pulsar] kirill-kosenko commented on issue #9114: org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId

2021-04-19 Thread GitBox
kirill-kosenko commented on issue #9114: URL: https://github.com/apache/pulsar/issues/9114#issuecomment-822766525 I observe similar errors when "Pulsar Transactions" feature is enabled ``` 23:26:41.278 [pulsar-io-52-11] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException

[GitHub] [pulsar] david-streamlio opened a new issue #10269: Pulsar IO Sink errors are not bubbled up correctly

2021-04-19 Thread GitBox
david-streamlio opened a new issue #10269: URL: https://github.com/apache/pulsar/issues/10269 **Describe the bug** When an Error or Exception occurs in write() method of the sink, the error is logged but no actions are performed. The reason why is the logic that was added to enabled

[GitHub] [pulsar] frank-montyne commented on issue #10263: Cannot delete non empty bundle

2021-04-19 Thread GitBox
frank-montyne commented on issue #10263: URL: https://github.com/apache/pulsar/issues/10263#issuecomment-822747814 I can do that but that’s quite arbitrary. Is it possible to fix this in the future? > On 19 Apr 2021, at 15:16, Marvin Cai ***@***.***> wrote: > >  > Seems

[GitHub] [pulsar] jerrypeng commented on pull request #10254: [Broker] Remove possible error case for system topic name checking

2021-04-19 Thread GitBox
jerrypeng commented on pull request #10254: URL: https://github.com/apache/pulsar/pull/10254#issuecomment-822708402 Why did the tests now actually run for this PR? There are code changes for this PR as well. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pulsar] dlg99 commented on pull request #9927: Add ability to use Kafka's sinks as pulsar sinks

2021-04-19 Thread GitBox
dlg99 commented on pull request #9927: URL: https://github.com/apache/pulsar/pull/9927#issuecomment-822641966 rebased on latest master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [pulsar] lhotari commented on issue #10054: [Geo-replicaiton] Subscription replication is not working across clusters

2021-04-19 Thread GitBox
lhotari commented on issue #10054: URL: https://github.com/apache/pulsar/issues/10054#issuecomment-822564089 > To "fix" the test I just put a `Thread.sleep(2 * config1.getReplicatedSubscriptionsSnapshotFrequencyMillis())` before the producer gets started. > > One other minor thing

[pulsar] branch master updated (b33aca9 -> d6d0e3a)

2021-04-19 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from b33aca9 [doc] Support generate document automatically for pulsar-perf (#10120) add d6d0e3a [Python schema]

[GitHub] [pulsar] lhotari commented on pull request #10260: Upgrade k8s java-client to fix dependency resolution

2021-04-19 Thread GitBox
lhotari commented on pull request #10260: URL: https://github.com/apache/pulsar/pull/10260#issuecomment-822547486 @jerrypeng What would be the best way to verify this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [pulsar] codelipenghui merged pull request #10265: [Python schema] Support python avro schema set default value.

2021-04-19 Thread GitBox
codelipenghui merged pull request #10265: URL: https://github.com/apache/pulsar/pull/10265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [pulsar] codelipenghui commented on pull request #10265: [Python schema] Support python avro schema set default value.

2021-04-19 Thread GitBox
codelipenghui commented on pull request #10265: URL: https://github.com/apache/pulsar/pull/10265#issuecomment-822547088 mark as release/2.7.2 since this is a bug when the users using the schema in the Python client. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [pulsar] codelipenghui commented on issue #10244: Pulsar 2.7.1 checks schema compatibility even if set to ALWAYS_COMPATIBLE

2021-04-19 Thread GitBox
codelipenghui commented on issue #10244: URL: https://github.com/apache/pulsar/issues/10244#issuecomment-822545570 @gvolpe Looks the problem is not about the compatibility, could you please share the schema definition? From the log, seem the problem is the the provided schema definition

[GitHub] [pulsar] 315157973 commented on pull request #9598: Make `LocalPolicies` immutable to avoid concurrent modify inconsistent.

2021-04-19 Thread GitBox
315157973 commented on pull request #9598: URL: https://github.com/apache/pulsar/pull/9598#issuecomment-822542634 Some points are fixed in this pr https://github.com/apache/pulsar/pull/9900 . Please take a look @WJL -- This is an automated message from the Apache Git Service. To

[GitHub] [pulsar] codelipenghui commented on issue #9962: [Tiered Storage] Eagerly Delete Offloaded Segments On Topic Deletion

2021-04-19 Thread GitBox
codelipenghui commented on issue #9962: URL: https://github.com/apache/pulsar/issues/9962#issuecomment-822541460 ping @zymap Could you please help take a look at this issue since you are working on tiered storage for now. -- This is an automated message from the Apache Git Service. To

[GitHub] [pulsar] codelipenghui commented on pull request #9598: Make `LocalPolicies` immutable to avoid concurrent modify inconsistent.

2021-04-19 Thread GitBox
codelipenghui commented on pull request #9598: URL: https://github.com/apache/pulsar/pull/9598#issuecomment-822538472 @WJL You can rebase to the current master branch to check if the CI can get passed. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pulsar] Jennifer88huang commented on pull request #10256: Issue 10236: [Doc] generate 2.7.2 docs

2021-04-19 Thread GitBox
Jennifer88huang commented on pull request #10256: URL: https://github.com/apache/pulsar/pull/10256#issuecomment-822530646 @Anonymitaet No, it's not needed. According to the workflow, this PR should have been submitted immediately after 2.7.1 release, so he follows the workflow for minor

[GitHub] [pulsar] codelipenghui commented on pull request #6364: Creating a topic does not wait for creating cursor of replicators

2021-04-19 Thread GitBox
codelipenghui commented on pull request #6364: URL: https://github.com/apache/pulsar/pull/6364#issuecomment-822525432 @merlimat Ok, make sense. I think we can only wait for the replicator cursors when the subscription of the replicator has not created yet? If the subscription of the

[pulsar] branch master updated: [doc] Support generate document automatically for pulsar-perf (#10120)

2021-04-19 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new b33aca9 [doc] Support generate document

[GitHub] [pulsar] codelipenghui merged pull request #10120: [doc] Support generate document automatically for pulsar-perf

2021-04-19 Thread GitBox
codelipenghui merged pull request #10120: URL: https://github.com/apache/pulsar/pull/10120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [pulsar] codelipenghui commented on issue #10042: refresh delay time for "Delayed message delivery"

2021-04-19 Thread GitBox
codelipenghui commented on issue #10042: URL: https://github.com/apache/pulsar/issues/10042#issuecomment-822521720 @hoswey Yes, it's not efficient for handling this case but seems this is only a way to achieve the purpose. Supports update delay time of the message will introduce more

[pulsar] branch master updated: [ISSUE 10153][Go Function] fix time unit ns -> ms (#10160)

2021-04-19 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new e9bf54a [ISSUE 10153][Go Function] fix time

[GitHub] [pulsar] codelipenghui closed issue #10153: [Go functions] Very high (abnormal) average process latency

2021-04-19 Thread GitBox
codelipenghui closed issue #10153: URL: https://github.com/apache/pulsar/issues/10153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [pulsar] codelipenghui merged pull request #10160: [ISSUE 10153]fix time unit ns -> ms

2021-04-19 Thread GitBox
codelipenghui merged pull request #10160: URL: https://github.com/apache/pulsar/pull/10160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[pulsar] branch master updated: [Documentation] Fix typo: maxPendingChuckedMessage should be maxPendingChunkedMessage (#10253)

2021-04-19 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 8b46d59 [Documentation] Fix typo:

[GitHub] [pulsar] codelipenghui merged pull request #10253: [Documentation] Fix typo: maxPendingChuckedMessage should be maxPendingChunkedMessage

2021-04-19 Thread GitBox
codelipenghui merged pull request #10253: URL: https://github.com/apache/pulsar/pull/10253 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [pulsar] adrianog commented on issue #6350: Reader.startMessageIdInclusive() is not working

2021-04-19 Thread GitBox
adrianog commented on issue #6350: URL: https://github.com/apache/pulsar/issues/6350#issuecomment-822507954 Is startMessageIdInclusive available in python? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pulsar] Anonymitaet edited a comment on pull request #10256: Issue 10236: [Doc] generate 2.7.2 docs

2021-04-19 Thread GitBox
Anonymitaet edited a comment on pull request #10256: URL: https://github.com/apache/pulsar/pull/10256#issuecomment-822506632 @Jennifer88huang 1. Does this PR need to update [`releases.json`](https://github.com/apache/pulsar/blob/master/site2/website/releases.json)? (according to

[GitHub] [pulsar] Anonymitaet commented on pull request #10256: Issue 10236: [Doc] generate 2.7.2 docs

2021-04-19 Thread GitBox
Anonymitaet commented on pull request #10256: URL: https://github.com/apache/pulsar/pull/10256#issuecomment-822506632 @Jennifer88huang For this PR: 1. Does this PR need to update [`releases.json`](https://github.com/apache/pulsar/blob/master/site2/website/releases.json)?

[GitHub] [pulsar] adrianog opened a new issue #10268: Retention to be based on number of messages

2021-04-19 Thread GitBox
adrianog opened a new issue #10268: URL: https://github.com/apache/pulsar/issues/10268 **Is your enhancement request related to a problem? Please describe.** Currently I can only specify the size and time for retention. **Describe the solution you'd like** I would like pulsar

[GitHub] [pulsar] congbobo184 commented on a change in pull request #10257: Fix transaction buffer lookup

2021-04-19 Thread GitBox
congbobo184 commented on a change in pull request #10257: URL: https://github.com/apache/pulsar/pull/10257#discussion_r615890305 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TransactionBufferHandlerImpl.java ## @@ -18,53 +18,68 @@

[GitHub] [pulsar] congbobo184 commented on a change in pull request #10254: [Broker] Remove possible error case for system topic name checking

2021-04-19 Thread GitBox
congbobo184 commented on a change in pull request #10254: URL: https://github.com/apache/pulsar/pull/10254#discussion_r615871300 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/systopic/SystemTopicClient.java ## @@ -167,8 +167,11 @@

[GitHub] [pulsar] congbobo184 commented on a change in pull request #10254: [Broker] Remove possible error case for system topic name checking

2021-04-19 Thread GitBox
congbobo184 commented on a change in pull request #10254: URL: https://github.com/apache/pulsar/pull/10254#discussion_r615871300 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/systopic/SystemTopicClient.java ## @@ -167,8 +167,11 @@

[GitHub] [pulsar] linlinnn removed a comment on pull request #10240: [consumer] fix order guarantee for MultiTopicsConsumerImpl

2021-04-19 Thread GitBox
linlinnn removed a comment on pull request #10240: URL: https://github.com/apache/pulsar/pull/10240#issuecomment-822426467 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pulsar] congbobo184 commented on a change in pull request #10254: [Broker] Remove possible error case for system topic name checking

2021-04-19 Thread GitBox
congbobo184 commented on a change in pull request #10254: URL: https://github.com/apache/pulsar/pull/10254#discussion_r615854206 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java ## @@ -643,7 +643,8 @@ protected void

[pulsar] branch master updated: [Security] Upgrade vertx to 3.9.7, addresses CVE-2018-12541 (#10261)

2021-04-19 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 0ce0bcc [Security] Upgrade vertx to 3.9.7,

[GitHub] [pulsar] eolivelli merged pull request #10261: [Security] Upgrade vertx to 3.9.7, addresses CVE-2018-12541

2021-04-19 Thread GitBox
eolivelli merged pull request #10261: URL: https://github.com/apache/pulsar/pull/10261 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [pulsar] eolivelli opened a new pull request #10267: Add documentation about Sink and Source

2021-04-19 Thread GitBox
eolivelli opened a new pull request #10267: URL: https://github.com/apache/pulsar/pull/10267 Add documentation about recently implemented features in Pulsar IO regarding: - Sink: Sinks that are not aware (at build time)of the schema type - Source: Sources that are not aware (at build

[GitHub] [pulsar] codelipenghui commented on issue #8214: Consumer pause does not work properly in case of multi topics consumers.

2021-04-19 Thread GitBox
codelipenghui commented on issue #8214: URL: https://github.com/apache/pulsar/issues/8214#issuecomment-822460843 Cool @sandrzejczak, looking forward to your PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pulsar] MarvinCai commented on issue #10263: Cannot delete non empty bundle

2021-04-19 Thread GitBox
MarvinCai commented on issue #10263: URL: https://github.com/apache/pulsar/issues/10263#issuecomment-822458610 Seems to be metadata syncing issue, the namespace service read metadata about topic from a cache, while the topic deletion operation finished and updated metadata in zookeeper,

[pulsar] branch master updated: [Security] Remove jackson-mapper-asl dependency to resolve multiple CVEs (#10262)

2021-04-19 Thread penghui
This is an automated email from the ASF dual-hosted git repository. penghui pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new e52a9dc [Security] Remove jackson-mapper-asl

[GitHub] [pulsar] codelipenghui merged pull request #10262: [Security] Remove jackson-mapper-asl dependency to resolve multiple CVEs

2021-04-19 Thread GitBox
codelipenghui merged pull request #10262: URL: https://github.com/apache/pulsar/pull/10262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [pulsar] lhotari commented on pull request #10266: [Security] Upgrade Netty to 4.1.63.Final to address CVE-2021-21409

2021-04-19 Thread GitBox
lhotari commented on pull request #10266: URL: https://github.com/apache/pulsar/pull/10266#issuecomment-822442361 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pulsar] lhotari edited a comment on pull request #10261: [Security] Upgrade vertx to 3.9.7, addresses CVE-2018-12541

2021-04-19 Thread GitBox
lhotari edited a comment on pull request #10261: URL: https://github.com/apache/pulsar/pull/10261#issuecomment-822420039 > can you please explain why we can upgrade without upgrading BK ? > > it looks like there is an API compatibility issue here > >

[GitHub] [pulsar] lhotari commented on pull request #10260: Upgrade k8s java-client to fix dependency resolution

2021-04-19 Thread GitBox
lhotari commented on pull request #10260: URL: https://github.com/apache/pulsar/pull/10260#issuecomment-822436160 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pulsar] adrianog removed a comment on issue #6523: Consume the last N messages from Pulsar

2021-04-19 Thread GitBox
adrianog removed a comment on issue #6523: URL: https://github.com/apache/pulsar/issues/6523#issuecomment-822034129 @jiazhai could you possibly expand on how to go about this. I'm not clear from your explanation. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [pulsar] linlinnn commented on pull request #10240: [consumer] fix order guarantee for MultiTopicsConsumerImpl

2021-04-19 Thread GitBox
linlinnn commented on pull request #10240: URL: https://github.com/apache/pulsar/pull/10240#issuecomment-822426467 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pulsar] lhotari opened a new pull request #10266: [Security] Upgrade Netty to 4.1.63.Final to address CVE-2021-21409

2021-04-19 Thread GitBox
lhotari opened a new pull request #10266: URL: https://github.com/apache/pulsar/pull/10266 ### Motivation Upgrade Netty to 4.1.63.Final to address CVE-2021-21409 . ### Modifications - Upgrade Netty to 4.1.63.Final - Upgrade netty-tcnative-boringssl-static to

[pulsar] branch master updated: Java Client: MessageImpl - ensure that AutoConsumeSchema downloaded the schema before decoding the payload (#10248)

2021-04-19 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 54523bb Java Client: MessageImpl - ensure

[GitHub] [pulsar] eolivelli commented on pull request #10248: Java Client: MessageImpl - ensure that AutoConsumeSchema downloaded the schema before decoding the payload

2021-04-19 Thread GitBox
eolivelli commented on pull request #10248: URL: https://github.com/apache/pulsar/pull/10248#issuecomment-822422238 thank you @congbobo184 @codelipenghui @lhotari for your review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [pulsar] eolivelli merged pull request #10248: Java Client: MessageImpl - ensure that AutoConsumeSchema downloaded the schema before decoding the payload

2021-04-19 Thread GitBox
eolivelli merged pull request #10248: URL: https://github.com/apache/pulsar/pull/10248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [pulsar] lhotari commented on pull request #10261: [Security] Upgrade vertx to 3.9.7, addresses CVE-2018-12541

2021-04-19 Thread GitBox
lhotari commented on pull request #10261: URL: https://github.com/apache/pulsar/pull/10261#issuecomment-822420039 > can you please explain why we can upgrade without upgrading BK ? > > it looks like there is an API compatibility issue here > >

[pulsar] branch master updated (a66dd5d -> 82c5b810)

2021-04-19 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from a66dd5d Issue #6054 - Adding more permits debug statements to better see changes to permits (#10217) add

[GitHub] [pulsar] eolivelli merged pull request #10259: Upgrade Lombok to 1.18.20

2021-04-19 Thread GitBox
eolivelli merged pull request #10259: URL: https://github.com/apache/pulsar/pull/10259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [pulsar] lhotari commented on pull request #10262: [Security] Remove jackson-mapper-asl dependency to resolve multiple CVEs

2021-04-19 Thread GitBox
lhotari commented on pull request #10262: URL: https://github.com/apache/pulsar/pull/10262#issuecomment-822400719 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pulsar] lhotari commented on pull request #10260: Upgrade k8s java-client to fix dependency resolution

2021-04-19 Thread GitBox
lhotari commented on pull request #10260: URL: https://github.com/apache/pulsar/pull/10260#issuecomment-822398417 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pulsar] lhotari commented on pull request #10261: [Security] Upgrade vertx to 3.9.7, addresses CVE-2018-12541

2021-04-19 Thread GitBox
lhotari commented on pull request #10261: URL: https://github.com/apache/pulsar/pull/10261#issuecomment-822398082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [pulsar] lhotari commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues

2021-04-19 Thread GitBox
lhotari commented on pull request #10217: URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822397831 > Looks like we missed a flaky test. @lhotari > org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest created 1 new threads > 2021-04-17 20:21:28,701 - INFO

[GitHub] [pulsar] eolivelli commented on pull request #10248: Java Client: MessageImpl - ensure that AutoConsumeSchema downloaded the schema before decoding the payload

2021-04-19 Thread GitBox
eolivelli commented on pull request #10248: URL: https://github.com/apache/pulsar/pull/10248#issuecomment-822395574 this patch also fixes a problem in the integration tests that made CI more flaky. as soon as we commit this patch CI will be in better state cc @congbobo184

[GitHub] [pulsar] BewareMyPower commented on pull request #10265: [Python schema] Support python avro schema set default value.

2021-04-19 Thread GitBox
BewareMyPower commented on pull request #10265: URL: https://github.com/apache/pulsar/pull/10265#issuecomment-822394008 It looks like there're still some problems with the implementation. ``` File "/tmp/schema_test.py", line 810, in test_avro_required_default "default":

[GitHub] [pulsar] lhotari commented on pull request #10262: [Security] Remove jackson-mapper-asl dependency to resolve multiple CVEs

2021-04-19 Thread GitBox
lhotari commented on pull request #10262: URL: https://github.com/apache/pulsar/pull/10262#issuecomment-822364976 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pulsar] 315157973 removed a comment on pull request #10190: Fix usage of seekAsync in MessageImpl.hasMessageAvailable and flaky-test

2021-04-19 Thread GitBox
315157973 removed a comment on pull request #10190: URL: https://github.com/apache/pulsar/pull/10190#issuecomment-820571040 Indeed, if we put the seek in this place, at this time, there is a high probability that the connection is not ready, causing the seek to fail, and eventually this

[GitHub] [pulsar] 315157973 commented on pull request #10190: Fix usage of seekAsync in MessageImpl.hasMessageAvailable and flaky-test

2021-04-19 Thread GitBox
315157973 commented on pull request #10190: URL: https://github.com/apache/pulsar/pull/10190#issuecomment-822341543 I tried to handle this situation in connectionOpen, but failed: 1 The first time call hasMessageAvailable, it will get the last msg in broker, and then trigger seek

[pulsar] branch branch-2.7 updated: Issue #6054 - Adding more permits debug statements to better see changes to permits (#10217)

2021-04-19 Thread eolivelli
This is an automated email from the ASF dual-hosted git repository. eolivelli pushed a commit to branch branch-2.7 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.7 by this push: new c243d18 Issue #6054 - Adding more

[GitHub] [pulsar] eolivelli commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues

2021-04-19 Thread GitBox
eolivelli commented on pull request #10217: URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822335349 @devinbost picked to branch-2.7 as c243d185261e964010a122b65f9c730bffbb2f0c -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pulsar] congbobo184 commented on pull request #10265: [Python schema] Support python avro schema set default value.

2021-04-19 Thread GitBox
congbobo184 commented on pull request #10265: URL: https://github.com/apache/pulsar/pull/10265#issuecomment-822333726 @BewareMyPower could you please review this pr? thanks :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [pulsar] sandrzejczak edited a comment on issue #8214: Consumer pause does not work properly in case of multi topics consumers.

2021-04-19 Thread GitBox
sandrzejczak edited a comment on issue #8214: URL: https://github.com/apache/pulsar/issues/8214#issuecomment-822313437 It seems that the bug described in this issue has not been fixed. The description mentions the case where consumers for new topics are discovered/added.

[GitHub] [pulsar] congbobo184 opened a new pull request #10265: [Python schema] Support python avro schema set default value.

2021-04-19 Thread GitBox
congbobo184 opened a new pull request #10265: URL: https://github.com/apache/pulsar/pull/10265 ## Motivation now python avro schema don't support set default value, it led to the python schema can't update. ## implement 1. add `required` field to control the type of schema can set

[GitHub] [pulsar] eolivelli opened a new issue #10264: Java Client: Provide PulsarAdminBuilder.loadConf as for PulsarClientBuilder

2021-04-19 Thread GitBox
eolivelli opened a new issue #10264: URL: https://github.com/apache/pulsar/issues/10264 Currently (Pulsar 2.7.x) it is very awkward to build an AdminClient instance from a configuration file. We already have a useful PulsarClientBuilder#loadConf method that allows you to pass named

[GitHub] [pulsar] frank-montyne commented on issue #10231: Exception using topic level policies

2021-04-19 Thread GitBox
frank-montyne commented on issue #10231: URL: https://github.com/apache/pulsar/issues/10231#issuecomment-822326642 I created a new issue that more specifically pinpoints the issue #10263 -- This is an automated message from the Apache Git Service. To respond to the message, please

  1   2   >