[GitHub] [pulsar] wolfstudy edited a comment on issue #5324: [go client] Producer::Close() may get stuck
wolfstudy edited a comment on issue #5324: [go client] Producer::Close() may get stuck URL: https://github.com/apache/pulsar/issues/5324#issuecomment-540932462 The issue has been fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5324: [go client] Producer::Close() may get stuck
wolfstudy commented on issue #5324: [go client] Producer::Close() may get stuck URL: https://github.com/apache/pulsar/issues/5324#issuecomment-540932462 The issue has been fixed。 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy closed issue #5324: [go client] Producer::Close() may get stuck
wolfstudy closed issue #5324: [go client] Producer::Close() may get stuck URL: https://github.com/apache/pulsar/issues/5324 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf
wolfstudy commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf URL: https://github.com/apache/pulsar/pull/5359#issuecomment-540930771 run java8 tests run cpp tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-540918400 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5347: [Docs]Fix website io connector link 404
tuteng commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540917631 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic
wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic URL: https://github.com/apache/pulsar-client-go/pull/61#issuecomment-540916004 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic
wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic URL: https://github.com/apache/pulsar-client-go/pull/61#issuecomment-540914171 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy removed a comment on issue #61: [issue:60] Fix partition topic message router logic
wolfstudy removed a comment on issue #61: [issue:60] Fix partition topic message router logic URL: https://github.com/apache/pulsar-client-go/pull/61#issuecomment-540913294 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy removed a comment on issue #61: [issue:60] Fix partition topic message router logic
wolfstudy removed a comment on issue #61: [issue:60] Fix partition topic message router logic URL: https://github.com/apache/pulsar-client-go/pull/61#issuecomment-540912249 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic
wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic URL: https://github.com/apache/pulsar-client-go/pull/61#issuecomment-540913762 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic
wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic URL: https://github.com/apache/pulsar-client-go/pull/61#issuecomment-540913294 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] addisonj opened a new pull request #5361: [proxy] Fix proxy to be able to re-send request body
addisonj opened a new pull request #5361: [proxy] Fix proxy to be able to re-send request body URL: https://github.com/apache/pulsar/pull/5361 Fixes #5360 This adds a small cache of the request body to ensure that it can be re-sent. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5233: Update full-width typo in schema-understand.md
codelipenghui commented on issue #5233: Update full-width typo in schema-understand.md URL: https://github.com/apache/pulsar/pull/5233#issuecomment-540913107 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic
wolfstudy commented on issue #61: [issue:60] Fix partition topic message router logic URL: https://github.com/apache/pulsar-client-go/pull/61#issuecomment-540912249 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] addisonj commented on issue #5360: Proxy doesn't send request body on redirects
addisonj commented on issue #5360: Proxy doesn't send request body on redirects URL: https://github.com/apache/pulsar/issues/5360#issuecomment-540902533 I have it implemented, going to to try it out and will push it up This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5360: Proxy doesn't send request body on redirects
merlimat commented on issue #5360: Proxy doesn't send request body on redirects URL: https://github.com/apache/pulsar/issues/5360#issuecomment-540898884 Nice finding! Seems similar to the fix for #4059. In that commit I had added a test to verify the auth headers were being propagated after a redirect, though I didn't think about testing with a request body. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] addisonj commented on issue #5360: Proxy doesn't send request body on redirects
addisonj commented on issue #5360: Proxy doesn't send request body on redirects URL: https://github.com/apache/pulsar/issues/5360#issuecomment-540895466 Okay, one more follow up. It seems like the default ProxyServlet provides its own ContentProvider, https://github.com/eclipse/jetty.project/blob/jetty-9.4.12.v20180830/jetty-proxy/src/main/java/org/eclipse/jetty/proxy/ProxyServlet.java#L244, that is a really thing wrapper over this class: https://github.com/eclipse/jetty.project/blob/jetty-9.4.12.v20180830/jetty-client/src/main/java/org/eclipse/jetty/client/util/InputStreamContentProvider.java#L37. As can be seen from the doc to the InputStreamContentProvider, the input stream can only be consumed once, additionally, no buffering is done by the proxied stream, that would seem to explain why the content can be replayed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] borlandor closed issue #5136: Java Function cannot work with any JSON parser JAR
borlandor closed issue #5136: Java Function cannot work with any JSON parser JAR URL: https://github.com/apache/pulsar/issues/5136 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] addisonj opened a new issue #5360: Proxy doesn't send request body on redirects
addisonj opened a new issue #5360: Proxy doesn't send request body on redirects
URL: https://github.com/apache/pulsar/issues/5360
**Describe the bug**
When using multiple brokers behind a proxy, requests with a request body
from the client fail. See #3702 as an example. The following seems to be what
happens:
- The client makes a request to proxy
- The proxy forward the requests to the broker
- The broker is not authoritative for that request, so it responds with a
redirect
- The proxy follows the redirect but DOES NOT forward on the original
request body
- The second broker, seeing a content-length header but never getting a
body, waits for a full response but then times out
- The proxy responds with a 504 gateway timeout
**To Reproduce**
Steps to reproduce the behavior:
1. Run multiple brokers
2. Run a proxy in front of the brokers
3. make an admin request that takes a request body and must be routed to a
specific broker, like creating a subscription to a specific ledger id (a
default subscription doesn't send a request body)
4. Try a few times, you should eventually get a gateway timeout
This appears to be the case with and without TLS. I haven't yet tried it
without authentication
**Expected behavior**
The request should succeed, with the request body being sent with the
redirect
**Screenshots**
I validate this with tcp dumps:
The initial request to the first broker:
```
PUT
/admin/v2/persistent/code/gerrit-events/message_bus%3Aref-updated/subscription/flink-pulsar-2a7cc468-6712-44b9-8b8f-0c8b86b78dae
HTTP/1.1
Authorization: Bearer ...
User-Agent: Pulsar-Java-v2.4.1
Host: pulsar-pdx.bus-beta.insk8s.net:8443
Accept: application/json
Content-Type: application/json
Via: http/1.1 pulsar-beta-proxy-6b87d7f8cc-thmmp
X-Forwarded-For: 10.11.62.69
X-Forwarded-Proto: https
X-Forwarded-Host: pulsar-pdx.bus-beta.insk8s.net:8443
X-Forwarded-Server: 10.11.61.62
X-Original-Principal: code-admin
Content-Length: 82
{"ledgerId":9223372036854775807,"entryId":9223372036854775807,"partitionIndex":-1}
HTTP/1.1 307 Temporary Redirect
Date: Thu, 10 Oct 2019 23:11:41 GMT
Location:
http://10.11.41.134:8080/admin/v2/persistent/code/gerrit-events/message_bus%3Aref-updated/subscription/flink-pulsar-2a7cc468-6712-44b9-8b8f-0c8b86b78dae?authoritative=false
broker-address: 10.11.54.198
Content-Length: 0
Server: Jetty(9.4.12.v20180830)
```
The redirect to the second broker (from the proxy)
```
PUT
/admin/v2/persistent/code/gerrit-events/message_bus%3Aref-updated/subscription/flink-pulsar-2a7cc468-6712-44b9-8b8f-0c8b86b78dae?authoritative=false
HTTP/1.1
User-Agent: Jetty/9.4.12.v20180830
User-Agent: Pulsar-Java-v2.4.1
Accept: application/json
Content-Type: application/json
Via: http/1.1 pulsar-beta-proxy-6b87d7f8cc-thmmp
X-Forwarded-For: 10.11.62.69
X-Forwarded-Proto: https
X-Forwarded-Host: pulsar-pdx.bus-beta.insk8s.net:8443
X-Forwarded-Server: 10.11.61.62
X-Original-Principal: code-admin
Authorization: Bearer ...
Host: 10.11.41.134:8080
Content-Length: 82
HTTP/1.1 500 Request failed.
Date: Thu, 10 Oct 2019 23:11:41 GMT
Content-Length: 0
Server: Jetty(9.4.12.v20180830)
```
Note that the content-length is set but no body is sent.
**Desktop (please complete the following information):**
Occurs with Pulsar 2.4.1 on linux
**Additional context**
I have spent a fair amount of time trying to debug this, here are some known
details:
In `AdminProxyHandler`, we copy the request, see
https://github.com/apache/pulsar/blob/v2.4.1/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/AdminProxyHandler.java#L152
The super method here,
https://github.com/eclipse/jetty.project/blob/jetty-9.4.12.v20180830/jetty-client/src/main/java/org/eclipse/jetty/client/HttpClient.java#L459,
copies the content.
However, the contentProvider may be an iterator, that depending on the
implementation, can't be reset. It isn't clear what implementation would be
used by the proxy.
Additionally, this is made more difficult as it seems like the admin proxy
isn't logging as expected. With debug level logs turned on, I didn't see logs
from the proxy for some code paths I am sure it should be in.
I think the next step might be to have the AdminProxyHandler consume the
request body before sending the first request and then re-attaching it for the
first and any subsequent proxied requests
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540880226 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540880226 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #4811: [Issue: 3161][Go client] Return Message ID for send
wolfstudy commented on issue #4811: [Issue: 3161][Go client] Return Message ID for send URL: https://github.com/apache/pulsar/pull/4811#issuecomment-540878424 ping @massakam @merlimat PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5233: Update full-width typo in schema-understand.md
codelipenghui commented on issue #5233: Update full-width typo in schema-understand.md URL: https://github.com/apache/pulsar/pull/5233#issuecomment-540877633 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5317: Allow to specify delivery delay in C++ client
codelipenghui commented on issue #5317: Allow to specify delivery delay in C++ client URL: https://github.com/apache/pulsar/pull/5317#issuecomment-540877439 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf
codelipenghui commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf URL: https://github.com/apache/pulsar/pull/5359#issuecomment-540877353 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf
codelipenghui commented on issue #5359: Set default ensemble size to 2 in service conf, to match broker.conf URL: https://github.com/apache/pulsar/pull/5359#issuecomment-540871582 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5317: Allow to specify delivery delay in C++ client
codelipenghui commented on issue #5317: Allow to specify delivery delay in C++ client URL: https://github.com/apache/pulsar/pull/5317#issuecomment-540871400 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-540868810 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-540870220 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540869577 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540869545 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540868911 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540869577 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540869545 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (daa546d -> 47d877f)
This is an automated email from the ASF dual-hosted git repository.
hjf pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from daa546d Not allow use acknowledgeCumulative on Key_shared
subscription. (#5339)
add 47d877f Add *Rabbit sink connector guide* (#5290)
No new revisions were added by this update.
Summary of changes:
.../{io-rabbitmq-source.md => io-rabbitmq-sink.md} | 31 --
1 file changed, 17 insertions(+), 14 deletions(-)
copy site2/docs/{io-rabbitmq-source.md => io-rabbitmq-sink.md} (70%)
[GitHub] [pulsar] Jennifer88huang merged pull request #5290: [Doc] Add *RabbitMQ sink connector guide*
Jennifer88huang merged pull request #5290: [Doc] Add *RabbitMQ sink connector guide* URL: https://github.com/apache/pulsar/pull/5290 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540563708 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540868911 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-540563453 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-540868810 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5347: [Docs]Fix website io connector link 404
tuteng commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540849121 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat opened a new pull request #5359: Set default ensemble size to 2 in service conf, to match broker.conf
merlimat opened a new pull request #5359: Set default ensemble size to 2 in service conf, to match broker.conf URL: https://github.com/apache/pulsar/pull/5359 ### Motivation The values for `managedLedgerDefaultEnsembleSize`, `managedLedgerDefaultWriteQuorum` and `managedLedgerDefaultAckQuorum` are set to `1-1-1` in `ServiceConfiguration`. The actual default is set to `2-2-2` in `broker.conf`. These default settings should be matching. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide*
aahmed-se commented on issue #5290: [Doc] Add *RabbitMQ sink connector guide* URL: https://github.com/apache/pulsar/pull/5290#issuecomment-540699610 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat opened a new pull request #5358: PIP-45: Switch ManagedLedger to use MetadataStore interface
merlimat opened a new pull request #5358: PIP-45: Switch ManagedLedger to use MetadataStore interface URL: https://github.com/apache/pulsar/pull/5358 ### Motivation Converted Managed Ledger to use the MetadataStore interface. Note: this doesn't yet address the instantiation of `MetadataStore`, rather just creates a `ZKMetadataStore` instance. The instantiation plumbing will be completed once all the other parts are already in place. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5327: typo: traffic
merlimat commented on issue #5327: typo: traffic URL: https://github.com/apache/pulsar/pull/5327#issuecomment-540643053 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #5339: Not allow use acknowledgeCumulative on Key_shared subscription.
merlimat merged pull request #5339: Not allow use acknowledgeCumulative on Key_shared subscription. URL: https://github.com/apache/pulsar/pull/5339 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (5c4c7b7 -> daa546d)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 5c4c7b7 [Doc] Add *Kinesis source connector guide* (#5211) add daa546d Not allow use acknowledgeCumulative on Key_shared subscription. (#5339) No new revisions were added by this update. Summary of changes: .../client/api/KeySharedSubscriptionTest.java | 27 +++ pulsar-client-cpp/include/pulsar/Result.h | 2 + pulsar-client-cpp/lib/ConsumerImpl.cc | 8 pulsar-client-cpp/lib/ConsumerImpl.h | 1 + pulsar-client-cpp/lib/Result.cc| 3 ++ pulsar-client-cpp/tests/BasicEndToEndTest.cc | 55 ++ .../apache/pulsar/client/impl/ConsumerBase.java| 4 +- 7 files changed, 98 insertions(+), 2 deletions(-)
[GitHub] [pulsar] merlimat commented on issue #5317: Allow to specify delivery delay in C++ client
merlimat commented on issue #5317: Allow to specify delivery delay in C++ client URL: https://github.com/apache/pulsar/pull/5317#issuecomment-540636431 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5320: Expose metrics of Caffeine caches
merlimat commented on issue #5320: Expose metrics of Caffeine caches URL: https://github.com/apache/pulsar/pull/5320#issuecomment-540636325 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5347: [Docs]Fix website io connector link 404
merlimat commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540630487 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #5211: [Doc] Add *Kinesis source connector guide*
merlimat merged pull request #5211: [Doc] Add *Kinesis source connector guide* URL: https://github.com/apache/pulsar/pull/5211 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (3e70d36 -> 5c4c7b7)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 3e70d36 [TIEREDSTORAGE] Don't require both region and endpoint to be specified (#5355) add 5c4c7b7 [Doc] Add *Kinesis source connector guide* (#5211) No new revisions were added by this update. Summary of changes: site2/docs/io-kinesis-source.md | 92 +++-- 1 file changed, 62 insertions(+), 30 deletions(-)
[GitHub] [pulsar] merlimat commented on issue #4158: Create more offloader indices per each block uploaded to 2nd tier
merlimat commented on issue #4158: Create more offloader indices per each block uploaded to 2nd tier URL: https://github.com/apache/pulsar/pull/4158#issuecomment-540629970 Closing this in favor of #5356 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat closed pull request #4158: Create more offloader indices per each block uploaded to 2nd tier
merlimat closed pull request #4158: Create more offloader indices per each block uploaded to 2nd tier URL: https://github.com/apache/pulsar/pull/4158 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #5355: [TIEREDSTORAGE] Don't require both region and endpoint to be specified
merlimat merged pull request #5355: [TIEREDSTORAGE] Don't require both region and endpoint to be specified URL: https://github.com/apache/pulsar/pull/5355 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [TIEREDSTORAGE] Don't require both region and endpoint to be specified (#5355)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 3e70d36 [TIEREDSTORAGE] Don't require both region and endpoint to be
specified (#5355)
3e70d36 is described below
commit 3e70d365fbef6d219bec2386afa8f962b6cd12fb
Author: Ivan Kelly
AuthorDate: Thu Oct 10 16:00:11 2019 +0100
[TIEREDSTORAGE] Don't require both region and endpoint to be specified
(#5355)
There's a bug in how user metadata is attached to a block that if the
user doesn't specify both the region and the endpoint, offloading will
throw an exception, as you can't add a null value to an immutable
map.
This change elides null to the empty string in these cases, so that
offloading can continue.
---
.../org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java | 1 +
.../mledger/offload/jcloud/impl/BlobStoreManagedLedgerOffloader.java | 4 ++--
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
index 0545274..94412a9 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
@@ -1630,6 +1630,7 @@ public class PersistentTopicsBase extends AdminResource {
} catch (AlreadyRunningException e) {
throw new RestException(Status.CONFLICT, e.getMessage());
} catch (Exception e) {
+log.warn("Unexpected error triggering offload", e);
throw new RestException(e);
}
}
diff --git
a/tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/BlobStoreManagedLedgerOffloader.java
b/tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/BlobStoreManagedLedgerOffloader.java
index 38fe880..7f7acaf 100644
---
a/tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/BlobStoreManagedLedgerOffloader.java
+++
b/tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/BlobStoreManagedLedgerOffloader.java
@@ -382,8 +382,8 @@ public class BlobStoreManagedLedgerOffloader implements
LedgerOffloader {
public Map getOffloadDriverMetadata() {
return ImmutableMap.of(
METADATA_FIELD_BUCKET, writeBucket,
-METADATA_FIELD_REGION, writeRegion,
-METADATA_FIELD_ENDPOINT, writeEndpoint
+METADATA_FIELD_REGION, Strings.nullToEmpty(writeRegion),
+METADATA_FIELD_ENDPOINT, Strings.nullToEmpty(writeEndpoint)
);
}
[GitHub] [pulsar] ivankelly commented on a change in pull request #5356: [TIEREDSTORAGE] Only seek when reading unexpected entry
ivankelly commented on a change in pull request #5356: [TIEREDSTORAGE] Only
seek when reading unexpected entry
URL: https://github.com/apache/pulsar/pull/5356#discussion_r333546836
##
File path:
tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/BlobStoreBackedInputStreamImpl.java
##
@@ -116,7 +116,7 @@ public int read(byte[] b, int off, int len) throws
IOException {
@Override
public void seek(long position) {
-log.debug("Seeking to {} on {}/{}, current position {}", position,
bucket, key, cursor);
+log.info("Seeking to {} on {}/{}, current position {} (bufStart:{},
bufEnd:{})", position, bucket, key, cursor, bufferOffsetStart, bufferOffsetEnd);
Review comment:
ah yes, should be debug
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540563708 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 commented on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-540563453 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
codelipenghui commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#issuecomment-540535113 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
codelipenghui commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#issuecomment-540535070 @merlimat Please help review this PR, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540533635 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540533586 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540533729 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540533729 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540533586 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #4847: Pulsar sql avro support schema version
congbobo184 commented on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540533635 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5317: Allow to specify delivery delay in C++ client
codelipenghui commented on issue #5317: Allow to specify delivery delay in C++ client URL: https://github.com/apache/pulsar/pull/5317#issuecomment-540532842 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5328: Increase the retries count for PulsarFunctionStateTest
codelipenghui commented on issue #5328: Increase the retries count for PulsarFunctionStateTest URL: https://github.com/apache/pulsar/pull/5328#issuecomment-540532683 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5331: Avoid rethrowing exception during test teardown
codelipenghui commented on issue #5331: Avoid rethrowing exception during test teardown URL: https://github.com/apache/pulsar/pull/5331#issuecomment-540532534 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5333: Do not use timed receive in BrokerClientIntegrationTest.testUnsupportedBatchMessageConsumer
codelipenghui commented on issue #5333: Do not use timed receive in BrokerClientIntegrationTest.testUnsupportedBatchMessageConsumer URL: https://github.com/apache/pulsar/pull/5333#issuecomment-540532339 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5342: Use simple GC settings when running integration tests
codelipenghui commented on issue #5342: Use simple GC settings when running integration tests URL: https://github.com/apache/pulsar/pull/5342#issuecomment-540531856 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5357: [functions][Issue:5350]Fix pulsar can't load the customized SerDe
wolfstudy commented on issue #5357: [functions][Issue:5350]Fix pulsar can't load the customized SerDe URL: https://github.com/apache/pulsar/pull/5357#issuecomment-540532129 @sijie @jerrypeng PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5339: Not allow use acknowledgeCumulative on Key_shared subscription.
codelipenghui commented on issue #5339: Not allow use acknowledgeCumulative on Key_shared subscription. URL: https://github.com/apache/pulsar/pull/5339#issuecomment-540532101 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy opened a new pull request #5357: [functions][Issue:5350]Fix pulsar can't load the customized SerDe
wolfstudy opened a new pull request #5357: [functions][Issue:5350]Fix pulsar can't load the customized SerDe URL: https://github.com/apache/pulsar/pull/5357 Signed-off-by: xiaolong.ran *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes #5350 ### Motivation When using the `--output-serde-classname` option, Pulsar Function used a wrong class loader to load the serde class. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (e8adb0a -> dbd48ab)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e8adb0a Build python 37 artifacts and use them for the docker image (#5335) add dbd48ab Fix bad_weak_ptr error when closing producer (#5315) No new revisions were added by this update. Summary of changes: pulsar-client-cpp/lib/ConsumerImpl.cc | 10 +++--- pulsar-client-cpp/lib/ConsumerImpl.h | 2 +- pulsar-client-cpp/lib/ProducerImpl.cc | 25 + pulsar-client-cpp/lib/ProducerImpl.h | 2 +- 4 files changed, 26 insertions(+), 13 deletions(-)
[GitHub] [pulsar] sijie merged pull request #5315: [pulsar-client-cpp] Fix bad_weak_ptr error when closing producer
sijie merged pull request #5315: [pulsar-client-cpp] Fix bad_weak_ptr error when closing producer URL: https://github.com/apache/pulsar/pull/5315 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #5335: Build python 37 artifacts and use them for the docker image
sijie merged pull request #5335: Build python 37 artifacts and use them for the docker image URL: https://github.com/apache/pulsar/pull/5335 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (0d0af45 -> e8adb0a)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 0d0af45 [Docs] Add putstate, set-replicator-dispatch-rate and
get-replicator-dispatch-rate cli docs for pulsar admin (#5338)
add e8adb0a Build python 37 artifacts and use them for the docker image
(#5335)
No new revisions were added by this update.
Summary of changes:
docker/pulsar/Dockerfile | 16 +---
docker/pulsar/pom.xml| 4 ++--
...l-pulsar-client-35.sh => install-pulsar-client-37.sh} | 4 ++--
3 files changed, 9 insertions(+), 15 deletions(-)
rename docker/pulsar/scripts/{install-pulsar-client-35.sh =>
install-pulsar-client-37.sh} (88%)
[pulsar] branch master updated (1611470 -> 0d0af45)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 1611470 [Doc] Add *HDFS2 sink connector guide* (#5226) add 0d0af45 [Docs] Add putstate, set-replicator-dispatch-rate and get-replicator-dispatch-rate cli docs for pulsar admin (#5338) No new revisions were added by this update. Summary of changes: site2/docs/reference-pulsar-admin.md | 43 1 file changed, 43 insertions(+)
[GitHub] [pulsar] sijie merged pull request #5338: [Docs] Add putstate, set-replicator-dispatch-rate and get-replicator-dispatch-rate cli docs for pulsar admin
sijie merged pull request #5338: [Docs] Add putstate, set-replicator-dispatch-rate and get-replicator-dispatch-rate cli docs for pulsar admin URL: https://github.com/apache/pulsar/pull/5338 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5356: [TIEREDSTORAGE] Only seek when reading unexpected entry
sijie commented on a change in pull request #5356: [TIEREDSTORAGE] Only seek
when reading unexpected entry
URL: https://github.com/apache/pulsar/pull/5356#discussion_r333461002
##
File path:
tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/BlobStoreBackedInputStreamImpl.java
##
@@ -116,7 +116,7 @@ public int read(byte[] b, int off, int len) throws
IOException {
@Override
public void seek(long position) {
-log.debug("Seeking to {} on {}/{}, current position {}", position,
bucket, key, cursor);
+log.info("Seeking to {} on {}/{}, current position {} (bufStart:{},
bufEnd:{})", position, bucket, key, cursor, bufferOffsetStart, bufferOffsetEnd);
Review comment:
Can this be annoying?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] Ranganaths commented on issue #4510: A host can't connect and won't try other hosts again in multi-hosts service url configuration
Ranganaths commented on issue #4510: A host can't connect and won't try other hosts again in multi-hosts service url configuration URL: https://github.com/apache/pulsar/issues/4510#issuecomment-540521138 i am having a same problem- I am Just trying to start pulsar using bin/pulsar standalone on mac book, It throws up unknownhostexception for my mac host- any fix for this is appreciated.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ivankelly opened a new pull request #5356: [TIEREDSTORAGE] Only seek when reading unexpected entry
ivankelly opened a new pull request #5356: [TIEREDSTORAGE] Only seek when reading unexpected entry URL: https://github.com/apache/pulsar/pull/5356 The normal pattern from reading from an offloaded ledger is that the reader will read the ledger sequentially from start to end. This means that once a user reads an entry, we should expect that the next entry they read will be the next entry in the ledger. The initial implementation of the BlobStoreBackedReadHandleImpl (and the S3 variant that preceeded it) didn't take this into account. Instead it did a lookup in the index each time, to find the block that contained the entry, and then read forward in the block until it found the entry requested. This is fine for the first few entries in the block, not so much for the last. This PR changes the read behaviour to only seek if entryId read from the block is either: - greater than the entry we were expecting to read, in which case we need to seek backwards in the block. - less than the entry expected, but also belonging to a different block to the expected entry, in which case we need to seek to the correct block. This change improves read performance significantly. Adhoc benchmarks shows that we can read from offloaded topics at ~160MB/s whereas previously we could only manage <10MB/s. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-540506247 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r333425375
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/SchemaRegistryServiceImpl.java
##
@@ -109,61 +110,55 @@
@Override
@NotNull
public CompletableFuture putSchemaIfAbsent(String schemaId,
SchemaData schema,
-
SchemaCompatibilityStrategy strategy) {
-return getSchema(schemaId, SchemaVersion.Latest)
-.thenCompose(
-(existingSchema) ->
-{
-CompletableFuture maxDeleteVersionFuture;
-if (existingSchema == null) {
-maxDeleteVersionFuture =
completedFuture(NO_DELETED_VERSION);
-} else if (existingSchema.schema.isDeleted()) {
-maxDeleteVersionFuture =
completedFuture(((LongSchemaVersion)schemaStorage
-
.versionFromBytes(existingSchema.version.bytes())).getVersion());
+
SchemaCompatibilityStrategy strategy, boolean isAllowAutoUpdateSchema) {
+return
trimDeletedSchemaAndGetList(schemaId).thenCompose(schemaAndMetadataList -> {
+final CompletableFuture completableFuture = new
CompletableFuture<>();
+SchemaVersion schemaVersion;
+for (SchemaAndMetadata schemaAndMetadata : schemaAndMetadataList) {
+if
(Arrays.equals(hashFunction.hashBytes(schemaAndMetadata.schema.getData()).asBytes(),
+hashFunction.hashBytes(schema.getData()).asBytes())) {
+schemaVersion = schemaAndMetadata.version;
+completableFuture.complete(schemaVersion);
+return completableFuture;
+}
+}
+if (isAllowAutoUpdateSchema) {
+CompletableFuture isCompatibility = new
CompletableFuture<>();
+if (schemaAndMetadataList.size() != 0) {
+if (isTransitiveStrategy(strategy)) {
+isCompatibility = checkCompatibilityWithAll(schema,
strategy, schemaAndMetadataList);
} else {
-if (isTransitiveStrategy(strategy)) {
-maxDeleteVersionFuture =
checkCompatibilityWithAll(schemaId, schema, strategy);
-
-} else {
-maxDeleteVersionFuture = new CompletableFuture<>();
-
trimDeletedSchemaAndGetList(schemaId).thenAccept(schemaAndMetadataList -> {
-checkCompatibilityWithLatest(schemaId, schema,
strategy).whenComplete((v, ex) -> {
-if (ex == null) {
-Long maxDeleteVersion =
((LongSchemaVersion)schemaStorage
-
.versionFromBytes(schemaAndMetadataList.get(0).version.bytes())).getVersion() -
1L;
-
maxDeleteVersionFuture.complete(maxDeleteVersion);
-} else {
-
maxDeleteVersionFuture.completeExceptionally(ex);
-}
-});
-});
-}
+isCompatibility =
checkCompatibilityWithLatest(schemaId, schema, strategy);
}
-return maxDeleteVersionFuture;
+} else {
+isCompatibility.complete(null);
}
-).thenCompose(maxDeleteVersion -> {
-byte[] context =
hashFunction.hashBytes(schema.getData()).asBytes();
-SchemaRegistryFormat.SchemaInfo info =
SchemaRegistryFormat.SchemaInfo.newBuilder()
-
.setType(Functions.convertFromDomainType(schema.getType()))
-.setSchema(ByteString.copyFrom(schema.getData()))
-.setSchemaId(schemaId)
-.setUser(schema.getUser())
-.setDeleted(false)
-.setTimestamp(clock.millis())
-.addAllProps(toPairs(schema.getProps()))
-.build();
-return schemaStorage.put(schemaId, info.toByteArray(),
context, maxDeleteVersion);
-});
+return isCompatibility.thenCompose((v) -> {
+byte[] context =
hashFunction.hashBytes(schema.getData()).asBytes();
+SchemaRegistryFormat.SchemaInfo info =
SchemaRegistryFormat.SchemaInfo.newBuilder()
+
.setType(Functions.convertFromDom
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#discussion_r333416528 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java ## @@ -19,12 +19,27 @@ package org.apache.pulsar.broker.service.persistent; import com.carrotsearch.hppc.ObjectObjectHashMap; + import com.google.common.annotations.VisibleForTesting; import com.google.common.collect.Lists; import com.google.common.collect.Maps; import com.google.common.collect.Sets; + import io.netty.buffer.ByteBuf; import io.netty.util.concurrent.FastThreadLocal; + +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import java.util.concurrent.CancellationException; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLongFieldUpdater; Review comment: Remove the unused imports This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r333435722
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/SchemaRegistryServiceImpl.java
##
@@ -245,6 +240,28 @@ private void checkCompatible(SchemaAndMetadata
existingSchema, SchemaData newSch
});
}
+@Override
+public CompletableFuture checkConsumerCompatibility(String schemaId,
SchemaData schemaData,
+
SchemaCompatibilityStrategy strategy) {
+return getSchema(schemaId).thenCompose(existingSchema -> {
+if (existingSchema != null && !existingSchema.schema.isDeleted()) {
+if (strategy == SchemaCompatibilityStrategy.BACKWARD ||
+strategy == SchemaCompatibilityStrategy.FORWARD ||
+strategy ==
SchemaCompatibilityStrategy.FORWARD_TRANSITIVE ||
+strategy == SchemaCompatibilityStrategy.FULL) {
+return checkCompatibilityWithLatest(schemaId,
schemaData, SchemaCompatibilityStrategy.BACKWARD);
+} else {
+return checkCompatibilityWithAll(schemaId, schemaData,
strategy);
+}
+} else {
+return FutureUtil.failedFuture(
+new IncompatibleSchemaException(
+"Trying to subscribe with incompatible schema"
+));
Review comment:
Shall we need to check the schema is exist, i think this can be handle in
checkCompatibilityWithLatest and checkCompatibilityWithAll ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r88320
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/SchemasResource.java
##
@@ -314,34 +315,38 @@ public void postSchema(
NamespaceName namespaceName = NamespaceName.get(tenant, namespace);
getNamespacePoliciesAsync(namespaceName).thenAccept(policies -> {
-SchemaCompatibilityStrategy schemaCompatibilityStrategy =
SchemaCompatibilityStrategy
-
.fromAutoUpdatePolicy(policies.schema_auto_update_compatibility_strategy);
-byte[] data;
-if (SchemaType.KEY_VALUE.name().equals(payload.getType())) {
-data = DefaultImplementation
-
.convertKeyValueDataStringToSchemaInfoSchema(payload.getSchema().getBytes(Charsets.UTF_8));
-} else {
-data = payload.getSchema().getBytes(Charsets.UTF_8);
-}
-pulsar().getSchemaRegistryService().putSchemaIfAbsent(
-buildSchemaId(tenant, namespace, topic),
-SchemaData.builder()
+SchemaCompatibilityStrategy schemaCompatibilityStrategy =
policies.schema_compatibility_strategy;
+if (schemaCompatibilityStrategy ==
SchemaCompatibilityStrategy.UNDEFINED) {
+schemaCompatibilityStrategy = SchemaCompatibilityStrategy
+
.fromAutoUpdatePolicy(policies.schema_auto_update_compatibility_strategy);
+}
+byte[] data;
+if (SchemaType.KEY_VALUE.name().equals(payload.getType())) {
+data = DefaultImplementation
+
.convertKeyValueDataStringToSchemaInfoSchema(payload.getSchema().getBytes(Charsets.UTF_8));
+} else {
+data = payload.getSchema().getBytes(Charsets.UTF_8);
+}
+pulsar().getSchemaRegistryService().putSchemaIfAbsent(
+buildSchemaId(tenant, namespace, topic),
+SchemaData.builder()
.data(data)
.isDeleted(false)
.timestamp(clock.millis())
.type(SchemaType.valueOf(payload.getType()))
.user(defaultIfEmpty(clientAppId(), ""))
.props(payload.getProperties())
.build(),
Review comment:
Add tab to format the builder
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r333439196
##
File path:
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/schema/SchemaServiceTest.java
##
@@ -252,16 +253,27 @@ public void dontReAddExistingSchemaAtRoot() throws
Exception {
putSchema(schemaId1, schema1, version(0));
}
+@Test
+public void testSchemaAutoUpdate() throws Exception {
+putSchema(schemaId1, schema1, version(0));
+try {
+schemaRegistryService.putSchemaIfAbsent(schemaId1, schema2,
SchemaCompatibilityStrategy.FULL, false).get();
Review comment:
Add fail() here;
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r93457
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/SchemasResource.java
##
@@ -314,34 +315,38 @@ public void postSchema(
NamespaceName namespaceName = NamespaceName.get(tenant, namespace);
getNamespacePoliciesAsync(namespaceName).thenAccept(policies -> {
-SchemaCompatibilityStrategy schemaCompatibilityStrategy =
SchemaCompatibilityStrategy
-
.fromAutoUpdatePolicy(policies.schema_auto_update_compatibility_strategy);
-byte[] data;
-if (SchemaType.KEY_VALUE.name().equals(payload.getType())) {
-data = DefaultImplementation
-
.convertKeyValueDataStringToSchemaInfoSchema(payload.getSchema().getBytes(Charsets.UTF_8));
-} else {
-data = payload.getSchema().getBytes(Charsets.UTF_8);
-}
-pulsar().getSchemaRegistryService().putSchemaIfAbsent(
-buildSchemaId(tenant, namespace, topic),
-SchemaData.builder()
+SchemaCompatibilityStrategy schemaCompatibilityStrategy =
policies.schema_compatibility_strategy;
+if (schemaCompatibilityStrategy ==
SchemaCompatibilityStrategy.UNDEFINED) {
+schemaCompatibilityStrategy = SchemaCompatibilityStrategy
+
.fromAutoUpdatePolicy(policies.schema_auto_update_compatibility_strategy);
+}
+byte[] data;
+if (SchemaType.KEY_VALUE.name().equals(payload.getType())) {
+data = DefaultImplementation
+
.convertKeyValueDataStringToSchemaInfoSchema(payload.getSchema().getBytes(Charsets.UTF_8));
+} else {
+data = payload.getSchema().getBytes(Charsets.UTF_8);
+}
+pulsar().getSchemaRegistryService().putSchemaIfAbsent(
+buildSchemaId(tenant, namespace, topic),
+SchemaData.builder()
.data(data)
.isDeleted(false)
.timestamp(clock.millis())
.type(SchemaType.valueOf(payload.getType()))
.user(defaultIfEmpty(clientAppId(), ""))
.props(payload.getProperties())
.build(),
-schemaCompatibilityStrategy
-).thenAccept(version ->
-response.resume(
-Response.accepted().entity(
-PostSchemaResponse.builder()
-.version(version)
-.build()
-).build()
-)
+schemaCompatibilityStrategy,
+true
Review comment:
I think we don't need to add a new parameter for putSchemaIfAbsent(), add
flag is_allow_auto_update_schema only effect the schema creation of producers
and consumers, so move the check logic to topic.addSchema is more simpler, so
that we don't need to transfer this parameter here
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#discussion_r333415821 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java ## @@ -164,14 +164,14 @@ void updateRates(NamespaceStats nsStats, NamespaceBundleStats currentBundleStats /** * Check if schema is compatible with current topic schema. */ -CompletableFuture isSchemaCompatible(SchemaData schema); +CompletableFuture isSchemaCompatible(SchemaData schema); Review comment: Same as above comment, if this method only use for consumer schema compatible check, it's better to rename to checkSchemaCompatibleForConsumer This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r333404477
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java
##
@@ -171,7 +172,7 @@ public String getReplicatorPrefix() {
String id = TopicName.get(base).getSchemaName();
return brokerService.pulsar()
.getSchemaRegistryService()
-.putSchemaIfAbsent(id, schema, schemaCompatibilityStrategy);
+.putSchemaIfAbsent(id, schema, schemaCompatibilityStrategy,
isAllowAutoUpdateSchema);
Review comment:
Same as above comments,
```java
if(!isAllowAutoUpdateSchema) {
return FutureUtil.failedFuture(ex);
} else {
return
brokerService.pulsar().getSchemaRegistryService().putSchemaIfAbsent(id, schema,
schemaCompatibilityStrategy);
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r333405829
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java
##
@@ -190,12 +191,12 @@ public String getReplicatorPrefix() {
}
@Override
-public CompletableFuture isSchemaCompatible(SchemaData schema) {
+public CompletableFuture isSchemaCompatible(SchemaData schema) {
Review comment:
If this method only use for consumer schema compatible check, it's better to
rename to checkSchemaCompatibleForConsumer
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
codelipenghui commented on a change in pull request #5227: [PIP-44] Separate
schema compatibility checker for producer and consumer
URL: https://github.com/apache/pulsar/pull/5227#discussion_r333423003
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/SchemaRegistryServiceImpl.java
##
@@ -109,61 +110,55 @@
@Override
@NotNull
public CompletableFuture putSchemaIfAbsent(String schemaId,
SchemaData schema,
-
SchemaCompatibilityStrategy strategy) {
-return getSchema(schemaId, SchemaVersion.Latest)
-.thenCompose(
-(existingSchema) ->
-{
-CompletableFuture maxDeleteVersionFuture;
-if (existingSchema == null) {
-maxDeleteVersionFuture =
completedFuture(NO_DELETED_VERSION);
-} else if (existingSchema.schema.isDeleted()) {
-maxDeleteVersionFuture =
completedFuture(((LongSchemaVersion)schemaStorage
-
.versionFromBytes(existingSchema.version.bytes())).getVersion());
+
SchemaCompatibilityStrategy strategy, boolean isAllowAutoUpdateSchema) {
+return
trimDeletedSchemaAndGetList(schemaId).thenCompose(schemaAndMetadataList -> {
+final CompletableFuture completableFuture = new
CompletableFuture<>();
+SchemaVersion schemaVersion;
+for (SchemaAndMetadata schemaAndMetadata : schemaAndMetadataList) {
+if
(Arrays.equals(hashFunction.hashBytes(schemaAndMetadata.schema.getData()).asBytes(),
+hashFunction.hashBytes(schema.getData()).asBytes())) {
+schemaVersion = schemaAndMetadata.version;
+completableFuture.complete(schemaVersion);
+return completableFuture;
+}
+}
+if (isAllowAutoUpdateSchema) {
+CompletableFuture isCompatibility = new
CompletableFuture<>();
Review comment:
it's better to rename to checkCompatibilityFurture
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] ivankelly opened a new pull request #5355: [TIEREDSTORAGE] Don't require both region and endpoint to be specified
ivankelly opened a new pull request #5355: [TIEREDSTORAGE] Don't require both region and endpoint to be specified URL: https://github.com/apache/pulsar/pull/5355 There's a bug in how user metadata is attached to a block that if the user doesn't specify both the region and the endpoint, offloading will throw an exception, as you can't add a null value to an immutable map. This change elides null to the empty string in these cases, so that offloading can continue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540456988 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version
congbobo184 removed a comment on issue #4847: Pulsar sql avro support schema version URL: https://github.com/apache/pulsar/pull/4847#issuecomment-540456889 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5347: [Docs]Fix website io connector link 404
tuteng commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-540470549 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
