[GitHub] [pulsar] sijie closed pull request #5507: Bump ffi from 1.9.23 to 1.11.1 in /site
sijie closed pull request #5507: Bump ffi from 1.9.23 to 1.11.1 in /site URL: https://github.com/apache/pulsar/pull/5507 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] dependabot[bot] commented on issue #5507: Bump ffi from 1.9.23 to 1.11.1 in /site
dependabot[bot] commented on issue #5507: Bump ffi from 1.9.23 to 1.11.1 in /site URL: https://github.com/apache/pulsar/pull/5507#issuecomment-550966913 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5507: Bump ffi from 1.9.23 to 1.11.1 in /site
sijie commented on issue #5507: Bump ffi from 1.9.23 to 1.11.1 in /site URL: https://github.com/apache/pulsar/pull/5507#issuecomment-550966903 /site will be removed in #5576 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed pull request #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site
sijie closed pull request #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site URL: https://github.com/apache/pulsar/pull/5506 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site
sijie commented on issue #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site URL: https://github.com/apache/pulsar/pull/5506#issuecomment-550966781 /site will be removed in #5576 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] dependabot[bot] commented on issue #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site
dependabot[bot] commented on issue #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site URL: https://github.com/apache/pulsar/pull/5506#issuecomment-550966794 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5576: Remove old Pulsar website
sijie commented on issue #5576: Remove old Pulsar website URL: https://github.com/apache/pulsar/pull/5576#issuecomment-550966462 @merlimat oops. I missed that. fixed all the references. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] candlerb commented on issue #5520: Missing documentation for window functions
candlerb commented on issue #5520: Missing documentation for window functions URL: https://github.com/apache/pulsar/issues/5520#issuecomment-550961392 Yep, I got that. But I don't see any documentation for how it would work for a *Java* Pulsar Function either. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5581: Fixed static linking on C++ lib on MacOS
aahmed-se commented on issue #5581: Fixed static linking on C++ lib on MacOS
URL: https://github.com/apache/pulsar/pull/5581#issuecomment-550942156
it's broken in jenkins
```
CMakeFiles/Makefile2:351: recipe for target
'examples/CMakeFiles/SampleConsumerListener.dir/all' failed
make[1]: *** [examples/CMakeFiles/SampleConsumerListener.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs
[ 81%] Linking CXX shared library _pulsar.so
[ 81%] Linking CXX executable perfConsumer
[ 81%] Linking CXX executable perfProducer
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
`google::protobuf::internal::WireFormatLite::WriteInt64(int, long,
google::protobuf::io::CodedOutputStream*)'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to `vtable for
google::protobuf::MessageLite'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
`google::protobuf::MessageLite::ParseFromArray(void const*, int)'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
`google::protobuf::io::CodedOutputStream::~CodedOutputStream()'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
`google::protobuf::internal::WireFormatLite::WriteDouble(int, double,
google::protobuf::io::CodedOutputStream*)'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
`google::protobuf::internal::VerifyVersion(int, int, char const*)'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
`google::protobuf::internal::WireFormatLite::WriteBool(int, bool,
google::protobuf::io::CodedOutputStream*)'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
`google::protobuf::internal::WireFormatLite::WriteUInt32(int, unsigned int,
google::protobuf::io::CodedOutputStream*)'
../lib/libpulsar.so.2.5.0-SNAPSHOT: undefined reference to
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #5578: Fix license file in the distribution package
merlimat commented on a change in pull request #5578: Fix license file in the distribution package URL: https://github.com/apache/pulsar/pull/5578#discussion_r343496639 ## File path: distribution/server/src/assemble/LICENSE.bin.txt ## @@ -360,10 +360,9 @@ The Apache Software License, Version 2.0 - io.netty-netty-resolver-4.1.43.Final.jar - io.netty-netty-resolver-dns-4.1.43.Final.jar - io.netty-netty-transport-4.1.43.Final.jar -- io.netty-netty-transport-native-epoll-4.1.43.Final-linux-x86_64.jar +- io.netty-netty-transport-native-epoll-4.1.33.Final-linux-x86_64.jar Review comment: Not directly related with this PR but this dependency doesn't look it's getting picked up correctly, we should pin the version correctly to avoid incompatibilities across versions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site
merlimat commented on issue #5506: Bump jekyll from 3.7.3 to 3.7.4 in /site URL: https://github.com/apache/pulsar/pull/5506#issuecomment-550900865 Should we just close, since `site/` is going away in any case? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #5542: [Issue 5541] cpp and python API: consumer and reader seek
merlimat merged pull request #5542: [Issue 5541] cpp and python API: consumer and reader seek URL: https://github.com/apache/pulsar/pull/5542 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (89d2d8d -> 4d0983a)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 89d2d8d [Issue 5561] Update Debezium version to 0.10.0.Final (#5573) add 4d0983a [Issue 5541] cpp and python API: consumer and reader seek (#5542) No new revisions were added by this update. Summary of changes: pulsar-client-cpp/include/pulsar/Consumer.h | 16 +++ pulsar-client-cpp/include/pulsar/Reader.h| 40 ++ pulsar-client-cpp/lib/Commands.cc| 10 + pulsar-client-cpp/lib/Commands.h | 1 + pulsar-client-cpp/lib/Consumer.cc| 20 + pulsar-client-cpp/lib/ConsumerImpl.cc| 32 ++ pulsar-client-cpp/lib/ConsumerImpl.h | 1 + pulsar-client-cpp/lib/ConsumerImplBase.h | 1 + pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc | 4 ++ pulsar-client-cpp/lib/MultiTopicsConsumerImpl.h | 1 + pulsar-client-cpp/lib/PartitionedConsumerImpl.cc | 4 ++ pulsar-client-cpp/lib/PartitionedConsumerImpl.h | 1 + pulsar-client-cpp/lib/Reader.cc | 32 ++ pulsar-client-cpp/lib/ReaderImpl.cc | 7 +++ pulsar-client-cpp/lib/ReaderImpl.h | 3 ++ pulsar-client-cpp/python/pulsar/__init__.py | 18 +++- pulsar-client-cpp/python/pulsar_test.py | 54 +++- pulsar-client-cpp/python/src/consumer.cc | 10 + pulsar-client-cpp/python/src/reader.cc | 20 + 19 files changed, 272 insertions(+), 3 deletions(-)
[GitHub] [pulsar] merlimat closed issue #5541: Python API: seek(eventtime)
merlimat closed issue #5541: Python API: seek(eventtime) URL: https://github.com/apache/pulsar/issues/5541 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5388: [PIP-51] Introduce sticky consumer
codelipenghui commented on issue #5388: [PIP-51] Introduce sticky consumer URL: https://github.com/apache/pulsar/pull/5388#issuecomment-550895531 @sijie I have addressed you comments except the new Range class in pulsar client api module, because currently the pulsar client api module do not add guava dependency, considering to keep api module thin, so i add the new Range class. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #5557: [Schema]Add SchemaDisabled for schema
merlimat commented on a change in pull request #5557: [Schema]Add SchemaDisabled for schema URL: https://github.com/apache/pulsar/pull/5557#discussion_r343493860 ## File path: pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java ## @@ -213,6 +213,16 @@ default void configureSchemaInfo(String topic, String componentName, return DefaultImplementation.newProtobufSchema(SchemaDefinition.builder().withPojo(clazz).build()); } +/** + * Disable SchemaInfo. Review comment: The javadoc should illustrate what's the behavior for this option: * We're just using the SerDe capabilities * No server side validation or enforcement of schema definition This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #5557: [Schema]Add SchemaDisabled for schema
merlimat commented on a change in pull request #5557: [Schema]Add
SchemaDisabled for schema
URL: https://github.com/apache/pulsar/pull/5557#discussion_r343494820
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java
##
@@ -213,6 +213,16 @@ default void configureSchemaInfo(String topic, String
componentName,
return
DefaultImplementation.newProtobufSchema(SchemaDefinition.builder().withPojo(clazz).build());
}
+/**
+ * Disable SchemaInfo.
+ *
+ * @param schema instance
+ * @return a Schema instance of bytes type
+ */
+static Schema disabled(Schema schema) {
Review comment:
Using the proposed API of wrapping the schema seems a bit weird to me:
```java
client.newProducer(Schema.disabled(Schema.JSON.of(MyClass.class))
.topic("my-topic")
.create();
```
In Pulsar the schema definition is used in 3 ways:
* Typed client API
* SerDe
* Server side schema validation
Using `disabled` doesn't convey which of these 3 aspect is being disabled.
What about having an option in the producer, like:
```java
client.newProducer(Schema.JSON.of(MyClass.class)
.topic("my-topic")
.schemaDisableValidation(true) // ... or similar
.create();
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[pulsar-manager] annotated tag v0.1.0-candidate-3 updated (bad43a7 -> 566a922)
This is an automated email from the ASF dual-hosted git repository. guangning pushed a change to annotated tag v0.1.0-candidate-3 in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git. *** WARNING: tag v0.1.0-candidate-3 was modified! *** from bad43a7 (commit) to 566a922 (tag) tagging bad43a70f30ff79d6ac6995561ebe96dbed283b3 (commit) replaces v0.1.0-candidate-1 by guangning on Thu Nov 7 14:35:36 2019 +0800 - Log - Release 0.1.0-candidate-3 -BEGIN PGP SIGNATURE- iQFJBAABCAAzFiEEfLgyKwIZSFWNsgIYV3wHyoZ66RAFAl3DuzgVHGd1YW5nbmlu Z0BhcGFjaGUub3JnAAoJEFd8B8qGeukQkSAIAMONYTz0rvTA8y14VM8eQl0NNkiu bxqQSoobaRTlWObyvH2nEMwNURLKLGUe5W62yD/QAtXxy7q9lOw1qyW1NtqBvJl8 i/uLrJCTKZ6cAPKyho0HQIBDACN6i5pPiCYKRCq3AY+aNPnv7sGXJfCmmweYUfEW tdVQasUDclrj9DWXs1LK1lybg6u4FkNBg93zfMf/wv6/A00YhB6mYE24sbzFPdbk +eHfMmAaGBWPqtXNnXonJU/Pm2NQ8LiLK/RaKeiFF65d/m+U7Rr9znfhWyQQcIGF LGafQyN7CKDco1I1q4PFY91BP9/EyCNvQjWYCuFsmbeRJ1yS3Jf0KpNQGJU= =KKC/ -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[GitHub] [pulsar] merlimat opened a new pull request #5581: Fixed static linking on C++ lib on MacOS
merlimat opened a new pull request #5581: Fixed static linking on C++ lib on MacOS URL: https://github.com/apache/pulsar/pull/5581 ### Motivation Static linking of C++ client lib on MacOS (which is used in building the Python wheel file) got broken in #4071. Making sure we link statically with all the libs minus `libz` and `libcurl`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#issuecomment-550843216 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5520: Missing documentation for window functions
Jennifer88huang commented on issue #5520: Missing documentation for window functions URL: https://github.com/apache/pulsar/issues/5520#issuecomment-550776824 @candlerb Currently, window function is supported in Java, not in Python. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5520: Missing documentation for window functions
wolfstudy edited a comment on issue #5520: Missing documentation for window
functions
URL: https://github.com/apache/pulsar/issues/5520#issuecomment-550707625
Sorry, I missed this message.
> But I'd still like to see documentation for what windowing is supposed to
do :-)
@candlerb Currently, does not support `window function` in Python Function.
```
private static void doPythonChecks(FunctionConfig functionConfig) {
if (functionConfig.getProcessingGuarantees() ==
FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
throw new RuntimeException("Effectively-once processing
guarantees not yet supported in Python");
}
if (functionConfig.getWindowConfig() != null) {
throw new IllegalArgumentException("There is currently no
support windowing in python");
}
if (functionConfig.getMaxMessageRetries() != null &&
functionConfig.getMaxMessageRetries() >= 0) {
throw new IllegalArgumentException("Message retries not yet
supported in python");
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5520: Missing documentation for window functions
wolfstudy commented on issue #5520: Missing documentation for window functions
URL: https://github.com/apache/pulsar/issues/5520#issuecomment-550707625
> But I'd still like to see documentation for what windowing is supposed to
do :-)
Sorry, I missed this message.
@candlerb Currently, does not support `window function` in Python Function.
```
private static void doPythonChecks(FunctionConfig functionConfig) {
if (functionConfig.getProcessingGuarantees() ==
FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
throw new RuntimeException("Effectively-once processing
guarantees not yet supported in Python");
}
if (functionConfig.getWindowConfig() != null) {
throw new IllegalArgumentException("There is currently no
support windowing in python");
}
if (functionConfig.getMaxMessageRetries() != null &&
functionConfig.getMaxMessageRetries() >= 0) {
throw new IllegalArgumentException("Message retries not yet
supported in python");
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5577: Fix schema static initialization sequence
sijie commented on issue #5577: Fix schema static initialization sequence URL: https://github.com/apache/pulsar/pull/5577#issuecomment-550696334 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5578: Fix license file in the distribution package
sijie commented on issue #5578: Fix license file in the distribution package URL: https://github.com/apache/pulsar/pull/5578#issuecomment-550696046 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5576: Remove old Pulsar website
Jennifer88huang commented on issue #5576: Remove old Pulsar website URL: https://github.com/apache/pulsar/pull/5576#issuecomment-550686511 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5552: [Pulsar Functions] Allowed to subscribe from an arbitrary point in time
Jennifer88huang commented on issue #5552: [Pulsar Functions] Allowed to subscribe from an arbitrary point in time URL: https://github.com/apache/pulsar/issues/5552#issuecomment-550665323 @wolfstudy will add feature and docs requested from @candlerb . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5553: [website] Lack of a coding-guide page on Pulsar website
Jennifer88huang commented on issue #5553: [website] Lack of a coding-guide page on Pulsar website URL: https://github.com/apache/pulsar/issues/5553#issuecomment-550657177 @tuteng Could you please help add coding-guide.js file? Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5558: When we use the pulsar-admin.sh script to delete partitioned topic, it will throw the http error, error code is 500
Jennifer88huang commented on issue #5558: When we use the pulsar-admin.sh script to delete partitioned topic, it will throw the http error, error code is 500 URL: https://github.com/apache/pulsar/issues/5558#issuecomment-550654624 @codelipenghui @tuteng Could you please help check the issue and give explanation? Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (0053dbc -> 89d2d8d)
This is an automated email from the ASF dual-hosted git repository. hjf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0053dbc Use StandardCharsets.UTF_8 for converting String to bytes (#5372) add 89d2d8d [Issue 5561] Update Debezium version to 0.10.0.Final (#5573) No new revisions were added by this update. Summary of changes: pom.xml| 2 +- .../java/org/apache/pulsar/io/debezium/PulsarDatabaseHistory.java | 5 +++-- .../org/apache/pulsar/io/debezium/PulsarDatabaseHistoryTest.java | 7 --- 3 files changed, 8 insertions(+), 6 deletions(-)
[GitHub] [pulsar] Jennifer88huang closed issue #5561: Update Debezium version to 0.10.0.Final
Jennifer88huang closed issue #5561: Update Debezium version to 0.10.0.Final URL: https://github.com/apache/pulsar/issues/5561 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang merged pull request #5573: [Issue 5561] Update Debezium version to 0.10.0.Final
Jennifer88huang merged pull request #5573: [Issue 5561] Update Debezium version to 0.10.0.Final URL: https://github.com/apache/pulsar/pull/5573 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] TC-oKozlov commented on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setu
TC-oKozlov commented on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setup URL: https://github.com/apache/pulsar/issues/5206#issuecomment-550630916 We are having similar issue in our setup, however in our case we run Pulsar on Kubernetes and trying to have SSL terminate on the load balancer in-front of the Pulsar Proxy, so the client has to send msgs using "pulsar+ssl", but it's not encrypted behind the load balancer, so it's not clear whether proxy and broker should use standard broker port 6650, or TLS broker port 6651? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5568: Function worker fails to be authenticated when TLS authentication is enabled in Pulsar standalone
Jennifer88huang commented on issue #5568: Function worker fails to be authenticated when TLS authentication is enabled in Pulsar standalone URL: https://github.com/apache/pulsar/issues/5568#issuecomment-550621003 @wolfstudy could you please help to look into the issue? Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ChangWinde opened a new issue #5580: error when using pulsar-client with Kerberos
ChangWinde opened a new issue #5580: error when using pulsar-client with
Kerberos
URL: https://github.com/apache/pulsar/issues/5580
**Describe the bug**
I use kerberos in pulsar. And I want use pulsar cli tools, but occuring some
error.
**To Reproduce**
Steps to reproduce the behavior:
bin/pulsar-client produce -m "dog" -n 10
persistent://test-tenant/test-sp/test-topic-partition-0
**Screenshots**
10:54:34.862 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar
producer perf with config: {
"topicName" : "persistent://test-tenant/test-sp/test-topic-partition-0",
"producerName" : null,
"sendTimeoutMs" : 3,
"blockIfQueueFull" : false,
"maxPendingMessages" : 1000,
"maxPendingMessagesAcrossPartitions" : 5,
"messageRoutingMode" : "RoundRobinPartition",
"hashingScheme" : "JavaStringHash",
"cryptoFailureAction" : "FAIL",
"batchingMaxPublishDelayMicros" : 1000,
"batchingMaxMessages" : 1000,
"batchingEnabled" : true,
"batcherBuilder" : { },
"compressionType" : "NONE",
"initialSequenceId" : null,
"autoUpdatePartitions" : true,
"properties" : { }
}
10:54:34.864 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config:
{
"serviceUrl" : "pulsar://localhost:6650/",
"authPluginClassName" : null,
"authParams" : null,
"operationTimeoutMs" : 3,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : "",
"tlsAllowInsecureConnection" : false,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 5,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 1,
"requestTimeoutMs" : 6,
"defaultBackoffIntervalNanos" : 1,
"maxBackoffIntervalNanos" : 300
}
10:54:34.882 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xc7a6a676,
L:/10.31.26.18:36758 - R:10.31.26.18/10.31.26.18:6650]] Connected to server
10:54:34.882 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.auth.PulsarSaslClient - Using JAAS/SASL/GSSAPI
auth to connect to server Principal broker/hadoop.hakafka.com,
10:54:34.882 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.auth.PulsarSaslClient -
即将创建一个新的SaslClient,clientPrincipal=brokerclient/hadoop.hakafka@hakafka.com,servicePrincipal=broker/hadoop.hakafka.com
10:54:34.889 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.ProducerImpl -
[persistent://test-tenant/test-sp/test-topic-partition-0] [null] Creating
producer on cnx [id: 0xc7a6a676, L:/10.31.26.18:36758 -
R:10.31.26.18/10.31.26.18:6650]
10:54:34.901 [pulsar-client-io-1-1] WARN
org.apache.pulsar.client.impl.ClientCnx - [id: 0xc7a6a676, L:/10.31.26.18:36758
- R:10.31.26.18/10.31.26.18:6650] Received error from server:
org.apache.bookkeeper.mledger.ManagedLedgerException: Attempted to access
ledger using the wrong password
10:54:34.902 [pulsar-client-io-1-1] ERROR
org.apache.pulsar.client.impl.ProducerImpl -
[persistent://test-tenant/test-sp/test-topic-partition-0] [null] Failed to
create producer: org.apache.bookkeeper.mledger.ManagedLedgerException:
Attempted to access ledger using the wrong password
10:54:34.903 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool -
Error while producing messages
10:54:34.903 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool -
org.apache.bookkeeper.mledger.ManagedLedgerException: Attempted to access
ledger using the wrong password
org.apache.pulsar.client.api.PulsarClientException$BrokerPersistenceException:
org.apache.bookkeeper.mledger.ManagedLedgerException: Attempted to access
ledger using the wrong password
**Additional context**
client.conf:
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationSasl
authParams={"saslJaasClientSectionName":"PulsarClient",
"serverType":"broker"}
pulsar_tools_env.sh:
PULSAR_EXTRA_OPTS="${PULSAR_EXTRA_OPTS} ${PULSAR_MEM} ${PULSAR_GC}
-Dio.netty.leakDetectionLevel=disabled
-Djava.security.auth.login.config=/home/pulsar/krbConfig/jaasFiles/pulsar_jaas.conf
-Djava.security.krb5.conf=/home/pulsar/krbConfig/krb5.conf"
pulsar_jaas.conf:
PulsarBroker {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
useTicketCache=false
keyTab="/home/pulsar/krbConfig/keytabs/broker.keytab"
principal="broker/hadoop.hakafka@hakafka.com";
};
PulsarClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
useTicketCache=false
ke
[GitHub] [pulsar] codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#issuecomment-550590051 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4621: [PIP-38] Support batch receive in java client.
codelipenghui commented on issue #4621: [PIP-38] Support batch receive in java client. URL: https://github.com/apache/pulsar/pull/4621#issuecomment-550565573 @sijie I have rebase this pull request, if you have time, please help take a look the license issue This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce
codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce URL: https://github.com/apache/pulsar/pull/5491#issuecomment-550561284 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5510: splitting bundle error
tuteng commented on issue #5510: splitting bundle error URL: https://github.com/apache/pulsar/issues/5510#issuecomment-550560845 @sijie @jiazhai This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on a change in pull request #5571: Add epoch for
connection handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r343390298
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java
##
@@ -139,7 +140,9 @@ public String toString() {
@Override
public ConcurrentOpenHashSet getProducers() {
-return producers;
+ConcurrentOpenHashSet result = new
ConcurrentOpenHashSet<>(16, 1);
Review comment:
@sijie i will try change the getProducers method to return a map.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5576: Remove old Pulsar website
codelipenghui commented on issue #5576: Remove old Pulsar website URL: https://github.com/apache/pulsar/pull/5576#issuecomment-550550318 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5574: Add Github workflow for gated checkin
sijie commented on issue #5574: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5574#issuecomment-550549203 > Most of the PRs have limited information and only limited group of people have the knowledge about them. I am trying to understand more what can I improve in the future. I looked into the first page of pull quests. I see most of the PRs do have the description of motivations and modifications, except #5578. #5578 is kind of self-described in the caption. Should I add the description there? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5574: Add Github workflow for gated checkin
sijie commented on issue #5574: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5574#issuecomment-550545810 @rdhabalia sorry my bad. Since it was a fork of Ali ‘s pull quest #5064 to verify if GitHub actions works at a forked repo, I didn’t copy the description from #5064 but just add the description linking back to the original PR to give the full credits back to Ali. I also updated the existing email thread in dev@ to resume the discussion using GitHub actions for CI infrastructure which was postponed due to GitHub actions weren’t able to run at a forked repo. Hence for reviewing this pull request, one has to follow the original CI efforts, CI discussions in the mailing list. I don’t expect this pull quest to be merged until a decision of using GitHub Action as the CI infra is made via an official vote in mailing list. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] cdbartholomew opened a new issue #5579: [broker] Every time PersistentMessageExpiryMonitor runs, it deletes a non-expired message when using TTL
cdbartholomew opened a new issue #5579: [broker] Every time PersistentMessageExpiryMonitor runs, it deletes a non-expired message when using TTL URL: https://github.com/apache/pulsar/issues/5579 **Describe the bug** I have a topic in a namespace with messageTtl set to 24 hours. The topic has a subscription without a connected consumer. I publish a handful of messages to the topic. Within 5 minutes the oldest message in the topic is deleted. Every 5 minutes after that, one more message gets deleted from the topic even though none have expired yet. I configured the expiry monitor to run every minute. I published 4 messages into the topic. You can see from these logs, that every minute it is expiring one message: ```21:55:39.997 [pulsar-msg-expiry-monitor-25-1] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Starting message expiry check, ttl= 172800 seconds 21:55:40.017 [bookkeeper-ml-workers-OrderedExecutor-6-0] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Expiring all messages until position 292:15 21:56:39.995 [pulsar-msg-expiry-monitor-25-1] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Starting message expiry check, ttl= 172800 seconds 21:56:39.999 [bookkeeper-ml-workers-OrderedExecutor-6-0] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Expiring all messages until position 292:16 21:57:39.995 [pulsar-msg-expiry-monitor-25-1] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Starting message expiry check, ttl= 172800 seconds 21:57:39.997 [bookkeeper-ml-workers-OrderedExecutor-6-0] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Expiring all messages until position 292:17 21:58:39.995 [pulsar-msg-expiry-monitor-25-1] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Starting message expiry check, ttl= 172800 seconds 21:58:39.997 [bookkeeper-ml-workers-OrderedExecutor-6-0] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageExpiryMonitor - [persistent://support-kafkaesque-io/local-useast2-aws/tc1-messages][tc2-consume] Expiring all messages until position 292:18 ``` My topic that started with 4 messages is now empty. None of these messages have actually expired so none should have been deleted. **To Reproduce** Steps to reproduce the behavior: 1. Configure namespace with a TTL, for example, 24 hours. 2. Create a topic in that namespace and add a subscription. Make sure there are no consumers so that messages are in the backlog. 3. Publish messages to the topic. Check that they are all in the backlog. 4. Every time the expiry monitor wakes up (5 minutes by default), it deletes one message from the backlog until they are all gone. **Expected behavior** None of the messages should be deleted from the backlog until they actually expire. **Additional context** Looking at the PersistentMessageExpiryMonitor class, it searches for expired messages and is expecting that if no expired messages are found, the search returns `null` . However, the search appears to be returning the oldest position in the backlog when none have expired, not `null`. Since a position is returned, the code expires messages up to that position. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #5574: Add Github workflow for gated checkin
rdhabalia commented on issue #5574: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5574#issuecomment-550444089 can we please start adding full description in the PR. Most of the PRs have limited information and only limited group of people have the knowledge about them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5574: Add Github workflow for gated checkin
sijie commented on issue #5574: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5574#issuecomment-550423890 This PR includes fixes #5578 and #5577 . Please don't merge this before merging #5578 and #5577. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5578: Fix license file in the distribution package
sijie commented on issue #5578: Fix license file in the distribution package URL: https://github.com/apache/pulsar/pull/5578#issuecomment-550423061 Not sure why Jenkins didn't capture this error. But I noticed that the LICENSE file is not updated to date in #5574 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie opened a new pull request #5578: Fix license file in the distribution package
sijie opened a new pull request #5578: Fix license file in the distribution package URL: https://github.com/apache/pulsar/pull/5578 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on a change in pull request #5571: Add epoch for
connection handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r343230102
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java
##
@@ -139,7 +140,9 @@ public String toString() {
@Override
public ConcurrentOpenHashSet getProducers() {
-return producers;
+ConcurrentOpenHashSet result = new
ConcurrentOpenHashSet<>(16, 1);
Review comment:
I have checked the used places of method `getProducers()`
Looks ok, two places used it, handle backlog exceeded backlog and prometheus
metrics aggregator, others are unit tests.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on a change in pull request #5571: Add epoch for
connection handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r343230102
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java
##
@@ -139,7 +140,9 @@ public String toString() {
@Override
public ConcurrentOpenHashSet getProducers() {
-return producers;
+ConcurrentOpenHashSet result = new
ConcurrentOpenHashSet<>(16, 1);
Review comment:
I have check the used places of method `getProducers()`
Looks ok, two places used it, handle backlog exceeded backlog and prometheus
metrics aggregator, others are unit tests.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] johnwagster commented on issue #5559: added deletion of state for Functions as part of the admin REST API
johnwagster commented on issue #5559: added deletion of state for Functions as part of the admin REST API URL: https://github.com/apache/pulsar/pull/5559#issuecomment-550410599 Updated to include license This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5576: Remove old Pulsar website
sijie commented on issue #5576: Remove old Pulsar website URL: https://github.com/apache/pulsar/pull/5576#issuecomment-550409236 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce
codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce URL: https://github.com/apache/pulsar/pull/5491#issuecomment-550403834 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5567: dashboard is unable to connect with borker cluster
sijie commented on issue #5567: dashboard is unable to connect with borker cluster URL: https://github.com/apache/pulsar/issues/5567#issuecomment-550401472 Can you share the command you used for starting the dashboard container? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5575: fix the typo error
sijie commented on issue #5575: fix the typo error URL: https://github.com/apache/pulsar/issues/5575#issuecomment-550396887 @chenyuyun-emc are you interested in sending a pull request to fix this typo? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng edited a comment on issue #5510: splitting bundle error
tuteng edited a comment on issue #5510: splitting bundle error URL: https://github.com/apache/pulsar/issues/5510#issuecomment-550365364 I seem to have reproduced this problem today. When multiple producers deliver messages to the same topic at the same time, the bundle will be split when the current bundle load is very high, so multiple producers will split the same bundle. reproduce: 1. Start three broker, loadBalancerAutoUnloadSplitBundlesEnabled=true 2. Execute on any broker ``` ./bin/pulsar-perf produce -o 10 -r 10 -n 3 test-topic-n ``` test-topic-n lookup broker-3 ``` 15:14:38.690 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/test-topic-o] Topic closed 15:14:38.693 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.broker.namespace.OwnershipCache - [/namespace/public/default/0xf63f_0xf65f] Removed zk lock for service unit: OK 15:14:38.693 [pulsar-ordered-OrderedExecutor-5-0] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Unloading public/default/0xf63f_0xf65f namespace-bundle with 1 topics completed in 262.0 ms 15:14:38.693 [pulsar-ordered-OrderedExecutor-5-0] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Disabling ownership: public/default/0xf65f_0xf67f 15:14:38.698 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.broker.namespace.OwnershipCache - [/namespace/public/default/0xf65f_0xf67f] Removed zk lock for service unit: OK 15:14:38.698 [pulsar-ordered-OrderedExecutor-5-0] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Unloading public/default/0xf65f_0xf67f namespace-bundle with 0 topics completed in 4.0 ms 15:14:38.698 [pulsar-web-32-3] INFO org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully split namespace bundle public/default/0xf63f_0xf67f 15:14:38.699 [pulsar-web-32-3] INFO org.eclipse.jetty.server.RequestLog - - - [06/Nov/2019:15:14:38 +] "PUT /admin/v2/namespaces/public/default/0xf63f_0xf67f/split?unload=true HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.4.1" 296 15:14:38.701 [pulsar-modular-load-manager-31-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Successfully split namespace bundle public/default/0xf63f_0xf67f 15:14:39.041 [pulsar-io-24-7] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:40652 15:14:39.043 [pulsar-io-24-8] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:40654 15:14:39.053 [pulsar-3-12] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - 2 brokers being considered for assignment of public/default/0xf63f_0xf65f 15:14:39.055 [pulsar-io-24-9] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:40656 15:14:40.865 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperDataCache - [State:CONNECTED Timeout:3 sessionid:0x20019517baa0024 local:/:48318 remoteserver:209.97.175.128/209.97.175.128:2181 lastZxid:4294973170 xid:788 sent:788 recv:850 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/loadbalance/brokers/159.65.9.22:8080 15:14:40.869 [pulsar-modular-load-manager-31-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Load-manager splitting bundle public/default/0xf63f_0xf67f and unloading true 15:14:40.874 [pulsar-web-32-5] INFO org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Split namespace bundle public/default/0xf63f_0xf67f 15:14:40.878 [pulsar-web-32-5] ERROR org.apache.pulsar.broker.web.PulsarWebResource - [null] Failed to validate namespace bundle public/default/0xf63f_0xf67f java.lang.IllegalArgumentException: Invalid upper boundary for bundle at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122) ~[com.google.guava-guava-21.0.jar:?] at org.apache.pulsar.common.naming.NamespaceBundles.validateBundle(NamespaceBundles.java:114) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleRange(PulsarWebResource.java:465) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleOwnership(PulsarWebResource.java:491) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.admin.impl.NamespacesBase.internalSplitNamespaceBundle(NamespacesBase.java:741) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.admin.v2.Namespaces.splitNamespaceBundle(Namespaces.java:330) ~[org.
[GitHub] [pulsar] tuteng commented on issue #5510: splitting bundle error
tuteng commented on issue #5510: splitting bundle error URL: https://github.com/apache/pulsar/issues/5510#issuecomment-550365364 I seem to have reproduced this problem today. When multiple producers deliver messages to the same topic at the same time, the bundle will be split when the current bundle load is very high, so multiple producers will split the same bundle. reproduce: 1. Start three broker, loadBalancerAutoUnloadSplitBundlesEnabled=true 2. Execute on any broker ``` ./bin/pulsar-perf produce -o 10 -r 10 -n 3 test-topic-n ``` test-topic-n lookup broker-3 ``` 15:14:38.690 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/test-topic-o] Topic closed 15:14:38.693 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.broker.namespace.OwnershipCache - [/namespace/public/default/0xf63f_0xf65f] Removed zk lock for service unit: OK 15:14:38.693 [pulsar-ordered-OrderedExecutor-5-0] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Unloading public/default/0xf63f_0xf65f namespace-bundle with 1 topics completed in 262.0 ms 15:14:38.693 [pulsar-ordered-OrderedExecutor-5-0] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Disabling ownership: public/default/0xf65f_0xf67f 15:14:38.698 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.broker.namespace.OwnershipCache - [/namespace/public/default/0xf65f_0xf67f] Removed zk lock for service unit: OK 15:14:38.698 [pulsar-ordered-OrderedExecutor-5-0] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Unloading public/default/0xf65f_0xf67f namespace-bundle with 0 topics completed in 4.0 ms 15:14:38.698 [pulsar-web-32-3] INFO org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully split namespace bundle public/default/0xf63f_0xf67f 15:14:38.699 [pulsar-web-32-3] INFO org.eclipse.jetty.server.RequestLog - - - [06/Nov/2019:15:14:38 +] "PUT /admin/v2/namespaces/public/default/0xf63f_0xf67f/split?unload=true HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.4.1" 296 15:14:38.701 [pulsar-modular-load-manager-31-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Successfully split namespace bundle public/default/0xf63f_0xf67f 15:14:39.041 [pulsar-io-24-7] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:40652 15:14:39.043 [pulsar-io-24-8] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:40654 15:14:39.053 [pulsar-3-12] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - 2 brokers being considered for assignment of public/default/0xf63f_0xf65f 15:14:39.055 [pulsar-io-24-9] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:40656 15:14:40.865 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperDataCache - [State:CONNECTED Timeout:3 sessionid:0x20019517baa0024 local:/:48318 remoteserver:209.97.175.128/209.97.175.128:2181 lastZxid:4294973170 xid:788 sent:788 recv:850 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/loadbalance/brokers/159.65.9.22:8080 15:14:40.869 [pulsar-modular-load-manager-31-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Load-manager splitting bundle public/default/0xf63f_0xf67f and unloading true 15:14:40.874 [pulsar-web-32-5] INFO org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Split namespace bundle public/default/0xf63f_0xf67f 15:14:40.878 [pulsar-web-32-5] ERROR org.apache.pulsar.broker.web.PulsarWebResource - [null] Failed to validate namespace bundle public/default/0xf63f_0xf67f java.lang.IllegalArgumentException: Invalid upper boundary for bundle at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122) ~[com.google.guava-guava-21.0.jar:?] at org.apache.pulsar.common.naming.NamespaceBundles.validateBundle(NamespaceBundles.java:114) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleRange(PulsarWebResource.java:465) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleOwnership(PulsarWebResource.java:491) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.admin.impl.NamespacesBase.internalSplitNamespaceBundle(NamespacesBase.java:741) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.admin.v2.Namespaces.splitNamespaceBundle(Namespaces.java:330) ~[org.apache.
[GitHub] [pulsar] sijie commented on issue #5577: Fix schema static initialization sequence
sijie commented on issue #5577: Fix schema static initialization sequence URL: https://github.com/apache/pulsar/pull/5577#issuecomment-550358574 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5576: Remove old Pulsar website
sijie commented on issue #5576: Remove old Pulsar website URL: https://github.com/apache/pulsar/pull/5576#issuecomment-550358255 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5576: Remove old Pulsar website
sijie commented on issue #5576: Remove old Pulsar website URL: https://github.com/apache/pulsar/pull/5576#issuecomment-550342670 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5557: [Schema]Add SchemaDisabled for schema
tuteng commented on issue #5557: [Schema]Add SchemaDisabled for schema URL: https://github.com/apache/pulsar/pull/5557#issuecomment-550336331 @merlimat PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie opened a new pull request #5577: Fix schema static initialization sequence
sijie opened a new pull request #5577: Fix schema static initialization sequence URL: https://github.com/apache/pulsar/pull/5577 *Motivation* Java doesn't guarantee the initialization order of the static fields of a class. It depends on the JVM version and race conditions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie opened a new pull request #5576: Remove old Pulsar website
sijie opened a new pull request #5576: Remove old Pulsar website URL: https://github.com/apache/pulsar/pull/5576 ### Motivation The new pulsar website is under `/site2`. There is no need to keep `/site` directory. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] chenyuyun-emc opened a new issue #5575: fix the typo error
chenyuyun-emc opened a new issue #5575: fix the typo error URL: https://github.com/apache/pulsar/issues/5575 I read the code, I found there is typo error in org.apache.pulsar.broker.admin.impl.BrokersBase.java  this should be getOwnedNamespaces This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
sijie commented on a change in pull request #5571: Add epoch for connection
handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r343092682
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java
##
@@ -139,7 +140,9 @@ public String toString() {
@Override
public ConcurrentOpenHashSet getProducers() {
-return producers;
+ConcurrentOpenHashSet result = new
ConcurrentOpenHashSet<>(16, 1);
Review comment:
oops. I didn't realize that there is a `getProducers` here. It is used in a
lot of places. hence it might be a performance problem if we changed to a hash
map.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce
codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce URL: https://github.com/apache/pulsar/pull/5491#issuecomment-550297121 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie opened a new pull request #5574: Add Github workflow for gated checkin
sijie opened a new pull request #5574: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5574 The change is a forked of #5064 to verify if the github action can run at forked repo. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5574: Add Github workflow for gated checkin
sijie commented on issue #5574: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5574#issuecomment-550292969 ^ @aahmed-se This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5342: Use simple GC settings when running integration tests
sijie commented on issue #5342: Use simple GC settings when running integration tests URL: https://github.com/apache/pulsar/pull/5342#issuecomment-550291475 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5386: Update presto to 0.226
sijie commented on issue #5386: Update presto to 0.226 URL: https://github.com/apache/pulsar/pull/5386#issuecomment-550291662 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (e0ea6b5 -> 0053dbc)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e0ea6b5 Validate topic name before creating partition/non partition topic via admin cli. (#5148) add 0053dbc Use StandardCharsets.UTF_8 for converting String to bytes (#5372) No new revisions were added by this update. Summary of changes: .../streaming/receiver/example/ProducerSparkReceiverData.java | 5 +++-- .../receiver/example/SparkStreamingPulsarReceiverExample.java | 4 ++-- .../src/main/java/org/apache/pulsar/utils/CmdUtility.java | 3 ++- .../java/org/apache/pulsar/utils/SimpleTextOutputStreamTest.java | 4 ++-- .../test/java/org/apache/pulsar/utils/StatsOutputStreamTest.java | 4 ++-- .../pulsar/client/kafka/test/KafkaProducerSimpleConsumerTest.java | 3 ++- .../test/java/org/apache/pulsar/admin/cli/utils/IOUtilsTest.java | 3 ++- .../src/main/java/org/apache/pulsar/io/kinesis/KinesisRecord.java | 4 ++-- .../src/main/java/org/apache/pulsar/io/mongodb/MongoSink.java | 6 +++--- .../src/main/java/org/apache/pulsar/io/mongodb/MongoSource.java| 6 +++--- .../org/apache/pulsar/zookeeper/ZookeeperClientFactoryImpl.java| 3 ++- .../test/java/org/apache/pulsar/zookeeper/DeserializersTest.java | 7 --- 12 files changed, 29 insertions(+), 23 deletions(-)
[GitHub] [pulsar] sijie commented on issue #5358: PIP-45: Switch ManagedLedger to use MetadataStore interface
sijie commented on issue #5358: PIP-45: Switch ManagedLedger to use MetadataStore interface URL: https://github.com/apache/pulsar/pull/5358#issuecomment-550290016 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #5372: Use StandardCharsets.UTF_8 for converting String to bytes
sijie merged pull request #5372: Use StandardCharsets.UTF_8 for converting String to bytes URL: https://github.com/apache/pulsar/pull/5372 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] krnaveen14 opened a new pull request #5573: [Issue 5561] Update Debezium version to 0.10.0.Final
krnaveen14 opened a new pull request #5573: [Issue 5561] Update Debezium version to 0.10.0.Final URL: https://github.com/apache/pulsar/pull/5573 Update Debezium version to 0.10.0.Final #5561 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#issuecomment-550261994 @sijie I have addressed your comments, please take a look again. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-550262242 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5278: Fixed reader cursor seek with disconnection
sijie commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-550258822 @merlimat I wasn't sure how is the ordering guaranteed if we don't clean up the cached message. Can you clarify it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5278: Fixed reader cursor seek with disconnection
jiazhai commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-550243471 > > If we don't disconnect the consumers, how do consumers reset their state? e.g. pending queue, acknowledgement, and etc. > > Consumers they won't reset, they just will see the messages restart from an earlier (or later) point in time. I don't believe that should affect the consumer logic since there will be no visible disruption from consumer perspective. Maybe Sijie was concern about the messages that already cached in consumer client. We may need some handling to clean the cached messages? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4806: A Pulsar producer should be able to produce messages of different schemas
sijie commented on issue #4806: A Pulsar producer should be able to produce messages of different schemas URL: https://github.com/apache/pulsar/issues/4806#issuecomment-550241039 This feature is implemented in PIP-43 (#5141). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #4806: A Pulsar producer should be able to produce messages of different schemas
sijie closed issue #4806: A Pulsar producer should be able to produce messages of different schemas URL: https://github.com/apache/pulsar/issues/4806 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5278: Fixed reader cursor seek with disconnection
wolfstudy commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-550239794 @merlimat I changed the milestone to 2.4.2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5278: Fixed reader cursor seek with disconnection
jiazhai commented on issue #5278: Fixed reader cursor seek with disconnection URL: https://github.com/apache/pulsar/pull/5278#issuecomment-550239914 seems the [test failures](https://builds.apache.org/job/pulsar_precommit_java8/12631/) related with this change: ``` org.apache.pulsar.compaction.CompactionTest.testCompactorReadsCompacted org.apache.pulsar.compaction.CompactionTest.testSeekEarliestAfterCompaction org.apache.pulsar.compaction.CompactorTest.testCompactAddCompact ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implement
congbobo184 commented on issue #5570: Transaction log implement URL: https://github.com/apache/pulsar/pull/5570#issuecomment-550239971 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on a change in pull request #5571: Add epoch for
connection handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r343010215
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionHandler.java
##
@@ -20,19 +20,22 @@
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
+
+import com.google.common.annotations.VisibleForTesting;
import org.apache.pulsar.client.api.PulsarClientException;
import org.apache.pulsar.client.impl.HandlerState.State;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-class ConnectionHandler {
+public class ConnectionHandler {
Review comment:
Ok, let me try
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#discussion_r343010113 ## File path: pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Producer.java ## @@ -38,6 +38,11 @@ */ String getProducerName(); +/** + * @return the name of producer is generated or user specified + */ +boolean isGeneratedName(); Review comment: ok, currently no need to expose to the interface This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] yittg commented on issue #4806: A Pulsar producer should be able to produce messages of different schemas
yittg commented on issue #4806: A Pulsar producer should be able to produce messages of different schemas URL: https://github.com/apache/pulsar/issues/4806#issuecomment-550238314 @sijie This feature addressed in PIP-43 #5141 has been completed, pls close this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
sijie commented on a change in pull request #5571: Add epoch for connection
handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r343000732
##
File path: pulsar-common/src/main/proto/PulsarApi.proto
##
@@ -386,6 +386,13 @@ message CommandProducer {
repeated KeyValue metadata= 6;
optional Schema schema = 7;
+
+// If producer reconnect to broker, the epoch of this producer will +1
+optional uint64 epoch = 8 [default = 0];
+
+// Indicate the name of the producer is generated or not(user specified)
+// Use default false here is in order to be forward compatible with the
client
+optional bool is_generated_name = 9 [default = false];
Review comment:
`is_generated_name` => `user_provided_producer_name` ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
sijie commented on a change in pull request #5571: Add epoch for connection
handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r342996128
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionHandler.java
##
@@ -20,19 +20,22 @@
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
+
+import com.google.common.annotations.VisibleForTesting;
import org.apache.pulsar.client.api.PulsarClientException;
import org.apache.pulsar.client.impl.HandlerState.State;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-class ConnectionHandler {
+public class ConnectionHandler {
Review comment:
nit: add `@VisiableForTesting` ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#discussion_r342999058 ## File path: pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ProducerConfigurationData.java ## @@ -57,6 +57,7 @@ private String topicName = null; private String producerName = null; +private boolean isGeneratedName = true; Review comment: do we really need adding another field in `ProducerConfigurationData`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
sijie commented on a change in pull request #5571: Add epoch for connection
handler to handle create producer timeout.
URL: https://github.com/apache/pulsar/pull/5571#discussion_r343008251
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
##
@@ -425,8 +425,22 @@ public void addProducer(Producer producer) throws
BrokerServiceException {
}
if (!producers.add(producer)) {
-throw new NamingException(
-"Producer with name '" + producer.getProducerName() +
"' is already connected to topic");
+boolean canOverwrite = false;
+for (Producer existProducer : producers.values()) {
Review comment:
Can we avoid iterating over the producers set? I.e. can you change the hash
set to a hash map?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout.
sijie commented on a change in pull request #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#discussion_r342995958 ## File path: pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Producer.java ## @@ -38,6 +38,11 @@ */ String getProducerName(); +/** + * @return the name of producer is generated or user specified + */ +boolean isGeneratedName(); Review comment: do we need to expose this to producer api? I think this is an implementation detail, which should be hidden in the implementation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#issuecomment-550235786 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5572: Fix create consumer on partitioned topic while disable topic auto creation.
codelipenghui commented on issue #5572: Fix create consumer on partitioned topic while disable topic auto creation. URL: https://github.com/apache/pulsar/pull/5572#issuecomment-550235353 @wolfstudy if this PR can complete before cut 2.4.2, please considering include it, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui opened a new pull request #5572: Fix create consumer on partitioned topic while disable topic auto creation.
codelipenghui opened a new pull request #5572: Fix create consumer on partitioned topic while disable topic auto creation. URL: https://github.com/apache/pulsar/pull/5572 Fixes #5565 ### Motivation Currently, disable the topic auto creation will cause consumer create failed on a partitioned topic. Since the partitioned topic is already created, so we should handle the topic partition create when disable the topic auto creation. ### Modifications Handle the topic partition create when disable the topic auto creation ### Verifying this change Add new unit tests for this change This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation - Does this pull request introduce a new feature? (no) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5571: Add epoch for connection handler to handle create producer timeout.
sijie commented on issue #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#issuecomment-550226789 @wolfstudy @codelipenghui This issue has been reported by many users. so let's include it in 2.4.2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5569: [Schema] [Pulsar Functions] Fix schema def build error with protobuf schema
sijie commented on issue #5569: [Schema] [Pulsar Functions] Fix schema def build error with protobuf schema URL: https://github.com/apache/pulsar/pull/5569#issuecomment-550225647 @wolfstudy I changed the milestone to 2.4.2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui commented on issue #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571#issuecomment-550217531 @wolfstudy if this PR can complete before cut 2.4.2, please considering include it, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui opened a new pull request #5571: Add epoch for connection handler to handle create producer timeout.
codelipenghui opened a new pull request #5571: Add epoch for connection handler to handle create producer timeout. URL: https://github.com/apache/pulsar/pull/5571 Fixes #5535 ### Motivation Currently, if user create producer timeout, the connection handler of producer will reconnect to the broker later, but if in broker already done the previous create producer request, the reconnection will failed with "producer with name xxx is already connected". So this PR will introduce epoch for connection handler and add a field named `isGeneratedName` for producer to handle above problem. This PR only handle the generated producer name scenario, so many users occur errors such like #5535, so we need to fix the generated producer name scenario first. For the scenario of user specified producer name, we can discuss later and find a simple approach to handle it, i left my idea here: using producer id and producer name as the identity of producer, producer name used for EO producer and producer id can used by the producer reconnect, but this approach depends on globally unique producer id generator. ### Modifications If the producer with generated producer name and epoch of the producer is bigger than the exists producer, the new producer will overwrite the old producer, so the `reconnect producer` will create succeed. ### Verifying this change Add unit tests to simulate producer timeout and reconnection ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation - Does this pull request introduce a new feature? (no) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implement
congbobo184 commented on issue #5570: Transaction log implement URL: https://github.com/apache/pulsar/pull/5570#issuecomment-550215995 @codelipenghui @sijie @zymap PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 opened a new pull request #5570: Transaction log implement
congbobo184 opened a new pull request #5570: Transaction log implement URL: https://github.com/apache/pulsar/pull/5570 Master Issue: [PIP31](https://github.com/apache/pulsar/wiki/PIP-31%3A-Transaction-Support) ### Motivation Implement of [PIP31](https://github.com/apache/pulsar/wiki/PIP-31%3A-Transaction-Support), transaction log ### Modifications Add TransactionMetadataStore implement by managed ledger ### Verifying this change Add the tests for it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
