[GitHub] [pulsar] codelipenghui commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
codelipenghui commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#issuecomment-545770764 ping @merlimat please help review the documentation for Delayed Message Delivery This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5421: [Broker] Fix log message bug
codelipenghui commented on issue #5421: [Broker] Fix log message bug URL: https://github.com/apache/pulsar/pull/5421#issuecomment-545769650 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5441: [pulsar-perf] record message failure and avoid exiting from process on publish failure
codelipenghui commented on issue #5441: [pulsar-perf] record message failure and avoid exiting from process on publish failure URL: https://github.com/apache/pulsar/pull/5441#issuecomment-545769302 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5075: [pulsar-client] support set read-position based on timestamp
codelipenghui commented on issue #5075: [pulsar-client] support set read-position based on timestamp URL: https://github.com/apache/pulsar/pull/5075#issuecomment-545768835 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5443: [pulsar-client] Fix message corruption on OOM for batch messages
codelipenghui commented on issue #5443: [pulsar-client] Fix message corruption on OOM for batch messages URL: https://github.com/apache/pulsar/pull/5443#issuecomment-545768544 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5451: SchemaUpdateStrategyTest is broken and not run
codelipenghui commented on issue #5451: SchemaUpdateStrategyTest is broken and not run URL: https://github.com/apache/pulsar/issues/5451#issuecomment-545765396 @congbobo184 Can you help take a look this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui opened a new pull request #5456: [integration-tests]Enable messaging tests to integration tests.
codelipenghui opened a new pull request #5456: [integration-tests]Enable messaging tests to integration tests. URL: https://github.com/apache/pulsar/pull/5456 Master Issue: #5452 ### Motivation Enable messaging tests to integration tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (0f57c60 -> 997a128)
This is an automated email from the ASF dual-hosted git repository. rxl pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0f57c60 [pulsar-perf] Add batching max message into perf-producer (#5440) add 997a128 [Docs] Add functions-worker commands for pulsar admin CLI (#5405) No new revisions were added by this update. Summary of changes: site2/docs/reference-pulsar-admin.md | 61 1 file changed, 61 insertions(+)
[GitHub] [pulsar] wolfstudy merged pull request #5405: [Docs] Add functions-worker commands for pulsar admin CLI
wolfstudy merged pull request #5405: [Docs] Add functions-worker commands for pulsar admin CLI URL: https://github.com/apache/pulsar/pull/5405 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liketic commented on issue #5423: [Test] Migrate deprecated Matchers to ArgumentMatchers
liketic commented on issue #5423: [Test] Migrate deprecated Matchers to ArgumentMatchers URL: https://github.com/apache/pulsar/pull/5423#issuecomment-545749679 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5453: [Cpp] Fixed negative ack tracker constructor sequence
merlimat commented on issue #5453: [Cpp] Fixed negative ack tracker constructor sequence URL: https://github.com/apache/pulsar/pull/5453#issuecomment-545727950 run cpp tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5385: Standalone should use crc32c by default as well
aahmed-se commented on issue #5385: Standalone should use crc32c by default as well URL: https://github.com/apache/pulsar/pull/5385#issuecomment-545727311 run cpp tests run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5455: Update test docker scripts and library
aahmed-se commented on issue #5455: Update test docker scripts and library URL: https://github.com/apache/pulsar/pull/5455#issuecomment-545727140 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5405: [Docs] Add functions-worker commands for pulsar admin CLI
wolfstudy commented on issue #5405: [Docs] Add functions-worker commands for pulsar admin CLI URL: https://github.com/apache/pulsar/pull/5405#issuecomment-545720569 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4621: [PIP-38] Support batch receive in java client.
codelipenghui commented on issue #4621: [PIP-38] Support batch receive in java client. URL: https://github.com/apache/pulsar/pull/4621#issuecomment-545718535 ping @sijie please take a look the license issue, i try it some times, but still has problem with license This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4621: [PIP-38] Support batch receive in java client.
codelipenghui commented on issue #4621: [PIP-38] Support batch receive in java client. URL: https://github.com/apache/pulsar/pull/4621#issuecomment-545718186 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-545717925 ping @sijie please take a look this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5388: [PIP-46] Introduce sticky consumer
codelipenghui commented on issue #5388: [PIP-46] Introduce sticky consumer URL: https://github.com/apache/pulsar/pull/5388#issuecomment-545717720 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-545717842 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5378: Trim messages which less than mark delete position for message redelivery
codelipenghui commented on issue #5378: Trim messages which less than mark delete position for message redelivery URL: https://github.com/apache/pulsar/pull/5378#issuecomment-545717626 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5435: Add support for partitioned topic consumer seek by time.
aahmed-se commented on issue #5435: Add support for partitioned topic consumer seek by time. URL: https://github.com/apache/pulsar/pull/5435#issuecomment-545716934 @sijie @merlimat please review this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5455: Update test docker scripts and library
aahmed-se commented on issue #5455: Update test docker scripts and library URL: https://github.com/apache/pulsar/pull/5455#issuecomment-545715766 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] xujianhai666 commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery*
xujianhai666 commented on issue #5293: [ISSUE #4675][docs] Add *Delayed Message Delivery* URL: https://github.com/apache/pulsar/pull/5293#issuecomment-545713737 > retest this please @codelipenghui so what can I do now ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] xujianhai666 commented on issue #69: [ISSUE #68][feat]add Option config for init (#68)
xujianhai666 commented on issue #69: [ISSUE #68][feat]add Option config for init (#68) URL: https://github.com/apache/pulsar-client-go/pull/69#issuecomment-545713574 @sijie PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer
congbobo184 removed a comment on issue #5227: [PIP-44] Separate schema compatibility checker for producer and consumer URL: https://github.com/apache/pulsar/pull/5227#issuecomment-545233810 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia edited a comment on issue #3705: [pulsar-broker] add support for connection expiry and renew-cnx
rdhabalia edited a comment on issue #3705: [pulsar-broker] add support for connection expiry and renew-cnx URL: https://github.com/apache/pulsar/pull/3705#issuecomment-545704545 @aahmed-se I have rebased the PR and it took a while as lot of changes have been merged since this PR was created. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3705: [pulsar-broker] add support for connection expiry and renew-cnx
rdhabalia commented on issue #3705: [pulsar-broker] add support for connection expiry and renew-cnx URL: https://github.com/apache/pulsar/pull/3705#issuecomment-545704545 @aahmed-se I have rebased the PR and it took a while as lot of changes have been merged since this PR was created. this PR was created in version-2.3 as this was one of our security requirement so, I am changing milestone to 2.4 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (e97ea92 -> 0f57c60)
This is an automated email from the ASF dual-hosted git repository. rdhabalia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e97ea92 [Docs] Add delete-dynamic-config command for brokers (#5389) add 0f57c60 [pulsar-perf] Add batching max message into perf-producer (#5440) No new revisions were added by this update. Summary of changes: .../client/impl/conf/ProducerConfigurationData.java | 10 +++--- .../apache/pulsar/testclient/PerformanceProducer.java | 19 ++- 2 files changed, 21 insertions(+), 8 deletions(-)
[GitHub] [pulsar] rdhabalia merged pull request #5440: [pulsar-perf] Add batching max message into perf-producer
rdhabalia merged pull request #5440: [pulsar-perf] Add batching max message into perf-producer URL: https://github.com/apache/pulsar/pull/5440 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liketic commented on issue #5421: [Broker] Fix log message bug
liketic commented on issue #5421: [Broker] Fix log message bug URL: https://github.com/apache/pulsar/pull/5421#issuecomment-545698154 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liketic commented on issue #5423: [Test] Migrate deprecated Matchers to ArgumentMatchers
liketic commented on issue #5423: [Test] Migrate deprecated Matchers to ArgumentMatchers URL: https://github.com/apache/pulsar/pull/5423#issuecomment-545697982 run java8 tests run integration tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5439: Multiple unit tests improvements
merlimat commented on issue #5439: Multiple unit tests improvements URL: https://github.com/apache/pulsar/pull/5439#issuecomment-545693767 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng edited a comment on issue #5454: mysql JDBC Sink - consumer error
tuteng edited a comment on issue #5454: mysql JDBC Sink - consumer error
URL: https://github.com/apache/pulsar/issues/5454#issuecomment-545688370
Please show your producer's code.
The following is an example, please refer to
```
@Builder
@Data
@NoArgsConstructor
@AllArgsConstructor
public class Foo3 {
public String field1;
public String field2;
public int field3;
}
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://localhost:6650")
.build();
AvroSchema schema =
AvroSchema.of(SchemaDefinition.builder().withPojo(Foo3.class).withAlwaysAllowNull(false).build());
Producer producer = client.newProducer(schema)
.topic("test-jdbc")
.create();
for (int i = 0; i < 20; i++) {
String key = "key-" + i;
Foo3 obj = new Foo3();
obj.setField1("field1_insert_" + i);
obj.setField2("field2_insert_" + i);
obj.setField3(i);
Map properties = Maps.newHashMap();
properties.put("EVENT", "INSERT");
producer.newMessage()
.properties(properties)
.key(key)
.value(obj)
.send();
}
for (int i = 0; i < 20; i++) {
String key = "key-" + i;
Foo3 obj = new Foo3();
obj.setField1("field1_insert_" + i);
obj.setField2("field2_update_" + i);
obj.setField3(i);
Map properties = Maps.newHashMap();
properties.put("EVENT", "UPDATE");
producer.newMessage()
.properties(properties)
.key(key)
.value(obj)
.send();
}
for (int i = 0; i < 20; i++) {
String key = "key-" + i;
Foo3 obj = new Foo3();
obj.setField1("field1_insert_" + i);
obj.setField2("field2_delete_" + i);
obj.setField3(i);
Map properties = Maps.newHashMap();
properties.put("EVENT", "DELETE");
producer.newMessage()
.properties(properties)
.key(key)
.value(obj)
.send();
}
producer.close();
client.close();
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] hrsakai commented on issue #5318: Pulsar 2.4.1 - Broker stops server Admin API at random
hrsakai commented on issue #5318: Pulsar 2.4.1 - Broker stops server Admin API at random URL: https://github.com/apache/pulsar/issues/5318#issuecomment-545688810 The issue has been fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] hrsakai closed issue #5318: Pulsar 2.4.1 - Broker stops server Admin API at random
hrsakai closed issue #5318: Pulsar 2.4.1 - Broker stops server Admin API at random URL: https://github.com/apache/pulsar/issues/5318 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5454: mysql JDBC Sink - consumer error
tuteng commented on issue #5454: mysql JDBC Sink - consumer error
URL: https://github.com/apache/pulsar/issues/5454#issuecomment-545688370
Please show your producer's code.
```
@Builder
@Data
@NoArgsConstructor
@AllArgsConstructor
public class Foo3 {
public String field1;
public String field2;
public int field3;
}
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://localhost:6650")
.build();
AvroSchema schema =
AvroSchema.of(SchemaDefinition.builder().withPojo(Foo3.class).withAlwaysAllowNull(false).build());
Producer producer = client.newProducer(schema)
.topic("test-jdbc")
.create();
for (int i = 0; i < 20; i++) {
String key = "key-" + i;
Foo3 obj = new Foo3();
obj.setField1("field1_insert_" + i);
obj.setField2("field2_insert_" + i);
obj.setField3(i);
Map properties = Maps.newHashMap();
properties.put("EVENT", "INSERT");
producer.newMessage()
.properties(properties)
.key(key)
.value(obj)
.send();
}
for (int i = 0; i < 20; i++) {
String key = "key-" + i;
Foo3 obj = new Foo3();
obj.setField1("field1_insert_" + i);
obj.setField2("field2_update_" + i);
obj.setField3(i);
Map properties = Maps.newHashMap();
properties.put("EVENT", "UPDATE");
producer.newMessage()
.properties(properties)
.key(key)
.value(obj)
.send();
}
for (int i = 0; i < 20; i++) {
String key = "key-" + i;
Foo3 obj = new Foo3();
obj.setField1("field1_insert_" + i);
obj.setField2("field2_delete_" + i);
obj.setField3(i);
Map properties = Maps.newHashMap();
properties.put("EVENT", "DELETE");
producer.newMessage()
.properties(properties)
.key(key)
.value(obj)
.send();
}
producer.close();
client.close();
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] hrsakai commented on issue #5453: [Cpp] Fixed negative ack tracker constructor sequence
hrsakai commented on issue #5453: [Cpp] Fixed negative ack tracker constructor sequence URL: https://github.com/apache/pulsar/pull/5453#issuecomment-545684595 rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5347: [Docs]Fix website io connector link 404
tuteng commented on issue #5347: [Docs]Fix website io connector link 404 URL: https://github.com/apache/pulsar/pull/5347#issuecomment-545679941 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #5455: Docker2
aahmed-se opened a new pull request #5455: Docker2 URL: https://github.com/apache/pulsar/pull/5455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia closed pull request #3785: [pulsar-test] use localhost to prevent failure due to dns-resolution
rdhabalia closed pull request #3785: [pulsar-test] use localhost to prevent failure due to dns-resolution URL: https://github.com/apache/pulsar/pull/3785 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3986: [pulsar-broker] Introduce publish rate-limiting on topic
rdhabalia commented on issue #3986: [pulsar-broker] Introduce publish rate-limiting on topic URL: https://github.com/apache/pulsar/pull/3986#issuecomment-545646014 @merlimat can you also review this PR before we merge? we have need of this feature so, want to merge if possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size
rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size URL: https://github.com/apache/pulsar/pull/4400#issuecomment-545645477 @jiazhai sorry for late response as I had missed your comment. > After PIP36 (#4247 ), pulsar could support message size large than 5MB. That's incorrect. #4247 is not a useful feature and that could be used for a specific usecase but we can not use for large scale and multi tenant system where same bookie is serving topics from different tenants. For example, If client wants to publish message with 100MB and if broker allows to store one message with 100MB into bookie then bookie will max out in 1 or 2 messages and it can directly impact to other tenants for publish and draining backlog. Also, it will be hard to force throttling for one message with 100MB payload size. I have also described this scenario into the PIP. Right now, we have multiple users who are chunking the payload at producer side and merging at consumer side and they need this feature to avoid such complexity. So, we need this feature available for multiple users. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia closed pull request #4523: [pulsar-client] fix protobuf dependency with shaded pulsar-client
rdhabalia closed pull request #4523: [pulsar-client] fix protobuf dependency with shaded pulsar-client URL: https://github.com/apache/pulsar/pull/4523 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #5075: [pulsar-client] support set read-position based on timestamp
rdhabalia commented on issue #5075: [pulsar-client] support set read-position based on timestamp URL: https://github.com/apache/pulsar/pull/5075#issuecomment-545642622 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on a change in pull request #5440: [pulsar-perf] Add batching max message into perf-producer
rdhabalia commented on a change in pull request #5440: [pulsar-perf] Add
batching max message into perf-producer
URL: https://github.com/apache/pulsar/pull/5440#discussion_r338284046
##
File path:
pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceProducer.java
##
@@ -156,6 +156,10 @@
@Parameter(names = { "-b",
"--batch-time-window" }, description = "Batch messages in 'x'
ms window (Default: 1ms)")
public double batchTimeMillis = 1.0;
+
+@Parameter(names = { "-bn",
+"--batch-max-msgs" }, description = "Number of max messages in
batch. (Default: 0 (disable))")
+public int batchMaxMsgs = 0;
Review comment:
ok..added default values.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] rdhabalia commented on a change in pull request #5441: [pulsar-perf] record message failure and avoid exiting from process on publish failure
rdhabalia commented on a change in pull request #5441: [pulsar-perf] record
message failure and avoid exiting from process on publish failure
URL: https://github.com/apache/pulsar/pull/5441#discussion_r338280539
##
File path:
pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceProducer.java
##
@@ -497,7 +500,7 @@ private static void runProducer(Arguments arguments,
}
}).exceptionally(ex -> {
log.warn("Write error on message", ex);
-System.exit(-1);
+messagesFailed.increment();
Review comment:
added a flag to exit on failure. default is disable and it continues.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #5444: [pulsar-client] print unknown exception stacktrace
rdhabalia commented on issue #5444: [pulsar-client] print unknown exception stacktrace URL: https://github.com/apache/pulsar/pull/5444#issuecomment-545635493 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jm7647 opened a new issue #5454: mysql JDBC Sink - consumer error
jm7647 opened a new issue #5454: mysql JDBC Sink - consumer error
URL: https://github.com/apache/pulsar/issues/5454
**Describe the bug**
Running mysql jdbc sink (helped in issue #5418) but unable to receive data
from producer.
**To Reproduce**
Steps to reproduce the behavior:
1. Define Test schema and send producer data
class Test(Record):
id = Integer()
name = String()
producer = client.create_producer(
'pulsar-mysql-jdbc-sink-topic',
schema=AvroSchema(Test))
2. Receiving error in SINK logs:
19:25:55.712 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.ConsumerImpl -
[pulsar-mysql-jdbc-sink-topic][public/default/pulsar-mysql-jdbc-sink]
Subscribing to topic on cnx [id: 0x79b6a514, L:/135.25.67.33:33184 -
R:mtn21cvaas04.mt.att.com/135.25.67.33:6650]
19:25:55.734 [pulsar-client-io-1-1] INFO
org.apache.pulsar.client.impl.ConsumerImpl -
[pulsar-mysql-jdbc-sink-topic][public/default/pulsar-mysql-jdbc-sink]
Subscribed to topic on mtn21cvaas04.mt.att.com/135.25.67.33:6650 -- consumer: 0
19:29:41.376 [pulsar-client-io-1-1] WARN
com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI
library. Falling back to Java based CRC32c provider
19:29:41.427 [public/default/pulsar-mysql-jdbc-sink-0] ERROR
org.apache.pulsar.functions.instance.JavaInstanceRunnable -
[public/default/pulsar-mysql-jdbc-sink:0] Uncaught exception in Java Instance
java.lang.NullPointerException: null
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:770)
~[com.google.guava-guava-21.0.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:4153)
~[com.google.guava-guava-21.0.jar:?]
at
com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4158)
~[com.google.guava-guava-21.0.jar:?]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5147)
~[com.google.guava-guava-21.0.jar:?]
at
org.apache.pulsar.client.impl.schema.StructSchema.decode(StructSchema.java:94)
~[org.apache.pulsar-pulsar-client-original-2.4.1.jar:2.4.1]
at
org.apache.pulsar.client.impl.schema.AutoConsumeSchema.decode(AutoConsumeSchema.java:72)
~[org.apache.pulsar-pulsar-client-original-2.4.1.jar:2.4.1]
at
org.apache.pulsar.client.impl.schema.AutoConsumeSchema.decode(AutoConsumeSchema.java:36)
~[org.apache.pulsar-pulsar-client-original-2.4.1.jar:2.4.1]
at org.apache.pulsar.client.api.Schema.decode(Schema.java:97)
~[java-instance.jar:?]
at
org.apache.pulsar.client.impl.MessageImpl.getValue(MessageImpl.java:268)
~[org.apache.pulsar-pulsar-client-original-2.4.1.jar:2.4.1]
at
org.apache.pulsar.functions.source.PulsarRecord.getValue(PulsarRecord.java:74)
~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:2.4.1]
at
org.apache.pulsar.functions.instance.JavaInstanceRunnable.readInput(JavaInstanceRunnable.java:463)
~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?]
at
org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:236)
[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
19:29:41.504 [public/default/pulsar-mysql-jdbc-sink-0] INFO
org.apache.pulsar.functions.instance.JavaInstanceRunnable - Closing instance
19:29:41.510 [public/default/pulsar-mysql-jdbc-sink-0] INFO
org.apache.pulsar.client.impl.ConsumerImpl - [pulsar-mysql-jdbc-sink-topic]
[public/default/pulsar-mysql-jdbc-sink] Closed consumer
19:29:41.512 [public/default/pulsar-mysql-jdbc-sink-0] INFO
org.apache.pulsar.io.jdbc.JdbcAbstractSink - Closed jdbc connection:
jdbc:mysql://127.0.0.1:3306/pulsar_mysql_jdbc_sink?useUnicode=true&characterEncoding=utf-8&useSSL=false
19:29:41.518 [public/default/pulsar-mysql-jdbc-sink-0] INFO
org.apache.pulsar.functions.instance.JavaInstanceRunnable - Unloading JAR files
for function InstanceConfig(instanceId=0,
functionId=5703beea-ca01-446b-8d1a-35167aeb2cac,
functionVersion=df4c3460-d718-4684-9109-3d081cfb6f1e, functionDetails=tenant:
"public"
namespace: "default"
name: "pulsar-mysql-jdbc-sink"
className: "org.apache.pulsar.functions.api.utils.IdentityFunction"
autoAck: true
parallelism: 1
source {
typeClassName: "org.apache.pulsar.client.api.schema.GenericRecord"
inputSpecs {
key: "pulsar-mysql-jdbc-sink-topic"
value {
}
}
cleanupSubscription: true
}
sink {
className: "org.apache.pulsar.io.jdbc.JdbcAutoSchemaSink"
configs:
"{\"userName\":\"root\",\"password\":\"jdbc\",\"jdbcUrl\":\"jdbc:mysql://127.0.0.1:3306/pulsar_mysql_jdbc_sink?useUnicode\\u003dtrue\\u0026characterEncoding\\u003dutf-8\\u0026useSSL\\u003dfalse\",\"tableName\":\"pulsar_mysql_jdbc_sink\"}"
ty
[GitHub] [pulsar] merlimat opened a new pull request #5453: [Cpp] Fixed negative ack tracker constructor sequence
merlimat opened a new pull request #5453: [Cpp] Fixed negative ack tracker constructor sequence URL: https://github.com/apache/pulsar/pull/5453 ### Motivation Negative ack tracker can leave the `timerInterval_` not properly initialized given the initialization sequence. When that happens, the timer interval is set to garbage value and the task won't be triggered. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #5443: [pulsar-client] Fix message corruption on OOM for batch messages
rdhabalia commented on issue #5443: [pulsar-client] Fix message corruption on OOM for batch messages URL: https://github.com/apache/pulsar/pull/5443#issuecomment-545591086 @merlimat addressed the comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #5411: Add test for links to other markdown docs.
jerrypeng commented on issue #5411: Add test for links to other markdown docs. URL: https://github.com/apache/pulsar/pull/5411#issuecomment-545585603 rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #5410: Fix broker function token authentication link
jerrypeng commented on issue #5410: Fix broker function token authentication link URL: https://github.com/apache/pulsar/pull/5410#issuecomment-545585425 rerun java8 tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #5409: Fix broken io connector links.
jerrypeng commented on issue #5409: Fix broken io connector links. URL: https://github.com/apache/pulsar/pull/5409#issuecomment-545585124 rerun cpp tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng opened a new issue #5452: PersistentTopicMessagingTest and NonPersistentTopicMessagingTest broken and not running
jerrypeng opened a new issue #5452: PersistentTopicMessagingTest and NonPersistentTopicMessagingTest broken and not running URL: https://github.com/apache/pulsar/issues/5452 Describe the bug Topic integration tests in PersistentTopicMessagingTest and NonPersistentTopicMessagingTest are not run because they are not added in https://github.com/apache/pulsar/blob/master/tests/integration/src/test/resources/pulsar.xml. Some of the tests also appear to be broken This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng opened a new issue #5451: SchemaUpdateStrategyTest is broken and not run
jerrypeng opened a new issue #5451: SchemaUpdateStrategyTest is broken and not run URL: https://github.com/apache/pulsar/issues/5451 **Describe the bug** Schema integration tests in SchemaUpdateStrategyTest are not run because they are not added in https://github.com/apache/pulsar/blob/master/tests/integration/src/test/resources/pulsar.xml. Some of the tests also appear to be broken This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on a change in pull request #5440: [pulsar-perf] Add batching max message into perf-producer
merlimat commented on a change in pull request #5440: [pulsar-perf] Add
batching max message into perf-producer
URL: https://github.com/apache/pulsar/pull/5440#discussion_r338194389
##
File path:
pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceProducer.java
##
@@ -156,6 +156,10 @@
@Parameter(names = { "-b",
"--batch-time-window" }, description = "Batch messages in 'x'
ms window (Default: 1ms)")
public double batchTimeMillis = 1.0;
+
+@Parameter(names = { "-bn",
+"--batch-max-msgs" }, description = "Number of max messages in
batch. (Default: 0 (disable))")
+public int batchMaxMsgs = 0;
Review comment:
Leaving the default to 0 is a bit confusing since it will not "disabling"
the batching, nor the default max batch size of 1000 messages. Should we
instead use 1000 as default here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #5389: [Docs] Add delete-dynamic-config command for brokers
merlimat merged pull request #5389: [Docs] Add delete-dynamic-config command for brokers URL: https://github.com/apache/pulsar/pull/5389 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (5d1529a -> e97ea92)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 5d1529a Give compacted ledger some time to update in python test(#2321) (#5448) add e97ea92 [Docs] Add delete-dynamic-config command for brokers (#5389) No new revisions were added by this update. Summary of changes: site2/docs/reference-pulsar-admin.md | 15 +++ 1 file changed, 15 insertions(+)
[pulsar] branch master updated (8b7245c -> 5d1529a)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 8b7245c Make Function Authentication Provider pluggable (#5404) add 5d1529a Give compacted ledger some time to update in python test(#2321) (#5448) No new revisions were added by this update. Summary of changes: pulsar-client-cpp/python/pulsar_test.py | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-)
[GitHub] [pulsar] merlimat merged pull request #5448: Give compacted ledger some time to update in python test (#2321)
merlimat merged pull request #5448: Give compacted ledger some time to update in python test (#2321) URL: https://github.com/apache/pulsar/pull/5448 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat closed issue #2321: Flaky python unit test test_publish_compact_and_consume
merlimat closed issue #2321: Flaky python unit test test_publish_compact_and_consume URL: https://github.com/apache/pulsar/issues/2321 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (13ea25a -> 7b0b550)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 13ea25a [pulsar-broker] Fix: race condition : Failed to read-more entries on dispatcher (#5391) add 7b0b550 Update version of dockerfile-maven plugin (#5416) No new revisions were added by this update. Summary of changes: pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (7b0b550 -> 8b7245c)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 7b0b550 Update version of dockerfile-maven plugin (#5416) add 8b7245c Make Function Authentication Provider pluggable (#5404) No new revisions were added by this update. Summary of changes: .../org/apache/pulsar/functions/LocalRunner.java | 3 +- .../functions/auth/FunctionAuthProvider.java | 5 + .../auth/KubernetesFunctionAuthProvider.java | 8 ++ .../auth/KubernetesSecretsTokenAuthProvider.java | 8 +- .../functions/auth/NoOpFunctionAuthProvider.java | 49 -- .../functions/runtime/KubernetesRuntime.java | 8 +- .../runtime/KubernetesRuntimeFactory.java | 35 +-- .../functions/runtime/ProcessRuntimeFactory.java | 18 +++- .../pulsar/functions/runtime/RuntimeFactory.java | 7 +- .../KubernetesSecretsTokenAuthProviderTest.java| 13 ++- .../runtime/KubernetesRuntimeFactoryTest.java | 97 ++- .../functions/runtime/KubernetesRuntimeTest.java | 4 +- .../functions/runtime/ProcessRuntimeTest.java | 3 +- .../pulsar/functions/worker/FunctionActioner.java | 30 +++--- .../functions/worker/FunctionRuntimeManager.java | 15 ++- .../pulsar/functions/worker/WorkerConfig.java | 8 ++ .../functions/worker/rest/api/FunctionsImpl.java | 104 +++-- .../functions/worker/rest/api/SinksImpl.java | 96 ++- .../functions/worker/rest/api/SourcesImpl.java | 96 ++- .../functions/worker/FunctionActionerTest.java | 7 +- 20 files changed, 377 insertions(+), 237 deletions(-) delete mode 100644 pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/NoOpFunctionAuthProvider.java
[GitHub] [pulsar] merlimat merged pull request #5416: Update version of dockerfile-maven plugin
merlimat merged pull request #5416: Update version of dockerfile-maven plugin URL: https://github.com/apache/pulsar/pull/5416 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #5404: Make Function Authentication Provider pluggable
merlimat merged pull request #5404: Make Function Authentication Provider pluggable URL: https://github.com/apache/pulsar/pull/5404 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5435: Add support for partitioned topic consumer seek by time.
codelipenghui commented on issue #5435: Add support for partitioned topic consumer seek by time. URL: https://github.com/apache/pulsar/pull/5435#issuecomment-545502737 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ivankelly commented on issue #5448: Give compacted ledger some time to update in python test (#2321)
ivankelly commented on issue #5448: Give compacted ledger some time to update in python test (#2321) URL: https://github.com/apache/pulsar/pull/5448#issuecomment-545471925 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] bright666 opened a new issue #5450: Some functional requirements about pulsar sql
bright666 opened a new issue #5450: Some functional requirements about pulsar sql URL: https://github.com/apache/pulsar/issues/5450 **Is your feature request related to a problem? Please describe.** There is currently a need to manage and display pulsar's unconsumed metadata. But currently pulsar SQL will display the consumed data. **Describe the solution you'd like** According to the cursor to get the largest markDeletePosition, which is the location of the last spent cursor, and then reassemble the messageId, like this __message_id__ > '(850,0,0)'. Data pagination can then be paged according to __publish_time__, like this: __publish_time__ > timestamp'2019-10-23 21:32:00' LIMIT 100.If it is a partition theme, filter the corresponding data according to the partition. **Describe alternatives you've considered** #Use the consumer to get the data directly, but not ack, do this to the data display. # If pulsar is logically deleted, it should have a logo, which can consume the consumed data according to that identifier. Finally thank all the staff of pulsar. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers
wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers URL: https://github.com/apache/pulsar/pull/5389#issuecomment-545409717 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers
wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers URL: https://github.com/apache/pulsar/pull/5389#issuecomment-545409649 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr opened a new issue #5449: function worker v2.4.1 issues on dependency shared
KannarFr opened a new issue #5449: function worker v2.4.1 issues on dependency shared URL: https://github.com/apache/pulsar/issues/5449 **Describe the bug** We bumped our functions workers to v2.4.1 from v2.4.0 and the function which is using several dependencies can't boot because of the following stack: ``` 13:10:51.038 [clevercloud/functions/accessLogsCleverCloudADCHaproxy-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [clevercloud/functions/accessLogsCleverCloudADCHaproxy:0] Uncaught exception in Java Instance java.lang.RuntimeException: User class constructor throws exception at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:126) ~[org.apache.pulsar-pulsar-functions-utils-2.4.1.jar:2.4.1] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupJavaInstance(JavaInstanceRunnable.java:189) ~[org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?] at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:234) [org.apache.pulsar-pulsar-functions-instance-2.4.1.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_192] Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_192] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_192] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_192] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_192] at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:118) ~[org.apache.pulsar-pulsar-functions-utils-2.4.1.jar:2.4.1] ... 3 more Caused by: java.lang.LinkageError: ClassCastException: attempting to castjar:file:/pulsar/lib/javax.ws.rs-javax.ws.rs-api-2.1.jar!/javax/ws/rs/client/ClientBuilder.class to file:/tmp/pulsar-nar/pulsar-functions-0.1.0-SNAPSHOT.jar-unpacked/javax/ws/rs/client/ClientBuilder.class at javax.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:81) ~[javax.ws.rs-javax.ws.rs-api-2.1.jar:2.1] at javax.ws.rs.client.ClientBuilder.newClient(ClientBuilder.java:97) ~[javax.ws.rs-javax.ws.rs-api-2.1.jar:2.1] at com.clevercloud.pulsar.util.GeoIPAPI.updateDatabase(GeoIPAPI.java:101) ~[?:?] at com.clevercloud.pulsar.util.GeoIPAPI.(GeoIPAPI.java:45) ~[?:?] at com.clevercloud.pulsar.function.ApplicationsAddonsHaproxyAccessLogs.(ApplicationsAddonsHaproxyAccessLogs.java:28) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_192] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_192] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_192] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_192] at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:118) ~[org.apache.pulsar-pulsar-functions-utils-2.4.1.jar:2.4.1] ... 3 more 13:10:51.047 [clevercloud/functions/accessLogsCleverCloudADCHaproxy-0] INFO org.apache.pulsar.functions.instance.JavaInstanceRunnable - Closing instance This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ivankelly commented on issue #2321: Flaky python unit test test_publish_compact_and_consume
ivankelly commented on issue #2321: Flaky python unit test test_publish_compact_and_consume URL: https://github.com/apache/pulsar/issues/2321#issuecomment-545406302 For the record I only managed one natural repro of this (with stress-ng --aggressive --class scheduler --all 5). It appears to be a very narrow race. It can be widened by changing the thenAccept to thenAcceptAsync at https://github.com/apache/pulsar/blob/13ea25a3e0b1d4b01ec678049964816a1275a95f/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/CompactorSubscription.java#L99, and then putting a sleep in the callback. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ivankelly opened a new pull request #5448: Give compacted ledger some time to update in python test (#2321)
ivankelly opened a new pull request #5448: Give compacted ledger some time to update in python test (#2321) URL: https://github.com/apache/pulsar/pull/5448 After compaction completes the compacted ledger is recorded as a property of a cursor. As persisting the cursor is async and we don't wait for the acknowledgement of the acknowledgement, there may be a race if we try to read the compacted ledger immediately. To avoid this race, I've added a 1 second sleep to allow the compacted ledger to be updated on the broker. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5435: Add support for partitioned topic consumer seek by time.
aahmed-se commented on issue #5435: Add support for partitioned topic consumer seek by time. URL: https://github.com/apache/pulsar/pull/5435#issuecomment-545393271 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5416: Update version of dockerfile-maven plugin
aahmed-se commented on issue #5416: Update version of dockerfile-maven plugin URL: https://github.com/apache/pulsar/pull/5416#issuecomment-545392996 rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (81e7950 -> 13ea25a)
This is an automated email from the ASF dual-hosted git repository. aahmed pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 81e7950 [Issue 5362][website]Update logo path for Pulsar users (#5374) add 13ea25a [pulsar-broker] Fix: race condition : Failed to read-more entries on dispatcher (#5391) No new revisions were added by this update. Summary of changes: .../PersistentDispatcherMultipleConsumers.java | 28 ++ 1 file changed, 7 insertions(+), 21 deletions(-)
[GitHub] [pulsar] aahmed-se merged pull request #5391: [pulsar-broker] Fix: race condition : Failed to read-more entries on dispatcher
aahmed-se merged pull request #5391: [pulsar-broker] Fix: race condition : Failed to read-more entries on dispatcher URL: https://github.com/apache/pulsar/pull/5391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed issue #5362: [website] Some company logos are not displayed well on poweredby website
aahmed-se closed issue #5362: [website] Some company logos are not displayed well on poweredby website URL: https://github.com/apache/pulsar/issues/5362 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (492d92b -> 81e7950)
This is an automated email from the ASF dual-hosted git repository. aahmed pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 492d92b Change state_ to closed when resultOk is returned (#5446) add 81e7950 [Issue 5362][website]Update logo path for Pulsar users (#5374) No new revisions were added by this update. Summary of changes: site2/website/data/users.js | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-)
[GitHub] [pulsar] aahmed-se merged pull request #5374: [Issue 5362][website]Update logo path for Pulsar users
aahmed-se merged pull request #5374: [Issue 5362][website]Update logo path for Pulsar users URL: https://github.com/apache/pulsar/pull/5374 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5356: [TIEREDSTORAGE] Only seek when reading unexpected entry
aahmed-se commented on issue #5356: [TIEREDSTORAGE] Only seek when reading unexpected entry URL: https://github.com/apache/pulsar/pull/5356#issuecomment-545382579 @ivankelly can you address the comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Replace foreach with System.arraycopy (#5424)
This is an automated email from the ASF dual-hosted git repository.
aahmed pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new e73fc00 Replace foreach with System.arraycopy (#5424)
e73fc00 is described below
commit e73fc0007bd60cfa21accb819abacd4b3ebccd40
Author: Like
AuthorDate: Wed Oct 23 18:38:36 2019 +0800
Replace foreach with System.arraycopy (#5424)
---
.../main/java/org/apache/pulsar/broker/stats/NamespaceStats.java | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/NamespaceStats.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/NamespaceStats.java
index e5c8735..6b7ce93 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/NamespaceStats.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/NamespaceStats.java
@@ -18,6 +18,7 @@
*/
package org.apache.pulsar.broker.stats;
+import java.util.Arrays;
import java.util.Map;
import static
org.apache.bookkeeper.mledger.impl.ManagedLedgerMBeanImpl.ENTRY_LATENCY_BUCKETS_USEC;
@@ -107,16 +108,13 @@ public class NamespaceStats {
public static void copy(long[] src, long[] dest) {
if (src != null && dest != null && src.length == dest.length) {
-for (int i = 0; i < src.length; i++) {
-dest[i] = src[i];
-}
+System.arraycopy(src, 0, dest, 0, src.length);
}
}
public static void clear(long[] list) {
if (list != null) {
-for (int i = 0; i < list.length; i++)
-list[i] = 0;
+Arrays.fill(list, 0);
}
}
[pulsar] branch master updated: Change state_ to closed when resultOk is returned (#5446)
This is an automated email from the ASF dual-hosted git repository.
aahmed pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 492d92b Change state_ to closed when resultOk is returned (#5446)
492d92b is described below
commit 492d92bd2bc14f8a4eed22bc9208d86141dc1c95
Author: hrsakai
AuthorDate: Wed Oct 23 19:39:15 2019 +0900
Change state_ to closed when resultOk is returned (#5446)
---
pulsar-client-cpp/lib/ConsumerImpl.cc | 6 +-
pulsar-client-cpp/lib/ProducerImpl.cc | 6 +-
2 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/pulsar-client-cpp/lib/ConsumerImpl.cc
b/pulsar-client-cpp/lib/ConsumerImpl.cc
index 1c616fe..1313a05 100644
--- a/pulsar-client-cpp/lib/ConsumerImpl.cc
+++ b/pulsar-client-cpp/lib/ConsumerImpl.cc
@@ -837,8 +837,12 @@ void ConsumerImpl::closeAsync(ResultCallback callback) {
return;
}
+LOG_INFO(getName() << "Closing consumer for topic " << topic_);
+state_ = Closing;
+
ClientConnectionPtr cnx = getCnx().lock();
if (!cnx) {
+state_ = Closed;
lock.unlock();
// If connection is gone, also the consumer is closed on the broker
side
if (callback) {
@@ -847,9 +851,9 @@ void ConsumerImpl::closeAsync(ResultCallback callback) {
return;
}
-LOG_INFO(getName() << "Closing consumer for topic " << topic_);
ClientImplPtr client = client_.lock();
if (!client) {
+state_ = Closed;
lock.unlock();
// Client was already destroyed
if (callback) {
diff --git a/pulsar-client-cpp/lib/ProducerImpl.cc
b/pulsar-client-cpp/lib/ProducerImpl.cc
index 220a9f8..8159c8f 100644
--- a/pulsar-client-cpp/lib/ProducerImpl.cc
+++ b/pulsar-client-cpp/lib/ProducerImpl.cc
@@ -492,6 +492,7 @@ void ProducerImpl::closeAsync(CloseCallback callback) {
ClientConnectionPtr cnx = getCnx().lock();
if (!cnx) {
+state_ = Closed;
lock.unlock();
if (callback) {
callback(ResultOk);
@@ -502,16 +503,19 @@ void ProducerImpl::closeAsync(CloseCallback callback) {
// Detach the producer from the connection to avoid sending any other
// message from the producer
connection_.reset();
-lock.unlock();
ClientImplPtr client = client_.lock();
if (!client) {
+state_ = Closed;
+lock.unlock();
// Client was already destroyed
if (callback) {
callback(ResultOk);
}
return;
}
+
+lock.unlock();
int requestId = client->newRequestId();
Future future =
cnx->sendRequestWithId(Commands::newCloseProducer(producerId_,
requestId), requestId);
[GitHub] [pulsar] aahmed-se merged pull request #5446: [pulsar-client-cpp]Change state_ to closed when resultOk is returned
aahmed-se merged pull request #5446: [pulsar-client-cpp]Change state_ to closed when resultOk is returned URL: https://github.com/apache/pulsar/pull/5446 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se merged pull request #5424: Replace foreach with System.arraycopy
aahmed-se merged pull request #5424: Replace foreach with System.arraycopy URL: https://github.com/apache/pulsar/pull/5424 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #5426: CI Experiment - Do not merge
aahmed-se closed pull request #5426: CI Experiment - Do not merge URL: https://github.com/apache/pulsar/pull/5426 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #3905: Add additional fields for Field Docs
aahmed-se closed pull request #3905: Add additional fields for Field Docs URL: https://github.com/apache/pulsar/pull/3905 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #3772: Force before and after methods to always run
aahmed-se closed pull request #3772: Force before and after methods to always run URL: https://github.com/apache/pulsar/pull/3772 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se closed pull request #2872: Update to use use new vagrant images
aahmed-se closed pull request #2872: Update to use use new vagrant images URL: https://github.com/apache/pulsar/pull/2872 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (ddf5429 -> 84a519f)
This is an automated email from the ASF dual-hosted git repository. nkurihar pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from ddf5429 Fixed StringSchema static initialization (#5445) add 84a519f [pulsar-broker] Fix bug that namespace policies does not take effect due to NPE (#5408) No new revisions were added by this update. Summary of changes: .../org/apache/pulsar/broker/service/BrokerService.java| 5 +++-- .../pulsar/broker/service/persistent/PersistentTopic.java | 14 ++ 2 files changed, 13 insertions(+), 6 deletions(-)
[GitHub] [pulsar] nkurihar merged pull request #5408: [pulsar-broker] Fix bug that namespace policies does not take effect due to NPE
nkurihar merged pull request #5408: [pulsar-broker] Fix bug that namespace policies does not take effect due to NPE URL: https://github.com/apache/pulsar/pull/5408 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers
wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers URL: https://github.com/apache/pulsar/pull/5389#issuecomment-545306874 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers
wolfstudy commented on issue #5389: [Docs] Add delete-dynamic-config command for brokers URL: https://github.com/apache/pulsar/pull/5389#issuecomment-545307090 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5389: [Docs] Add delete-dynamic-config command for brokers
wolfstudy edited a comment on issue #5389: [Docs] Add delete-dynamic-config command for brokers URL: https://github.com/apache/pulsar/pull/5389#issuecomment-545306874 run integration tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] svenhornberg opened a new issue #5447: Apache Beam support for Pulsar
svenhornberg opened a new issue #5447: Apache Beam support for Pulsar URL: https://github.com/apache/pulsar/issues/5447 I would like to use Apache Beam and Pulsar as my source, for writing my batch or streaming jobs. Please consider adding a PulsarIO class to the sdk like Kafka (https://beam.apache.org/releases/javadoc/2.4.0/org/apache/beam/sdk/io/kafka/KafkaIO.html) I found the pulsar-flink and pulsar-spark folders, but i really like the Apache Beam API. Maybe you could add a pulsar-beam example or subfolder This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on issue #5148: Validate topic name before creating partition/non partition topic via admin cli.
MarvinCai commented on issue #5148: Validate topic name before creating partition/non partition topic via admin cli. URL: https://github.com/apache/pulsar/pull/5148#issuecomment-545300463 @jiazhai I addressed your comments, can you please take another look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
