[GitHub] [pulsar-client-go] grtfou commented on issue #112: Can't install go module.
grtfou commented on issue #112: Can't install go module. URL: https://github.com/apache/pulsar-client-go/issues/112#issuecomment-565948483 Install module fail. ``` $ go get -u github.com/apache/pulsar/pulsar-client-go/pulsar go: finding github.com/apache/pulsar/pulsar-client-go/pulsar latest go: finding github.com/apache/pulsar/pulsar-client-go latest go: finding golang.org/x/crypto latest go: finding golang.org/x/sys latest go: finding golang.org/x/tools latest go: finding gopkg.in/check.v1 latest go: finding golang.org/x/net latest go: finding golang.org/x/sync latest go: finding golang.org/x/xerrors latest # github.com/apache/pulsar/pulsar-client-go/pulsar In file included from ../../../pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191214024039-b52449a6604c/pulsar/c_client.go:24: ./c_go_pulsar.h:22:10: fatal error: 'pulsar/c/client.h' file not found #include ^~~ 1 error generated. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] grtfou edited a comment on issue #112: Can't install go module.
grtfou edited a comment on issue #112: Can't install go module. URL: https://github.com/apache/pulsar-client-go/issues/112#issuecomment-565948483 Install module fail. ``` $ go get -u github.com/apache/pulsar/pulsar-client-go/pulsar go: finding github.com/apache/pulsar/pulsar-client-go/pulsar latest go: finding github.com/apache/pulsar/pulsar-client-go latest go: finding golang.org/x/crypto latest go: finding golang.org/x/sys latest go: finding golang.org/x/tools latest go: finding gopkg.in/check.v1 latest go: finding golang.org/x/net latest go: finding golang.org/x/sync latest go: finding golang.org/x/xerrors latest # github.com/apache/pulsar/pulsar-client-go/pulsar In file included from ../../../pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191214024039-b52449a6604c/pulsar/c_client.go:24: ./c_go_pulsar.h:22:10: fatal error: 'pulsar/c/client.h' file not found #include ^~~ 1 error generated. $ go version go version go1.12.4 darwin/amd64 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] grtfou edited a comment on issue #112: Can't install go module.
grtfou edited a comment on issue #112: Can't install go module. URL: https://github.com/apache/pulsar-client-go/issues/112#issuecomment-565948483 Install module fail by latest version (v20191214024039). ``` $ go get -u github.com/apache/pulsar/pulsar-client-go/pulsar go: finding github.com/apache/pulsar/pulsar-client-go/pulsar latest go: finding github.com/apache/pulsar/pulsar-client-go latest go: finding golang.org/x/crypto latest go: finding golang.org/x/sys latest go: finding golang.org/x/tools latest go: finding gopkg.in/check.v1 latest go: finding golang.org/x/net latest go: finding golang.org/x/sync latest go: finding golang.org/x/xerrors latest # github.com/apache/pulsar/pulsar-client-go/pulsar In file included from ../../../pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191214024039-b52449a6604c/pulsar/c_client.go:24: ./c_go_pulsar.h:22:10: fatal error: 'pulsar/c/client.h' file not found #include ^~~ 1 error generated. $ go version go version go1.12.4 darwin/amd64 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] murong00 commented on issue #5843: Duplicate flags when running bin/pulsar bookie
murong00 commented on issue #5843: Duplicate flags when running bin/pulsar bookie URL: https://github.com/apache/pulsar/issues/5843#issuecomment-565950644 @hangc0276 yes, it is not elegant to override the setting, from my point view I can accept both as long as the doc is detailed. The issue here is duplicate flags may confuse some users. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-translation] Jennifer88huang commented on issue #16: Trans docs
Jennifer88huang commented on issue #16: Trans docs URL: https://github.com/apache/pulsar-translation/pull/16#issuecomment-565966569 @sijia-w which one will you finish this week? Please add the translation link in the description when pull request. Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] frank-dkvan commented on issue #4976: Pulsar SQL can not query the latest ingested event in a topic , but Pulsar Consumer/Read always can.
frank-dkvan commented on issue #4976: Pulsar SQL can not query the latest ingested event in a topic , but Pulsar Consumer/Read always can. URL: https://github.com/apache/pulsar/issues/4976#issuecomment-565981876 > @frank-dkvan > > I think you can add `bookkeeperExplicitLacIntervalInMills=5` in the JVM system properties when starting the broker. Hi @codelipenghui "5" means there would be 5-ms delay before we can see the latest message in Pulsar SQL? Am I understanding correctly? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] frank-dkvan commented on issue #4976: Pulsar SQL can not query the latest ingested event in a topic , but Pulsar Consumer/Read always can.
frank-dkvan commented on issue #4976: Pulsar SQL can not query the latest ingested event in a topic , but Pulsar Consumer/Read always can. URL: https://github.com/apache/pulsar/issues/4976#issuecomment-565982279 > @frank-dkvan @sijie I have create a new pull request #5822 for expose the config in broker.conf. Thanks @codelipenghui This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] murong00 commented on issue #5844: [Reopen][Issue 5597] Retry when getPartitionedTopicMetadata failed
murong00 commented on issue #5844: [Reopen][Issue 5597] Retry when getPartitionedTopicMetadata failed URL: https://github.com/apache/pulsar/pull/5844#issuecomment-565982406 @jiazhai ok, will improve it later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ant8e opened a new pull request #5864: [doc] Fix Typo in 'schema get started'
ant8e opened a new pull request #5864: [doc] Fix Typo in 'schema get started' URL: https://github.com/apache/pulsar/pull/5864 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng opened a new pull request #5865: [website]Fix version replace for old version document
tuteng opened a new pull request #5865: [website]Fix version replace for old version document URL: https://github.com/apache/pulsar/pull/5865 Fixes https://github.com/apache/pulsar/issues/5839 Master Issue: https://github.com/apache/pulsar/issues/5839 ### Motivation Currently, for different contents, we use the same variables in multiple versions. ### Modifications * When replacing the latest version, the old version is ignored * Set option `dry` to true for enable replace ### Verifying this change yarn build node scripts/replace.js This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on a change in pull request #124: [Issue #123] Ensure message is sent if no room in current batch.
wolfstudy commented on a change in pull request #124: [Issue #123] Ensure
message is sent if no room in current batch.
URL: https://github.com/apache/pulsar-client-go/pull/124#discussion_r358188134
##
File path: pulsar/producer_partition.go
##
@@ -256,10 +256,17 @@ func (p *partitionProducer) internalSend(request
*sendRequest) {
}
if sendAsBatch {
- ok := p.batchBuilder.Add(smm, sequenceID, msg.Payload, request,
msg.ReplicationClusters)
- if ok == false {
+ added := p.batchBuilder.Add(smm, sequenceID, msg.Payload,
request, msg.ReplicationClusters)
+ if !added {
// The current batch is full.. flush it and retry
p.internalFlushCurrentBatch()
+
+ // after flushing try again to add the current payload
+ if ok := p.batchBuilder.Add(smm, sequenceID,
msg.Payload, request, msg.ReplicationClusters); !ok {
+ p.log.WithField("size", len(msg.Payload)).
+ WithField("sequenceID", sequenceID).
+ Error("unable to send message")
+ }
}
} else {
// Send individually
Review comment:
Agree with @tuteng , we need to handle the return value of
`p.batchBuilder.Add()`:
```
if add := p.batchBuilder.Add(smm, sequenceID, msg.Payload, request,
msg.ReplicationClusters); !add {
// print warn log
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy commented on issue #107: Producer deadlocks when pulsar downtime
wolfstudy commented on issue #107: Producer deadlocks when pulsar downtime URL: https://github.com/apache/pulsar-client-go/issues/107#issuecomment-566027195 @reugn Maybe the issue is similar to #124. Can you try it again? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy commented on issue #5695: Go: Segfault on messageId serialize after
getting it from producer
URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566044885
Thanks @lspgn, Unfortunately, I can't reproduce the issue in my local.
Use code as follows:
```
package main
import (
"context"
"fmt"
log "github.com/apache/pulsar/pulsar-client-go/logutil"
"github.com/apache/pulsar/pulsar-client-go/pulsar"
)
func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
IOThreads: 5,
})
if err != nil {
log.Fatal(err)
}
defer client.Close()
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "topic-1",
})
if err != nil {
log.Fatal(err)
}
defer producer.Close()
ctx := context.Background()
for i := 0; i < 1000; i++ {
msg := pulsar.ProducerMessage{
Payload: []byte(fmt.Sprintf("hello-%d", i)),
}
msgID, err := producer.SendAndGetMsgID(ctx, msg)
if err != nil {
log.Fatal(err)
}
fmt.Printf("the %s successfully published with id %v\n",
string(msg.Payload), msgID.Serialize())
}
}
```
The broker version:
- 2.4.1
- 2.4.2
- 2.5.0
The client version:
- github.com/apache/pulsar/pulsar-client-go
v0.0.0-20191214024039-b52449a6604c
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize
after getting it from producer
URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566044885
Thanks @lspgn, Unfortunately, I can't reproduce the issue in my local.
Use code as follows:
```
package main
import (
"context"
"fmt"
log "github.com/apache/pulsar/pulsar-client-go/logutil"
"github.com/apache/pulsar/pulsar-client-go/pulsar"
)
func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
IOThreads: 5,
})
if err != nil {
log.Fatal(err)
}
defer client.Close()
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "topic-1",
})
if err != nil {
log.Fatal(err)
}
defer producer.Close()
ctx := context.Background()
for i := 0; i < 1000; i++ {
msg := pulsar.ProducerMessage{
Payload: []byte(fmt.Sprintf("hello-%d", i)),
}
msgID, err := producer.SendAndGetMsgID(ctx, msg)
if err != nil {
log.Fatal(err)
}
fmt.Printf("the %s successfully published with id %v\n",
string(msg.Payload), msgID.Serialize())
}
}
```
The broker version:
- 2.4.1
- 2.4.2
- 2.5.0
The client version:
- github.com/apache/pulsar/pulsar-client-go
v0.0.0-20191214024039-b52449a6604c
Can you provide a stable reproduce code?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize
after getting it from producer
URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566044885
Thanks @lspgn, Unfortunately, I can't reproduce the issue in my local.
Use code as follows:
```
package main
import (
"context"
"fmt"
log "github.com/apache/pulsar/pulsar-client-go/logutil"
"github.com/apache/pulsar/pulsar-client-go/pulsar"
)
func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
IOThreads: 5,
})
if err != nil {
log.Fatal(err)
}
defer client.Close()
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "topic-1",
})
if err != nil {
log.Fatal(err)
}
defer producer.Close()
ctx := context.Background()
for i := 0; i < 1000; i++ {
msg := pulsar.ProducerMessage{
Payload: []byte(fmt.Sprintf("hello-%d", i)),
}
msgID, err := producer.SendAndGetMsgID(ctx, msg)
if err != nil {
log.Fatal(err)
}
fmt.Printf("the %s successfully published with id %v\n",
string(msg.Payload), msgID.Serialize())
}
}
```
The broker version:
- 2.4.1
- 2.4.2
- 2.5.0
The client version:
- github.com/apache/pulsar/pulsar-client-go
v0.0.0-20191214024039-b52449a6604c
Build cpp lib:
```
$ cd pulsar-client-cpp
$ cmake .
$ make -j12 && make install
```
Can you provide a stable reproduce code?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize
after getting it from producer
URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566044885
Thanks @lspgn, Unfortunately, I can't reproduce the issue in my local.
Use code as follows:
```
package main
import (
"context"
"fmt"
log "github.com/apache/pulsar/pulsar-client-go/logutil"
"github.com/apache/pulsar/pulsar-client-go/pulsar"
)
func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
IOThreads: 5,
})
if err != nil {
log.Fatal(err)
}
defer client.Close()
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "topic-1",
})
if err != nil {
log.Fatal(err)
}
defer producer.Close()
ctx := context.Background()
for i := 0; i < 1000; i++ {
msg := pulsar.ProducerMessage{
Payload: []byte(fmt.Sprintf("hello-%d", i)),
}
msgID, err := producer.SendAndGetMsgID(ctx, msg)
if err != nil {
log.Fatal(err)
}
fmt.Printf("the %s successfully published with id %v\n",
string(msg.Payload), msgID.Serialize())
}
}
```
Output as follows:
```
the hello-18773 successfully published with id [8 12 16 213 146 1]
the hello-18774 successfully published with id [8 12 16 214 146 1]
the hello-18775 successfully published with id [8 12 16 215 146 1]
the hello-18776 successfully published with id [8 12 16 216 146 1]
the hello-18777 successfully published with id [8 12 16 217 146 1]
the hello-18778 successfully published with id [8 12 16 218 146 1]
the hello-18779 successfully published with id [8 12 16 219 146 1]
the hello-18780 successfully published with id [8 12 16 220 146 1]
```
The broker version:
- 2.4.1
- 2.4.2
- 2.5.0
The client version:
- github.com/apache/pulsar/pulsar-client-go
v0.0.0-20191214024039-b52449a6604c
Build cpp lib:
```
$ cd pulsar-client-cpp
$ cmake .
$ make -j12 && make install
```
Can you provide a stable reproduce code?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] subbu107 opened a new issue #5866: Flink event time based windowing functions not working with pulsar
subbu107 opened a new issue #5866: Flink event time based windowing functions
not working with pulsar
URL: https://github.com/apache/pulsar/issues/5866
We are using Event time based windowing functions in Flink. The source is a
partitioned Pulsar topic . The timestamp field is decided by the processing
function in flink.
The event time is extracted from the message by attaching a
TimeStampExtractor to the stream which extracts time from the message
Expected behavior
The window should get triggered once the watermark is crossed.
Actual behavior
We noticed that the window does not triggered though the watermark is
crossed.
Steps to reproduce
Attaching the sample code.
[pulsar-flink-preload-new.zip](https://github.com/apache/pulsar/files/3968426/pulsar-flink-preload-new.zip)
System configuration
**Pulsar version**: 2.4.2
Flink version : 1.8.2
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lspgn commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer
lspgn commented on issue #5695: Go: Segfault on messageId serialize after
getting it from producer
URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566095147
@wolfstudy thanks a lot.
This code is also giving me the segfault.
I realize the only variable in our setups is I'm using a Pulsar
docker-container.
```
version: '3'
services:
pulsar:
image: 'apachepulsar/pulsar:2.4.2'
ports:
- '6650:6650'
- '8080:8080'
command: bin/pulsar standalone
```
(either 2.4.1 or 2.4.2, there's no 2.5.0 yet on Docker Hub).
```
$ docker inspect apachepulsar/pulsar:2.4.2
[
{
"Id":
"sha256:08287f8732ada886938e1cad6a635f74ec3d55e2386488839928ec95a1c3baea
```
My hunch is: on Mac OS, Docker is in a VM, for heavy tasks, they are usually
slowed down. Would it be possible that the broker does not reply with the ID
fast enough to the client?
Will try with non VM pulsar and let you know.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] julio666 opened a new pull request #5867: [issue 5847][dashboard] postgresql correct version (9.6 to 11)
julio666 opened a new pull request #5867: [issue 5847][dashboard] postgresql correct version (9.6 to 11) URL: https://github.com/apache/pulsar/pull/5867 Fixes #5847 ### Motivation *Pulsar dashboard was not starting correctly anymore because the postgresql version pulled by the docker image was not 9.6 anymore but 11.* ### Modifications *I have changed reference to 9.6 version with 11 version. Maybe I could have made something more automatic, like looking for the version without hardcoding it, but I not very confident with this king of stuff :)* ### Verifying this change This change is a trivial rework / code cleanup without any test coverage. ### Does this pull request potentially affect one of the following parts: - **Dependencies (does it add or upgrade a dependency): (yes) => Dashboard comes with PostgreSQL 11 now** - The public API: no - The schema: no - The default values of configurations: no - The wire protocol: no - The rest endpoints: no - The admin cli options: no - **Anything that affects deployment: don't know => Dashboard doesn't look like a big deal for Pulsar but a database version can affects deployment** ### Documentation - Does this pull request introduce a new feature? no This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] cckellogg commented on a change in pull request #124: [Issue #123] Ensure message is sent if no room in current batch.
cckellogg commented on a change in pull request #124: [Issue #123] Ensure
message is sent if no room in current batch.
URL: https://github.com/apache/pulsar-client-go/pull/124#discussion_r358337580
##
File path: pulsar/producer_partition.go
##
@@ -256,10 +256,17 @@ func (p *partitionProducer) internalSend(request
*sendRequest) {
}
if sendAsBatch {
- ok := p.batchBuilder.Add(smm, sequenceID, msg.Payload, request,
msg.ReplicationClusters)
- if ok == false {
+ added := p.batchBuilder.Add(smm, sequenceID, msg.Payload,
request, msg.ReplicationClusters)
+ if !added {
// The current batch is full.. flush it and retry
p.internalFlushCurrentBatch()
+
+ // after flushing try again to add the current payload
+ if ok := p.batchBuilder.Add(smm, sequenceID,
msg.Payload, request, msg.ReplicationClusters); !ok {
+ p.log.WithField("size", len(msg.Payload)).
+ WithField("sequenceID", sequenceID).
+ Error("unable to send message")
+ }
}
} else {
// Send individually
Review comment:
Sounds good. I'll add a log there.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jeloba opened a new issue #5868: JDBC Sink Leaks DB Credentials
jeloba opened a new issue #5868: JDBC Sink Leaks DB Credentials URL: https://github.com/apache/pulsar/issues/5868 **Describe the bug** The JDBC sink leaks db credentials (the in contents of 'configs') into logs and API results. **To Reproduce** Steps to reproduce the behavior: 0. Start Pulsar standalone 1. Register a JDBC sink 2. Observe the log output on a standalone pulsar instance 3. It contains the credentials in a line starting with 'ProcessBuilder starting the process with' in a JSON object called '--function_details' 4. Fetch the sink details using 'bin/pulsar-admin sinks get' 5. Observe that the result also contains the credentials **Expected behavior** Secrets should not be logged or returned by the API. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] roman-popenov opened a new issue #5869: [terraform] Error while using terraform init command in pulsar/deployment/terraform-ansible/aws
roman-popenov opened a new issue #5869: [terraform] Error while using terraform init command in pulsar/deployment/terraform-ansible/aws URL: https://github.com/apache/pulsar/issues/5869 I am trying to run the terraform `init command` to set-up my AWS cluster so I can deploy a Pulsar cluster, and I am getting the following error: ``` >terraform init Initializing the backend... Initializing provider plugins... - Checking for available provider plugins... - Downloading plugin for provider "random" (hashicorp/random) 2.2.1... Provider "registry.terraform.io/-/aws" v1.5.0 is not compatible with Terraform 0.12.18. Provider version 2.7.0 is the earliest compatible version. Select it with the following version constraint: version = "~> 2.7" Terraform checked all of the plugin versions matching the given constraint: 1.5 Consult the documentation for this provider for more information on compatibility between provider and Terraform versions. Error: incompatible provider version ``` I have installed the terraform on my Mac using `brew install terraform` command and the output of `terraform -version` is `Terraform v0.12.18` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lspgn edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer
lspgn edited a comment on issue #5695: Go: Segfault on messageId serialize
after getting it from producer
URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566095147
@wolfstudy thanks a lot.
This code is also giving me the segfault.
I realize the only variable in our setups is I'm using a Pulsar
docker-container.
```
version: '3'
services:
pulsar:
image: 'apachepulsar/pulsar:2.4.2'
ports:
- '6650:6650'
- '8080:8080'
command: bin/pulsar standalone
```
(either 2.4.1 or 2.4.2, there's no 2.5.0 yet on Docker Hub).
```
$ docker inspect apachepulsar/pulsar:2.4.2
[
{
"Id":
"sha256:08287f8732ada886938e1cad6a635f74ec3d55e2386488839928ec95a1c3baea
```
My hunch is: on Mac OS, Docker is in a VM, for heavy tasks, they are usually
slowed down. Would it be possible that the broker does not reply with the ID
fast enough to the client?
Will try with non Docker pulsar and let you know.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lspgn commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer
lspgn commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566190061 Just tried in a Debian VM with 4GB RAM with openjdk-11-jre, 2.4.2, still segfault :/ . But I also installed the deb dev library and ran the go program from the Linux: it is working without issues. So it's likely the libpulsar lib on Mac OS. :( Sorry I thought it would be the Go API. Is there something I can do to troubleshoot more the libpulsar brew package? ``` $ brew info libpulsar libpulsar: stable 2.4.1 (bottled) Apache Pulsar C++ library https://pulsar.apache.org/ /usr/local/Cellar/libpulsar/2.4.1_1 (50 files, 14.0MB) * Poured from bottle on 2019-11-16 at 19:13:31 From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/libpulsar.rb ==> Dependencies Build: cmake ✔, pkg-config ✔ Required: boost ✔, openssl@1.1 ✔, protobuf ✘, zstd ✘ ==> Analytics install: 126 (30 days), 404 (90 days), 1,246 (365 days) install-on-request: 126 (30 days), 404 (90 days), 1,247 (365 days) build-error: 0 (30 days) ``` ``` $ brew list libpulsar /usr/local/Cellar/libpulsar/2.4.1_1/include/pulsar/ (42 files) /usr/local/Cellar/libpulsar/2.4.1_1/lib/libpulsar.2.4.1.dylib /usr/local/Cellar/libpulsar/2.4.1_1/lib/ (2 other files) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] RAbraham commented on issue #4540: Pulsar Client has no support in Mac 10.14.5 (18F132) ?
RAbraham commented on issue #4540: Pulsar Client has no support in Mac 10.14.5 (18F132) ? URL: https://github.com/apache/pulsar/issues/4540#issuecomment-566196954 > @nathan-zhu try this > > ``` > pip3 install pulsar-client==2.4.1.post1 > ``` works on python 3.7, Mojave This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] julio666 edited a comment on issue #5867: [issue 5847][dashboard] postgresql correct version (9.6 to 11)
julio666 edited a comment on issue #5867: [issue 5847][dashboard] postgresql correct version (9.6 to 11) URL: https://github.com/apache/pulsar/pull/5867#issuecomment-566220423 Hm, it looks like really too much build errors for a that small modification no ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] julio666 commented on issue #5867: [issue 5847][dashboard] postgresql correct version (9.6 to 11)
julio666 commented on issue #5867: [issue 5847][dashboard] postgresql correct version (9.6 to 11) URL: https://github.com/apache/pulsar/pull/5867#issuecomment-566220423 Hm, it looks like really too much errors for a that small modification no ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] phreed opened a new pull request #5870: Added catch for NoClassDefFoundError wherever there was a ClassNotFou…
phreed opened a new pull request #5870: Added catch for NoClassDefFoundError wherever there was a ClassNotFou… URL: https://github.com/apache/pulsar/pull/5870 Fixes #5726 ### Motivation When running pulsar-io connectors and functions from the Intellij IDE some actions fail due to uncaught class-not-found throwables. The expectation being that the class is being dynamically loaded and only the ClassNotFoundException will occur if the class is not found. When the function is created or run with https://pulsar.apache.org/docs/en/functions-deploying/#local-run-mode this is indeed the case. When running under the control https://pulsar.apache.org/docs/en/functions-debug/#debug-with-localrun-mode as a gradle plugin the class may already be known and throw a NoClassDefFoundError. It seems to me that any time ClassNotFoundException is handled then NoClassDefFoundError should also be caught. ### Modifications Wherever there was a `catch (ClassNotFoundException ` I replaced it with `catch (ClassNotFoundException | NoClassDefFoundError ` . There were multiple cases where the ClassNotFoundException were handled e.g. the jar loader failed so the nar loader was used to handle the jar loader's failure. ### Verifying this change - [ ] Make sure that the change passes the CI checks. This change is a trivial rework / code cleanup without any test coverage. ### Documentation - Does this pull request introduce a new feature? (no) - This is a fix that provides capabilities claimed in the current documentation. https://pulsar.apache.org/docs/en/functions-debug/#debug-with-localrun-mode This fix is needed to allow gradle plugins based on localrun (and its derivatives) to be constructed. ### Notes When testing this change the number of totalMessages in pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java had to be increased from 10 to 17 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] phreed closed pull request #5848: [Issue 5726] Added catch for NoClassDefFoundError wherever there was a ClassNotFoundException
phreed closed pull request #5848: [Issue 5726] Added catch for NoClassDefFoundError wherever there was a ClassNotFoundException URL: https://github.com/apache/pulsar/pull/5848 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] phreed commented on issue #5848: [Issue 5726] Added catch for NoClassDefFoundError wherever there was a ClassNotFoundException
phreed commented on issue #5848: [Issue 5726] Added catch for NoClassDefFoundError wherever there was a ClassNotFoundException URL: https://github.com/apache/pulsar/pull/5848#issuecomment-566224951 @sijie I made a new pull request from master as you requested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] phreed opened a new pull request #5871: [Issue 5725][localrun] Set connector's directory from system properties as well as environment parameter.
phreed opened a new pull request #5871: [Issue 5725][localrun] Set connector's
directory from system properties as well as environment parameter.
URL: https://github.com/apache/pulsar/pull/5871
#5725
### Motivation
I am developing a gradle plugin for pulsar.
The main task for the pulsar-gradle-plugin is to run sink-connectors,
source-connectors, functions as localrun processes.
As outlined in
https://pulsar.apache.org/docs/en/functions-debug/#debug-with-localrun-mode
This makes debugging of these components from withing IDE (idea) simpler.
### Modifications
System.getenv("PULSAR_HOME")
cannot be changed programmatically from java and consequently from the
gradle plugin (written in java).
This can be corrected by first checking for the "pulsar.home.path" system
property.
System.getProperty("pulsar.home.path").
To allow more flexibility in the placement of the connectors the use of
System.getProperty("pulsar.connector.path") is also allowed.
### Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
### Does this pull request potentially affect one of the following parts:
- The public API: is extended by this change. It is possible to set
system properties which change the behavior.
### Documentation
This will change the documentation for
https://pulsar.apache.org/docs/en/functions-debug/#debug-with-localrun-mode
Currently the documentation is missing a mention of the PULSAR_HOME
environment variable.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] addisonj opened a new pull request #5872: [broker] Allow for namespace default of offload threshold
addisonj opened a new pull request #5872: [broker] Allow for namespace default of offload threshold URL: https://github.com/apache/pulsar/pull/5872 ### Motivation Most namespace level configurations have corresponding cluster configuration that set a namespace default. The offload threshold does not, which makes it more difficult to ensure that namespaces have the cluster wide namespace defaults. ### Modifications This commit adds a the namespace default and also adds a new API endpoint and CLI action to clear the namespace default, this brings it in line with how the offloadDeletionLag is done and brings more consistency to the API. ### Verifying this change - [ ] Make sure that the change passes the CI checks. This change ideally would have some tests but does not appear like these API methods are under tests ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): no - The public API: no - The schema: no - The default values of configurations: yes, adds a new config option that is the same as previous hardcoded default - The wire protocol: no - The rest endpoints: yes, adds the action to clear the default offload theshold - The admin cli options: yes, adds the command to clear the overridden default offload threshold - Anything that affects deployment: yes ### Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? docs This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page
Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page URL: https://github.com/apache/pulsar/pull/5852#issuecomment-566314433 @codelipenghui thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page
Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page URL: https://github.com/apache/pulsar/pull/5852#issuecomment-566314724 run ci tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet removed a comment on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page
Anonymitaet removed a comment on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page URL: https://github.com/apache/pulsar/pull/5852#issuecomment-566314724 run ci tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page
Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page URL: https://github.com/apache/pulsar/pull/5852#issuecomment-566315547 run ci tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page
Anonymitaet commented on issue #5852: [Doc][Issue 5397] Generate a complete list of connectors on the Pulsar Download Page URL: https://github.com/apache/pulsar/pull/5852#issuecomment-566315492 run continuous tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5866: Flink event time based windowing functions not working with pulsar
jiazhai commented on issue #5866: Flink event time based windowing functions not working with pulsar URL: https://github.com/apache/pulsar/issues/5866#issuecomment-566341740 @yjshen to take a look This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] atlantic2099 opened a new pull request #5873: [issue#5779][docs]Reorganize "tiered storage" content
atlantic2099 opened a new pull request #5873: [issue#5779][docs]Reorganize "tiered storage" content URL: https://github.com/apache/pulsar/pull/5873 ### Motivation To reorganize "tiered storage" content to make it logically smooth. ### Modifications Refine "tiered storage" under concepts and architecture, create new chapter for tiered storage by reorganizing content under cookbooks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5868: JDBC Sink Leaks DB Credentials
jiazhai commented on issue #5868: JDBC Sink Leaks DB Credentials URL: https://github.com/apache/pulsar/issues/5868#issuecomment-566345347 @tuteng Would you please help on this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5869: [terraform] Error while using terraform init command in pulsar/deployment/terraform-ansible/aws
jiazhai commented on issue #5869: [terraform] Error while using terraform init command in pulsar/deployment/terraform-ansible/aws URL: https://github.com/apache/pulsar/issues/5869#issuecomment-566350599 @roman-popenov you need to update the aws provider to minimum 2.7 to use this: https://www.terraform.io/docs/providers/aws/guides/version-2-upgrade.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5873: [Issue#5779][docs]Reorganize "tiered storage" content
Jennifer88huang commented on issue #5873: [Issue#5779][docs]Reorganize "tiered storage" content URL: https://github.com/apache/pulsar/pull/5873#issuecomment-566351224 @atlantic2099 Thank you very much for your contributions. Except those modification, you also need to update the `../site2/website/sidebars.json` file. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5582: pulsar dashboard collector encountered error message
jiazhai commented on issue #5582: pulsar dashboard collector encountered error message URL: https://github.com/apache/pulsar/issues/5582#issuecomment-566353902 @ltamber Would you please help provide detailed steps for reproduce? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5868: JDBC Sink Leaks DB Credentials
tuteng commented on issue #5868: JDBC Sink Leaks DB Credentials URL: https://github.com/apache/pulsar/issues/5868#issuecomment-566359600 Thank you for reporting this problem. The current configuration is printed in function_worker. We may need to plan how to replace this sensitive information. Please let us know if you have any good suggestions. @jeloba This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566360406 OK, thanks @lspgn > But I also installed the deb dev library and ran the go program from the Linux: it is working without issues. So, we can ensure there is no problem with the implementation of this `SendAndGetMsgID `, right? > So it's likely the libpulsar lib on Mac OS. :( I agree with you, maybe we can change the way of build cpp lib: ``` cd pulsar-client-cpp cmake . -DBUILD_TESTS=OFF make -j12 && make install ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer URL: https://github.com/apache/pulsar/issues/5695#issuecomment-566360406 OK, thanks @lspgn > But I also installed the deb dev library and ran the go program from the Linux: it is working without issues. So, we can ensure there is no problem with the implementation of this `SendAndGetMsgID `, right? > So it's likely the libpulsar lib on Mac OS. :( I agree with you, maybe we can change the way of build cpp lib: ``` cd pulsar-client-cpp cmake . -DBUILD_TESTS=OFF make -j12 && make install ``` In 2.4.1 of `libpulsar`, there is no function to `SendAndGetMsgID`, so we can't use it by `brew install libpulsar`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-translation] SylviaBABY opened a new pull request #17: [Update] trans schedule
SylviaBABY opened a new pull request #17: [Update] trans schedule URL: https://github.com/apache/pulsar-translation/pull/17 ## Motivation *Translate docs* ## Modifications *Update some doc status* This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-translation] SylviaBABY commented on issue #17: [Update] trans schedule
SylviaBABY commented on issue #17: [Update] trans schedule URL: https://github.com/apache/pulsar-translation/pull/17#issuecomment-566362494 @sijia-w 你可以直接看我新提的这个 PR 上边填的描述，更改 schedule 名字的我都直接在网页上操作更改的，所以会简单一些，你可以参考一下吼 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] addisonj commented on issue #5872: [broker] Allow for namespace default of offload threshold
addisonj commented on issue #5872: [broker] Allow for namespace default of offload threshold URL: https://github.com/apache/pulsar/pull/5872#issuecomment-566362528 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai edited a comment on issue #5582: pulsar dashboard collector encountered error message
jiazhai edited a comment on issue #5582: pulsar dashboard collector encountered error message URL: https://github.com/apache/pulsar/issues/5582#issuecomment-566353902 @ltamber Would you please help provide detailed steps for reproduce? so @tuteng could help on it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5873: [Issue#5779][docs]Reorganize "tiered storage" content
Jennifer88huang commented on issue #5873: [Issue#5779][docs]Reorganize "tiered storage" content URL: https://github.com/apache/pulsar/pull/5873#issuecomment-566364392 @atlantic2099 Good job. When you add a new section for "Tiered storage", remove the file in line113 in the `sidebar.json` file as well.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang edited a comment on issue #5873: [Issue#5779][docs]Reorganize "tiered storage" content
Jennifer88huang edited a comment on issue #5873: [Issue#5779][docs]Reorganize "tiered storage" content URL: https://github.com/apache/pulsar/pull/5873#issuecomment-566364392 @atlantic2099 Good job. When you add a new section for "Tiered storage", remove the file in line113 in the `sidebars.json` file as well.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5873: [Issue#5779][docs]Reorganize "tiered storage" content
Anonymitaet commented on a change in pull request #5873: [Issue#5779][docs]Reorganize "tiered storage" content URL: https://github.com/apache/pulsar/pull/5873#discussion_r358586108 ## File path: site2/website/sidebars.json ## @@ -120,6 +120,12 @@ "cookbooks-message-queue", "cookbooks-bookkeepermetadata" ], +"Tiered Storage": [ + "tiered-storage-overview", + "tiered-principle-of-offloaders", + "tiered-how-to-manage-offloaders", + "tiered-how-to-use-offloaders" Review comment: A tiny suggestion for `id` in `sidebars.json`: give it a short name for easier maintenance. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy merged pull request #124: [Issue #123] Ensure message is sent if no room in current batch.
wolfstudy merged pull request #124: [Issue #123] Ensure message is sent if no room in current batch. URL: https://github.com/apache/pulsar-client-go/pull/124 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar-client-go] branch master updated: [Issue #123] Ensure message is sent if no room in current batch. (#124)
This is an automated email from the ASF dual-hosted git repository.
rxl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git
The following commit(s) were added to refs/heads/master by this push:
new 85bbf45 [Issue #123] Ensure message is sent if no room in current
batch. (#124)
85bbf45 is described below
commit 85bbf458b960be81d236bd58d424f189d8934d71
Author: cckellogg
AuthorDate: Mon Dec 16 20:35:53 2019 -0800
[Issue #123] Ensure message is sent if no room in current batch. (#124)
* [Issue #123] Ensure message is sent if no room in current batch.
* Add issue to test comment.
* Increase test wait timeout.
* Add timeout log for test.
* Add log when a single message fails to be added to batch.
---
pulsar/consumer_test.go | 3 +-
pulsar/negative_acks_tracker_test.go | 8 ++---
pulsar/producer_partition.go | 19 +--
pulsar/producer_test.go | 63
4 files changed, 85 insertions(+), 8 deletions(-)
diff --git a/pulsar/consumer_test.go b/pulsar/consumer_test.go
index 5ff734c..c2a2073 100644
--- a/pulsar/consumer_test.go
+++ b/pulsar/consumer_test.go
@@ -787,7 +787,8 @@ func TestConsumerMetadata(t *testing.T) {
t.Fatal(err)
}
subs := stats["subscriptions"].(map[string]interface{})
- meta :=
subs["my-sub"].(map[string]interface{})["consumers"].([]interface{})[0].(map[string]interface{})["metadata"].(map[string]interface{})
+ cons :=
subs["my-sub"].(map[string]interface{})["consumers"].([]interface{})[0].(map[string]interface{})
+ meta := cons["metadata"].(map[string]interface{})
assert.Equal(t, len(props), len(meta))
for k, v := range props {
mv := meta[k].(string)
diff --git a/pulsar/negative_acks_tracker_test.go
b/pulsar/negative_acks_tracker_test.go
index 4dac66e..93c1af2 100644
--- a/pulsar/negative_acks_tracker_test.go
+++ b/pulsar/negative_acks_tracker_test.go
@@ -29,9 +29,9 @@ import (
const testNackDelay = 300 * time.Millisecond
type nackMockedConsumer struct {
- ch chan messageID
+ ch chan messageID
closed bool
- lock sync.Mutex
+ lock sync.Mutex
msgIds []messageID
}
@@ -42,7 +42,7 @@ func newNackMockedConsumer() *nackMockedConsumer {
go func() {
// since the client ticks at an interval of delay / 3
// wait another interval to ensure we get all messages
- time.Sleep(testNackDelay + 101 * time.Millisecond)
+ time.Sleep(testNackDelay + 101*time.Millisecond)
t.lock.Lock()
defer t.lock.Unlock()
t.closed = true
@@ -69,7 +69,7 @@ func sortMessageIds(msgIds []messageID) []messageID {
return msgIds
}
-func (nmc *nackMockedConsumer) Wait() <- chan messageID {
+func (nmc *nackMockedConsumer) Wait() <-chan messageID {
return nmc.ch
}
diff --git a/pulsar/producer_partition.go b/pulsar/producer_partition.go
index c5345cb..b3b8937 100644
--- a/pulsar/producer_partition.go
+++ b/pulsar/producer_partition.go
@@ -256,14 +256,27 @@ func (p *partitionProducer) internalSend(request
*sendRequest) {
}
if sendAsBatch {
- ok := p.batchBuilder.Add(smm, sequenceID, msg.Payload, request,
msg.ReplicationClusters)
- if ok == false {
+ added := p.batchBuilder.Add(smm, sequenceID, msg.Payload,
request, msg.ReplicationClusters)
+ if !added {
// The current batch is full.. flush it and retry
p.internalFlushCurrentBatch()
+
+ // after flushing try again to add the current payload
+ if ok := p.batchBuilder.Add(smm, sequenceID,
msg.Payload, request, msg.ReplicationClusters); !ok {
+ p.log.WithField("size", len(msg.Payload)).
+ WithField("sequenceID", sequenceID).
+ WithField("properties", msg.Properties).
+ Error("unable to add message to batch")
+ }
}
} else {
// Send individually
- p.batchBuilder.Add(smm, sequenceID, msg.Payload, request,
msg.ReplicationClusters)
+ if added := p.batchBuilder.Add(smm, sequenceID, msg.Payload,
request, msg.ReplicationClusters); !added {
+ p.log.WithField("size", len(msg.Payload)).
+ WithField("sequenceID", sequenceID).
+ WithField("properties", msg.Properties).
+ Error("unable to send single message")
+ }
p.internalFlushCurrentBatch()
}
diff --git a/pulsar/producer_test.go b/pulsar/producer_test.go
i
[GitHub] [pulsar-client-go] wolfstudy closed issue #76: Transfer data is blocked by flush
wolfstudy closed issue #76: Transfer data is blocked by flush URL: https://github.com/apache/pulsar-client-go/issues/76 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy closed issue #123: The client batch is full, resulting in data loss or client is blocked
wolfstudy closed issue #123: The client batch is full, resulting in data loss or client is blocked URL: https://github.com/apache/pulsar-client-go/issues/123 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] wolfstudy closed issue #114: The pulsar go client missing data when transfer
wolfstudy closed issue #114: The pulsar go client missing data when transfer URL: https://github.com/apache/pulsar-client-go/issues/114 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se opened a new pull request #5874: Downgrade maven for license check
aahmed-se opened a new pull request #5874: Downgrade maven for license check URL: https://github.com/apache/pulsar/pull/5874 The build fails with license check with maven 3.6.2 which is part of ubuntu-latest in gitlab actions , here we downgrade it get it to pass. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy opened a new pull request #5875: Update missing import libs in function-develop.md
wolfstudy opened a new pull request #5875: Update missing import libs in function-develop.md URL: https://github.com/apache/pulsar/pull/5875 Signed-off-by: xiaolong.ran ### Modifications - Add import libs for `WordCountFunction` - Add import libs for `JavaNativeExclamationFunction ` - Add import libs for `ExclamationFunction ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5875: Update missing import libs in function-develop.md
wolfstudy commented on issue #5875: Update missing import libs in function-develop.md URL: https://github.com/apache/pulsar/pull/5875#issuecomment-566388342 @Jennifer88huang PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5874: Downgrade maven for license check
jiazhai commented on issue #5874: Downgrade maven for license check URL: https://github.com/apache/pulsar/pull/5874#issuecomment-566395542 Let's wait for the action running results This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
