2020-05-04 10:12:13 UTC - Ruian: Hi, I encounter the 500 HTTP internal server
error when using pulsar-admin on a specific topic after publishing messages
into it for days. How can I debug this?
----
2020-05-04 10:14:14 UTC - VanderChen: In other words, is there any way to
consume a message multi times? For example, sending message M in topic T1, can
all consumers who subscribe to T1 consume this message.
----
2020-05-04 10:15:55 UTC - Ruian: since there are a log of INFO logs printed by
the brokers, is there any keywords to narrow the search range?
----
2020-05-04 10:17:50 UTC - VanderChen: Hi all, a stupid question. In _key-shared
modes_, can the consumers specify which key to consuming?
----
2020-05-04 10:43:29 UTC - Adelina Brask: hi Guys, when I use Netty with TCP I
get : Exception in thread "Thread-2" java.lang.NoClassDefFoundError:
org/apache/pulsar/io/netty/tcp/NettyTCPChannelInitializer . any help ?
----
2020-05-04 11:07:12 UTC - JG: Server sent events yes
----
2020-05-04 11:57:17 UTC - Poul Henriksen: As @Vladimir Shchur said, the
consumers need to subscribe to the topic with different subscription names.
Each subscription will get the message "M".
----
2020-05-04 12:25:26 UTC - Kirill Merkushev: Have a question regarding a
classpath isolation. For example could I put the jackson into function and
should I shade it, or it could be just a simple fat jar?
----
2020-05-04 13:42:40 UTC - Ebere Abanonu: @Sijie Guo how does pulsar transaction
work? The current example is run on the same process - producer and consumer in
the same process. How can transaction be used with remote consumers?
----
2020-05-04 13:54:35 UTC - Ming: Disclosure - beam author here. Pulsar Function
is a serverless within the cluster so it has better latency. However Pulsar
Beam can push events to lambda, gcp function that allows you integrate with
aws/gcp/azure eco-system. They have better monitoring and trouble shooting
features but we are improving how to trouble shoot Pulsar Function too. 2. Beam
use http so there could be better language and OS agnostic since Pulsar
Function has limited number of language options. So basically if you have a
heterogeneous environment and integration requirements with other cloud vendor,
you might want to consider pulsar beam.
----
2020-05-04 13:56:18 UTC - Ming: We have also baked Beam into our pulsar helm
chart, that has benefits to be deployed with in the same Pulsar cluster and can
be optionally/separately deployed.
----
2020-05-04 14:09:43 UTC - Ming: You can deploy pulsar function within the same
cluster usually there is no need to deploy another webhook.
----
2020-05-04 14:28:00 UTC - Penghui Li: You can specify key hash ranges for a
consumer. More details about the key shared policy:
```/**
* Set KeyShared subscription policy for consumer.
*
* <p>By default, KeyShared subscription use auto split hash range to
maintain consumers. If you want to
* set a different KeyShared policy, you can set by following example:
*
* <pre>
* client.newConsumer()
*
.keySharedPolicy(KeySharedPolicy.stickyHashRange().ranges(Range.of(0, 10)))
* .subscribe();
* </pre>
* Details about sticky hash range policy, please see {@link
KeySharedPolicy.KeySharedPolicySticky}.
*
* <p>Or
* <pre>
* client.newConsumer()
* .keySharedPolicy(KeySharedPolicy.autoSplitHashRange())
* .subscribe();
* </pre>
* Details about auto split hash range policy, please see {@link
KeySharedPolicy.KeySharedPolicyAutoSplit}.
*
* @param keySharedPolicy The {@link KeySharedPolicy} want to specify
*/
ConsumerBuilder<T> keySharedPolicy(KeySharedPolicy
keySharedPolicy);```
----
2020-05-04 14:39:55 UTC - Alexander Ursu: Hello, was looking at how to set
automatic offload here
(<https://pulsar.apache.org/docs/en/cookbooks-tiered-storage/#configuring-offload-to-run-automatically>)
I'm wondering what the equivalent is in the Admin REST API, I can't seem to
find what I need under the namespaces section
(<https://pulsar.apache.org/admin-rest-api/?version=2.5.1#tag/namespaces>)
----
2020-05-04 14:46:44 UTC - Ebere Abanonu: that should be `SetOffloadThreshold`
+1 : Alexander Ursu
----
2020-05-04 15:31:36 UTC - VanderChen: Thanks a lot! :grinning:
----
2020-05-04 15:34:36 UTC - Alexander Ursu: Follow up to before. I don't seem to
see documented how values are expected to be passed for PUT/POST requests like
with setOffloadThreshold. Where is the number of bytes specified?
(<https://pulsar.apache.org/admin-rest-api/?version=2.5.1#operation/setOffloadThreshold>)
----
2020-05-04 15:41:29 UTC - Franck Schmidlin: Thanks. I like beam, but i think my
use case will end up being better met by functions. Will play and report
+1 : Ming
----
2020-05-04 15:45:05 UTC - Ebere Abanonu: You need to use a client, java maybe
----
2020-05-04 15:47:05 UTC - Alexander Ursu: There is no way to do this with curl
at all?
----
2020-05-04 15:50:13 UTC - Ebere Abanonu: the answer here will help you:
<https://stackoverflow.com/questions/7172784/how-do-i-post-json-data-with-curl>
you need to pass it as the request body
----
2020-05-04 15:50:54 UTC - Ebere Abanonu: here too
<https://stackoverflow.com/questions/43054195/how-to-post-raw-body-data-with-curl/43056956>
----
2020-05-04 16:38:11 UTC - David Kjerrumgaard: A simple fat jar or NAR file
(containing ALL the dependencies) are the preferred deployment methods. The NAR
bundling provides very good classpath isolation.
----
2020-05-04 17:06:02 UTC - Santhi: @Santhi has joined the channel
----
2020-05-04 17:21:04 UTC - Franck Schmidlin: Are there dotPulsar maintainers or
users about?
----
2020-05-04 17:25:21 UTC - Tim Corbett: <#CUAMT2NJE|dev-dotpulsar> exists, if
that helps you
+1 : Franck Schmidlin, Daniel Blankensteiner
----
2020-05-04 17:29:11 UTC - Hannes: @Hannes has joined the channel
----
2020-05-04 17:44:04 UTC - Franck Schmidlin: Ta, found it. I'm not used to slack
----
2020-05-04 17:58:28 UTC - Kirill Merkushev: when you say all - it means pulsar
function api as well?
----
2020-05-04 17:59:47 UTC - Kirill Merkushev: basically nothing provided except
the jdk?
----
2020-05-04 18:15:07 UTC - Kirill Merkushev: actually checked with compile-only
scope for functions-api - works fine
----
2020-05-04 18:37:27 UTC - David Kjerrumgaard: Yes, the Pulsar related classes
will be provided, so you can use the `provided` scope in the pom.xml
----
2020-05-04 20:51:53 UTC - Sijie Guo: It will be an integer value.
`-1 will revert to using the cluster default. A negative value disables
automatic offloading.`
----
2020-05-04 20:52:24 UTC - Sijie Guo: Do you have an example about the use case
you are looking for?
----
2020-05-04 20:52:49 UTC - Sijie Guo: It seems that there is something related
to shading problem.
----
2020-05-04 20:53:19 UTC - Sijie Guo: Use the topic name as the keywords to
narrow the search range.
----
2020-05-04 20:54:56 UTC - Sijie Guo: It is “pulsar” and “pulsar”
----
2020-05-05 07:12:36 UTC - Ebere Abanonu: Working on it
----
2020-05-05 07:13:40 UTC - Ebere Abanonu: The transaction has to be committed
first by the producer before it can be visible to the consumer?
----
2020-05-05 07:14:14 UTC - Ebere Abanonu: What if the consumer is offline?
----
2020-05-05 07:29:59 UTC - Adelina Brask: @Sijie Guo Kan you please elaborate?
Does it have something to do with
<https://github.com/streamnative/pulsar-flink/issues/19> ?
----
2020-05-05 07:37:37 UTC - Erik Jansen: Hi, we are considering using Pulsar for
a new product offering we are currently designing/architecting. We’ve looked
into the Pulsar docs, did some testing and looked into the source code. We
noticed that we really need some experienced Pulsar developers who truly
understand how Pulsar works. Where can I find resources or people willing to
join us in developing our new product offering?
----
2020-05-05 07:43:07 UTC - Sijie Guo: @Adelina Brask I think they have very
similar problems. related to different versions of netty in the same classpath.
----
2020-05-05 07:43:33 UTC - Sijie Guo: It seems that @Penghui Li already reached
out @yijie to see if he has fixed the flink issue.
----
2020-05-05 08:00:12 UTC - Adelina Brask: ok thanks
----
2020-05-05 08:08:19 UTC - Sijie Guo: Pulsar Slack Channel is a good place for
consulting any questions. Most of us (PMC members and committers) are hanging
out here. If you have any questions, you can find the right channels to post
questions.
----
2020-05-05 08:55:59 UTC - Adelina Brask: hvis jeg skifter til http fra tcp får
jeg:
----
2020-05-05 08:56:08 UTC - Adelina Brask: `08:47:27.725 [nioEventLoopGroup-3-1]
ERROR
<http://org.apache.pulsar.io|org.apache.pulsar.io>.netty.http.NettyHttpServerHandler
- Error when processing incoming data`
`java.lang.NoClassDefFoundError:
org/apache/pulsar/io/netty/http/NettyHttpServerHandler$NettyHttpRecord`
`at
<http://org.apache.pulsar.io|org.apache.pulsar.io>.netty.http.NettyHttpServerHandler.channelRead0(NettyHttpServerHandler.java:88)
~[pulsar-io-netty-2.5.0.nar-unpacked/:?]`
`at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
~[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
[netty-codec-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
[netty-codec-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
[netty-transport-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
[netty-common-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[netty-common-4.1.43.Final.jar:4.1.43.Final]`
`at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[netty-common-4.1.43.Final.jar:4.1.43.Final]`
`at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]`
----
2020-05-05 08:56:28 UTC - Adelina Brask: Tror du det er relateret eller er det
en decoding issue?
----
