2019-12-26 13:53:28 UTC - Sijie Guo: sorry I missed your message.
which instance are you running these commands?
----
2019-12-26 14:08:53 UTC - Roman Popenov: I have ended up doing the following
function: read a filename off a topic, stream the s3 object contents and do
some data processing, and then write it to an S3 bucket. The code ended up
rather simple and works pretty fast. I still need to see how to make that code
as a sink/source IO module.
----
2019-12-26 14:27:32 UTC - Ruian: May I try to implement the `assign`,
`assignment`, and `partitionsFor` methods ?
----
2019-12-26 14:32:26 UTC - Martin Raymond: @Martin Raymond has joined the channel
----
2019-12-26 14:38:16 UTC - Mathieu Druart: hi @Sijie Guo, there are no active
consumer as far as I can see :
```root@precise-opossum-pulsar-broker-5f69cff595-2khzn:/pulsar#
./bin/pulsar-admin topics stats topic-mathieu
{
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"averageMsgSize" : 0.0,
"storageSize" : 0,
"backlogSize" : 0,
"publishers" : [ ],
"subscriptions" : {
"mathieu-function-sub" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"msgRateRedeliver" : 0.0,
"msgBacklog" : 0,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"unackedMessages" : 0,
"msgRateExpired" : 0.0,
"consumers" : [ ],
"isReplicated" : false
}
},
"replication" : { },
"deduplicationStatus" : "Disabled"
}```
----
2019-12-26 14:45:13 UTC - Mathieu Druart: It seems there is already a parameter
in functions named "cleanupSubscription", with "true" as default value. I found
this former issue that seems to be related :
<https://github.com/apache/pulsar/pull/3299> but does it really work ?
----
2019-12-26 16:21:00 UTC - Roman Popenov: It’s safe to assume that `nar` is an
overloaded term, and it’s not Apache NiFi `nar` that we refer in the connectors?
----
2019-12-26 16:23:03 UTC - Sijie Guo: we are using the same format
----
2019-12-26 16:56:08 UTC - Mathieu Druart: I retried to delete the subscription
and it worked ...
----
2019-12-26 16:57:40 UTC - Mathieu Druart: Maybe there is a delay between the
moment I delete the function and the moment it considers the consumer as
unactive ?
----
2019-12-26 16:57:59 UTC - Mathieu Druart: but thanks for the answer anymay
----
2019-12-26 17:04:21 UTC - Roman Popenov:
<https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/java/org/apache/pulsar/common/nar/NarClassLoader.java>
----
2019-12-26 17:04:33 UTC - Roman Popenov: Looks like NiFi archive can be loaded?
----
2019-12-26 17:05:16 UTC - Roman Popenov: Is there documentation about this?
----
2019-12-26 18:46:49 UTC - Julius S: @Julius S has joined the channel
----
2019-12-26 21:27:10 UTC - Mathieu Druart: Hi ! I'm trying to use a custom class
as input for a function, I use an Avro schema for that. I added a custom SerDe
class inside the jar of the function and referenced it in the inputSpecs of the
function when I created it. When I try to create and start the function I have
this error : `java.lang.RuntimeException: User class must be in class path` as
if the class was not inside the classpath/jar. I found this already corrected
issue : <https://github.com/apache/pulsar/pull/5357> is it possible that the
issue was corrected only for Sinks and not for Sources (using the wrong
classloader) ? Thanks !
----
2019-12-26 22:19:10 UTC - Mathieu Druart: maybe it's better for me to define
AVRO as schemaType with just a reference to my POJO class in the jar rather
than trying to use a custom SerDe ? But I still think that the Sources code has
the same issue that Sinks had.
----
2019-12-27 02:40:04 UTC - koonchen: @koonchen has joined the channel
----
2019-12-27 02:55:24 UTC - Sijie Guo: we are using the same package format. but
the class interface to load is different from a NiFi processor.
----
2019-12-27 02:55:44 UTC - Sijie Guo: so I don’t think you can load a NIFI
processor directly
----
