2020-07-17 09:32:07 UTC - Ebere Abanonu: Hi it works only if the tenant consumer public key is what you used to encrypt the message. For each new message you have to instantiate a new producer with the consumer's public key (which can be in a data store) of the destination consumer topic. +1 : Yezen ---- 2020-07-17 10:44:02 UTC - Ebere Abanonu: @Sijie Guo, @David Kjerrumgaard "Pulsar Functions, that makes it easy to deploy lightweight computing logic using simple functions written in one of several popular programming languages including Python, Java, .NET, and Go" is Pulsar Function coming to .Net and when? ---- 2020-07-17 10:46:05 UTC - Ali Ahmed: there hasn’t been enough demand to do a .net function runner yet ---- 2020-07-17 10:54:38 UTC - Ebere Abanonu: what about the need before the demand? I do need it. ---- 2020-07-17 10:59:47 UTC - Ali Ahmed: generally we track github requests and mailing list threads to prioritize development effort ---- 2020-07-17 11:10:54 UTC - Ebere Abanonu: is pulsar okay with just java, python and go developers? No interest in winning over .net developers? github requests and mailing list may not be a reliable way to gauge demands. .Net function runner need to get off the ground as fast as possible. ---- 2020-07-17 12:07:38 UTC - Arthur: I installed pulsar with helm without tls. I added an ingress on it but it does not work. Is it possible to expose pulsar using ingress? I created an ingress on pulsar-proxy 6650 and 6651 but I get a "unable to find valid certificate" on client side (even if I provide certificate with tlsTrustCertsFilePath). If I ignore cert validation, I get "TooLongFrameException : Adjusted frame length exceeds 5253120: 1213486164 - discarded" ---- 2020-07-17 12:14:37 UTC - Emmanuel Marchand (eXenSa): Hi there, I was wondering why (Java, `v2.5.2`) `CompletableFutre<Messages<T>>` from `Consumer.batchReceiveAsync()` are not completed exceptionally when closing `Consumer` ? This is the case for `CompletableFuture<Message<T>>` from `Consumer.receiveAsync()`. Is it a bug or else ? Thanks. ---- 2020-07-17 12:15:21 UTC - Ali Ahmed: I am now aware of anyone having built a widely deployed stream processing system written in .net, java is by far the dominant one followed by python, Do you have a specific project for stream processing that has to be written in .net ? ---- 2020-07-17 12:23:57 UTC - Imran: @Imran has joined the channel ---- 2020-07-17 12:44:52 UTC - Joshua Decosta: Did you take a look at the end to end encryption docs? ---- 2020-07-17 12:47:44 UTC - Arushi Sharma: Yes. I have been following the following docs: `<https://pulsar.apache.org/docs/en/security-encryption/|https://pulsar.apache.org/docs/en/security-encryption/>` ---- 2020-07-17 13:04:23 UTC - Aaron: Are there any recommendations for pulsar client configuration settings like connections per broker, io threads and listener threads for improving throughout against a cluster? ---- 2020-07-17 14:14:46 UTC - Matteo Merli: return type: you want to retain the type also on the output topic, so that consumers of that topic can use schema as well
encode: the encoding (object->bytes) should not be done in the annotation, we need to publish the object on the pulsar producer and have that producer configured with the schema (same for consumer) ---- 2020-07-17 15:12:45 UTC - Addison Higham: That's a good question... Do you have so more details with a sample? That would be a really good issue to raise in the repo ---- 2020-07-17 15:23:27 UTC - Addison Higham: @Aaron generally, they are pretty dependent on your usage patterns. If you have lots topics (or topic partitions) you may need to raise these values a lot, that said, the defaults are pretty conservative. Raising io threads and listener threads to your number of CPUs cores is common. Raising your connections is also needed for max performance, it is basically a connection pool, so using something like 4 connections is reasonable. In benchmarking, like pulsar-perf, it gets raise to 100 connections just to make sure it is never blocked waiting for a socket, but that can lead to real issues if you have lots of clients ---- 2020-07-17 15:32:08 UTC - Addison Higham: @Arushi Sharma because `CryptoKeyReader` is an instance of a class, there isn't a way to use that API currently. One idea, you could use `loadConf` and then conditionally look at some other properties of your dynamic config to do the final setting of those few fields. I don't believe `loadConf` complains about unknown properties ---- 2020-07-17 15:35:33 UTC - Arushi Sharma: @Addison Higham CryptoKeyReader does not give an issue even if I pass null value. However, encryptionKey does give an error. I tried passing it using loadConf but it complained about unrecognized parameter for producer configs. ---- 2020-07-17 15:39:39 UTC - Addison Higham: Are you using `encryptionKey` or `encryptionKeys`, the internal class calls the field `encryptionKeys` ---- 2020-07-17 15:43:25 UTC - Arushi Sharma: Oh... you are right. I have used wrong name. I will try with that and see if the issue still exists. Thanks. ---- 2020-07-17 17:42:59 UTC - Yezen: Thanks for confirming @Ebere Abanonu I had to confirm for myself so I ended up creating a producer service and a consumer service and set up pulsar locally on a docker container. Can confirm. I had to instantiate a new producer for every message I wanted encrypted with a different public key. On the consumer side I used the keyName passed into `RawFileKeyReader` to figure out which key I needed. A couple of cool things I observed. The same topic can have many messages encrypted in using different public keys. On the consumer side it will not be able to use the message if it can't decrypt it and once it gets the right private key it will be able to consume the encrypted message. ---- 2020-07-17 17:46:28 UTC - Varghese C: @Shivji Kumar Jha - have you created the PIP for this? ---- 2020-07-17 17:48:27 UTC - Shivji Kumar Jha: @Varghese C not yet, let me dump what i have at end (and stuck with not much time) in a PIP early next week. +1 : Varghese C ---- 2020-07-17 20:57:31 UTC - Vladimir Shchur: I have two, in one pulsar was rejected and this was one of two reasons to reject it, in another one we have to use java since there's no other option although .net would be more preferable ---- 2020-07-17 22:26:52 UTC - Ivo Lopes: @Ivo Lopes has joined the channel ----
