2019-03-04 09:34:49 UTC - Darragh: Could anyone give me some insight into the numbers behind the default suggested number of instances when deploying on k8s. (Looking at the deployment on kubernetes guide, the default setup lists 2 bk clusters, 3 zk clusters and 3 broker clusters. ) ---- 2019-03-04 09:34:56 UTC - Darragh: Are 3 brokers required? could I do with less? ---- 2019-03-04 09:36:47 UTC - Ali Ahmed: you can do with less brokers ---- 2019-03-04 09:36:59 UTC - Ali Ahmed: they are stateless systems ---- 2019-03-04 09:40:41 UTC - Darragh: and what about the bookkeepers? ---- 2019-03-04 10:45:46 UTC - Yuvaraj Loganathan: @Darragh For Durability & HA atleast 2 book keepers needed if you replication factor is 2. If you are not worried then bookeeper can work with one but if you loose the single node then you may loose the data. ---- 2019-03-04 10:47:09 UTC - Darragh: @Yuvaraj Loganathan ok thanks, I want to start with a small setup to test some things and was wondering whether there were some required minimas ---- 2019-03-04 12:39:27 UTC - bhagesharora: Hello everyone, I am trying to connect pulsar with cassandra IO connector, following <http://pulsar.apache.org/docs/en/io-quickstart/> URl, currently I am stuck in Submit a Cassandra Sink step, Here I am executing one command bin/pulsar-admin sink create --tenant public --namespace default --name cassandra-test-sink --sink-type cassandra --sink-config-file examples/cassandra-sink.yml --inputs test_cassandra but It's saying "Was passed main parameter 'sink' but no main parameter was defined" what would be the reason ?? ---- 2019-03-04 13:23:44 UTC - Darragh: how many k8s nodes should I ideally have for the base setup ? ---- 2019-03-04 15:34:18 UTC - David Kjerrumgaard: @bhagesharora What are the contents of your `cassandra-sink.yml` file? ---- 2019-03-04 17:03:10 UTC - Marc Le Labourier: Which version of python are pulsar functions running on (can’t manage to find it in the doc)? ---- 2019-03-04 17:14:37 UTC - Matteo Merli: @Byron The assumption is correct. If that’s not true it would a bug ---- 2019-03-04 17:27:19 UTC - Sanjeev Kulkarni: @Marc Le Labourier python functions support both python2 and python3. Whatever is the default version of python command line is what a function running on that machine uses ---- 2019-03-04 17:29:29 UTC - Grant Wu: The Docker images provided by apachepulsar use Python 2.7 ---- 2019-03-04 17:29:53 UTC - Grant Wu: But yeah, it just takes what’s in your PATH ---- 2019-03-04 17:29:56 UTC - Sanjeev Kulkarni: @bhagesharora is your pulsar-admin 2.3.0? ---- 2019-03-04 17:30:24 UTC - Guy Feldman: This is probably closer to the java api anyway. Schema.AVRO accepts a specific record which already has the avro schema ---- 2019-03-04 17:33:25 UTC - Byron: Ok, then it may be a bug. This is with 2.3.0. I will see if I can make a simple reproducible test. ---- 2019-03-04 17:49:24 UTC - Matteo Merli: Thanks that would be helpful ---- 2019-03-04 18:04:16 UTC - bhagesharora: @David Kjerrumgaard configs: roots: "localhost:9042" keyspace: "pulsar_test_keyspace" columnFamily: "pulsar_test_table" keyname: "key" columnName: "col" ---- 2019-03-04 18:04:19 UTC - Anton Goretsky: @Anton Goretsky has joined the channel ---- 2019-03-04 18:08:33 UTC - Anton Goretsky: Hello, can I achieve this scenario with Pulsar? Messages with the same attribute should always be handled by the same worker. Like message groups in Apache ActiveMQ ---- 2019-03-04 18:15:35 UTC - Matteo Merli: @Anton Goretsky You can do that with multiple partitions and using a failover subscription (and setting a key on the messages to ensure messages with same key get routed to the same partition) ---- 2019-03-04 18:25:59 UTC - Anton Goretsky: As I understand from the docs, the subscription is made for the whole topic, so every failover subscription will handle every partition, isn't it? ---- 2019-03-04 18:31:03 UTC - Matteo Merli: In a partitioned topic, you can add multiple consumers to a single subscription.
Each consumer will get assigned some partitions. Each partition will have 1 single active consumer ---- 2019-03-04 18:31:22 UTC - Matteo Merli: (this when using failover subscription type) ---- 2019-03-04 18:42:02 UTC - Anton Goretsky: So, inside one failover subscription multiple consumers can be simultaneously active if topic is partitioned? ---- 2019-03-04 18:42:20 UTC - Matteo Merli: correct, one per partition ---- 2019-03-04 18:42:28 UTC - Anton Goretsky: cool, thanks! ---- 2019-03-04 19:50:01 UTC - Alexandre DUVAL: Hi, is JWT authentication available for administrator? ---- 2019-03-04 19:54:14 UTC - Alexandre DUVAL: I read about subject on token creation is like principal (or role), but is it working for role "super", if my broker has parameter like `superUserRoles=super`? ---- 2019-03-04 19:55:22 UTC - Matteo Merli: Yes, you just have to create a token with subject `super` and use that for admin purposes (or broker-to-broker) ---- 2019-03-04 20:10:48 UTC - Byron: <https://gist.github.com/bruth/305926636fd3f6e8ad40a13e3c0b5406> ---- 2019-03-04 20:11:08 UTC - Byron: Left a comment with a short gif demonstrating the issue (based on the script) ---- 2019-03-04 20:25:40 UTC - Alexandre DUVAL: Cool, it works! Thanks. ---- 2019-03-04 20:59:14 UTC - Alexandre DUVAL: @Matteo Merli I just have this error: `21:52:43.113 [pulsar-client-io-1-1] ERROR org.apache.pulsar.client.impl.ClientCnx - [id: 0x96afa005, L:/192.168.0.14:57952 - R:c1 URL:PORT] Close connection because received internal-server error org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Proxy Client is not authorized to Lookup` I'm not sure it's related. ---- 2019-03-04 21:59:04 UTC - Alexandre DUVAL: I think it's something about proxy roles :smile:. ---- 2019-03-04 22:06:20 UTC - Alexandre DUVAL: Or not :confused:. ---- 2019-03-05 03:45:08 UTC - bossbaby: @Matteo Merli @Sijie Guo Does token authentication have support create token through the RestApi? ---- 2019-03-05 04:35:19 UTC - Matteo Merli: No, you need access to the “secret” key (or the “private” key, when using asymmetric sig) ---- 2019-03-05 04:35:58 UTC - Matteo Merli: the idea is that it gets managed by the administrator +1 : bossbaby ----
