2019-02-04 18:48:17 UTC - Emma Pollum: I'm seeing strange behavior with prometheus metrics. If I let a topic go idle for a length of time (has a subscription with backlog, but no consumer or producer) the metrics about that topic stop showing up on prometheus' ---- 2019-02-04 18:48:24 UTC - Emma Pollum: Is there a setting where I can tell it to display all info all the time? ---- 2019-02-04 19:24:23 UTC - Ambud Sharma: Is there a way to specify partition id in Message like Kafka? ---- 2019-02-04 19:29:23 UTC - Matteo Merli: That shouldn’t be the case.
If the topic exists, its metrics should be being reported through Prometheus even if there’s no activity. The only reason for not showing up there could be that broker was restarted, and no topic in the same namespace is being used at all as well. That will prevent the topic from being re-assigned to a new broker ---- 2019-02-04 19:30:07 UTC - Ambud Sharma: @Emma Pollum there is also automated topic deletion for idle topics that could cause this ---- 2019-02-04 19:30:20 UTC - Matteo Merli: Not exactly in the same way, though you can specify a custom `MessageRouter` <https://pulsar.apache.org/api/client/org/apache/pulsar/client/api/ProducerBuilder.html#messageRouter-org.apache.pulsar.client.api.MessageRouter-> ---- 2019-02-04 19:31:02 UTC - Matteo Merli: Correct, though topic would only be deleted if no subscriptions are created (and retention time expired too) ---- 2019-02-04 19:31:24 UTC - Matteo Merli: sorry: “no subscriptions are present” ---- 2019-02-04 19:31:53 UTC - Ambud Sharma: @Matteo Merli confused between hashingScheme and messagerouter ---- 2019-02-04 19:32:50 UTC - Matteo Merli: Default message router is hashing message key to find out partition, uses hashing scheme for selecting hashing function ---- 2019-02-04 19:33:12 UTC - Matteo Merli: message router allows custom logic, similar to Kafka `Partitioner` class ---- 2019-02-04 19:47:45 UTC - Emma Pollum: @Matteo Merli I just unloaded and moved around a bunch of topics, i bet thats exactly it! ---- 2019-02-04 19:51:43 UTC - Matteo Merli: Yes, so assignments are not triggered if there’s no activity on any topic within a namespace ---- 2019-02-04 19:57:24 UTC - Ambud Sharma: thanks @Matteo Merli ---- 2019-02-04 20:11:54 UTC - Chris DiGiovanni: How important is it that the journal disks and ledger disks for the bookies be on ssds? What would be the consequence if they were pointing to a glusterfs volume? ---- 2019-02-04 20:48:07 UTC - Ambud Sharma: are we missing javadoc.jars for Pulsar from Maven central? <https://repo1.maven.org/maven2/org/apache/pulsar/pulsar-client/2.2.1/> the sources jar seems to be empty as well ---- 2019-02-04 21:07:00 UTC - Matteo Merli: This is due to the Maven shading plugin stuff. We have fixed it in 2.3, by having an additional `pulsar-client-api` which is pulled in automatically, and is not shaded, so it will carry sources and Javadoc ---- 2019-02-04 21:20:40 UTC - Matteo Merli: It all boils down to performance requirements. Keep in mind that BK already replicates data, so you don’t strictly need a distributed filesystem to run on. The only thing to be careful, using a distributed filesystem is that Bookkeeper by default will fsync the data. Since it’s not a local disk, the write latency might be higher. If that’s a problem, you can also play with `journalSyncData=false` in `bookkeeper.conf`. ---- 2019-02-04 21:24:42 UTC - Alex Grant: @Alex Grant has joined the channel ---- 2019-02-04 22:38:24 UTC - Chris DiGiovanni: Thanks, I'll give that a try! ---- 2019-02-05 05:12:40 UTC - Ambud Sharma: thanks for the info ----
