2020-06-22 09:42:13 UTC - jujugrrr: Seems to work @Addison Higham, I had to restart all brokers though in my test though, after a full restore. But it's quite good. I means technically I could lose all bookies and recover from S3 :thumbsup: ---- 2020-06-22 09:47:51 UTC - jujugrrr: @Marcio Martins Good morning, have you tried 2.6.0 with s3 ? and OIDC? ---- 2020-06-22 09:48:47 UTC - Marcio Martins: Hey, no - I have moved on to other things - got 2.5.1 working stable. Will try 2.6.0 in a couple of weeks. ---- 2020-06-22 09:54:45 UTC - jujugrrr: oky, I'll give it a go ---- 2020-06-22 09:57:32 UTC - dionjansen: Hi I’m trying to upgrade my v2.5.0 Pulsar cluster on Kubernetes to v2.6.0 but my bookkeeper cluster initialise job is failing with the following error:
```kubectl logs -f job.batch/pulsar-bookie-init [conf/bookkeeper.conf] Applying config autoRecoveryDaemonEnabled = false [conf/bookkeeper.conf] Applying config dbStorage_readAheadCacheMaxSizeMb = 32 [conf/bookkeeper.conf] Applying config dbStorage_rocksDB_blockCacheSize = 8388608 [conf/bookkeeper.conf] Applying config dbStorage_rocksDB_writeBufferSizeMB = 8 [conf/bookkeeper.conf] Applying config dbStorage_writeCacheMaxSizeMb = 32 [conf/bookkeeper.conf] Applying config httpServerEnabled = true [conf/bookkeeper.conf] Applying config httpServerPort = 8000 [conf/bookkeeper.conf] Applying config journalMaxBackups = 0 [conf/bookkeeper.conf] Applying config ledgerDirectories = /pulsar/data/bookkeeper/ledgers [conf/bookkeeper.conf] Applying config statsProviderClass = org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider [conf/bookkeeper.conf] Applying config useHostNameAsBookieID = true [conf/bookkeeper.conf] Applying config zkLedgersRootPath = /ledgers [conf/bookkeeper.conf] Applying config zkServers = pulsar-zookeeper:2181 [conf/bookkeeper.conf] Adding config journalDirectories = /pulsar/data/bookkeeper/journal JMX enabled by default Error: Could not find or load main class " JMX enabled by default Error: Could not find or load main class "``` Does anybody have any idea what could cause this? ---- 2020-06-22 10:06:21 UTC - dionjansen: Not sure if this issue is related: <https://github.com/apache/pulsar/issues/6355> seems like it’s related to JVM env vars ---- 2020-06-22 10:17:00 UTC - Madhavan Narayanan: @Madhavan Narayanan has joined the channel ---- 2020-06-22 10:27:07 UTC - Marcio Martins: Cool, let me know your findings :slightly_smiling_face: ---- 2020-06-22 10:51:26 UTC - Jermaine Oppong: @Jermaine Oppong has joined the channel ---- 2020-06-22 13:46:34 UTC - Rahul Vashishth: which pulsar chart should i use among <https://github.com/apache/pulsar-helm-chart> and <https://github.com/streamnative/charts>, while both chart repo says *official helm chart for apache pulsar* :confused: ---- 2020-06-22 14:11:22 UTC - Vladimir Shchur: Same issue <https://github.com/apache/pulsar/issues/7243> +1 : dionjansen ---- 2020-06-22 15:11:54 UTC - Jeff Schneller: Asking again... does anyone have the windows c++ client binaries? Has anyone been able to compile them? ---- 2020-06-22 15:52:43 UTC - rwaweber: Hey all! Sorry for pestering, just wanna confirm that this is expected behavior ---- 2020-06-22 16:04:49 UTC - Chris Hansen: when this was asked before, the answer was to use the apache one. Both are maintained by the same people. +1 : Rahul Vashishth, Sijie Guo ---- 2020-06-22 16:09:41 UTC - Kevin Barresi: @Kevin Barresi has joined the channel ---- 2020-06-22 16:31:23 UTC - Addison Higham: For the same topic? If yes, that would be out of spec for Prometheus ---- 2020-06-22 16:31:37 UTC - Addison Higham: Possibly a regression ---- 2020-06-22 16:32:14 UTC - Addison Higham: If for different topics then that is expected as it gives consumer per topic ---- 2020-06-22 17:03:02 UTC - Devin G. Bost: Has anyone tried using the Debezium connector to connect to Oracle? ---- 2020-06-22 18:01:36 UTC - rwaweber: Oh sorry — I meant mostly with respect to the metric format itself being in spec for prometheus, regardless of whether its for a topic or not E.g. in this metric ```pulsar_consumers_count{cluster="pulsar_bdx",namespace="ns/global",topic="<persistent://ns/global/queue>"} 1.0 1558518241859``` 1558518241859 isn’t going to be captured by prometheus, so I’m mostly curious as to what it could represent, and whether its supposed to be there at all ---- 2020-06-22 18:25:19 UTC - Rahul Vashishth: @Sijie Guo is it fair to say that streamNative charts are ahead from apache charts. how these two are related in terms of development and release cadence. ---- 2020-06-22 19:46:16 UTC - Joshua Decosta: @Joshua Decosta has joined the channel ---- 2020-06-22 22:50:06 UTC - Addison Higham: @rwaweber, that is part of the prometheus spec, an optional timestamp as milliseconds since epoch after the value ---- 2020-06-22 22:50:31 UTC - Addison Higham: <https://prometheus.io/docs/concepts/data_model/#samples> doesn't provide an example, but does mention it ---- 2020-06-22 22:52:12 UTC - jujugrrr: Seems to work well :thumbsup: ---- 2020-06-22 22:52:18 UTC - Addison Higham: oh this is the better doc that describes it :slightly_smiling_face: ---- 2020-06-22 23:10:57 UTC - Addison Higham: <https://prometheus.io/docs/instrumenting/exposition_formats/> ---- 2020-06-22 23:16:36 UTC - rwaweber: Oh cool! I had no idea, thanks for the links! I’ve got some brushing up to do it seems :smile: Really appreciate it! ---- 2020-06-23 02:37:32 UTC - sjmittal: Even I had got same issues ---- 2020-06-23 02:46:15 UTC - sjmittal: Folks can anyone tell me difference between <https://hub.docker.com/r/apachepulsar/pulsar-all> and <https://hub.docker.com/r/apachepulsar/pulsar> docker images ---- 2020-06-23 06:07:32 UTC - Curtis Cook: > If you are <https://pulsar.apache.org/docs/en/standalone-docker|running Pulsar in Docker> or deploying Pulsar using a docker image (e.g. <https://pulsar.apache.org/docs/en/deploy-kubernetes|K8S> or <https://pulsar.apache.org/docs/en/deploy-dcos|DCOS>), you can use the `apachepulsar/pulsar-all` image instead of the `apachepulsar/pulsar` image. `apachepulsar/pulsar-all` image has already bundled <https://pulsar.apache.org/docs/en/io-overview#working-with-connectors|all builtin connectors>. ---- 2020-06-23 06:07:37 UTC - Curtis Cook: <https://pulsar.apache.org/docs/en/standalone/> ---- 2020-06-23 08:21:35 UTC - Marcio Martins: Thanks! That makes me hopeful :smile: ---- 2020-06-23 08:21:43 UTC - jujugrrr: :wink: ---- 2020-06-23 08:22:12 UTC - Marcio Martins: Was the upgrade smooth or did you try from scratch? ---- 2020-06-23 08:22:55 UTC - jujugrrr: couldn't find the exact permissions required for S3 on the documentation. Do you just give more access than required right now? ---- 2020-06-23 08:23:42 UTC - jujugrrr: I gave : ``` "s3:CompleteMultipartUpload", "s3:CreateMultipartUpload", "s3:DeleteObjects", "s3:GetObject", "s3:PutObject", "s3:UploadPart"``` ---- 2020-06-23 08:24:27 UTC - jujugrrr: on the objects and: ``` "s3:ListBucket", "s3:GetBucketLocation"``` On the bucket. That's all I could see from S3 logs ---- 2020-06-23 08:24:54 UTC - Marcio Martins: Yeah, I have full access at the moment ---- 2020-06-23 08:25:01 UTC - jujugrrr: @Addison Higham Do you know if it's documented somewhere? ---- 2020-06-23 08:25:19 UTC - jujugrrr: > Yeah, I have full access at the moment Oky ---- 2020-06-23 08:25:31 UTC - Marcio Martins: Nop, couldn't find it anywhere either... I was meant to investigate that, but forgot about after the OIDC thing not working... ---- 2020-06-23 08:26:39 UTC - jujugrrr: cool, let me know if you get back and find missing permissions ---- 2020-06-23 08:29:44 UTC - Marcio Martins: sure! thanks ---- 2020-06-23 09:10:15 UTC - Galen Suen: @Galen Suen has joined the channel ----
