2019-12-11 10:21:14 UTC - hamu: @hamu has joined the channel ---- 2019-12-11 14:27:04 UTC - Daniel Ferreira Jorge: Hey guys... how can I make a REST API call to a cluster that requires token authentication? I tried sending an `AUTHORIZATION` header with the token, but it did not work... ---- 2019-12-11 14:33:49 UTC - Yong Zhang: You need to add header named `Authorization` and the value like this `Bearer tokenString` ---- 2019-12-11 14:35:22 UTC - Yong Zhang: There has a space between the `Bearer` and you token. ---- 2019-12-11 14:42:15 UTC - Daniel Ferreira Jorge: Thanks for the answer! That was EXACTLY what I did... ---- 2019-12-11 14:44:54 UTC - Yong Zhang: Do you have any error messages? ---- 2019-12-11 14:46:23 UTC - Daniel Ferreira Jorge: `Authentication required for url` ---- 2019-12-11 14:53:40 UTC - Yong Zhang: How did you send the request? Using code or curl? ---- 2019-12-11 15:06:04 UTC - Daniel Ferreira Jorge: Python code, the requests lib... ---- 2019-12-11 15:06:37 UTC - Daniel Ferreira Jorge: If I disable the authentication in the cluster, everything works... ---- 2019-12-11 15:08:46 UTC - Daniel Ferreira Jorge: Also, authentication in the cluster is working... I setup an access node and configured the token in the client.conf file, and I was able to use the pulsar-admin CLI ---- 2019-12-11 15:21:09 UTC - Yong Zhang: That seems something wrong in the python code… ---- 2019-12-11 15:32:52 UTC - Julius.b: @Julius.b has joined the channel ---- 2019-12-11 15:53:09 UTC - Brian Doran: @Sijie Guo Do you have any thoughts on this? I can;t quite get to the bottom of why we are bottlenecked on the client ---- 2019-12-11 16:00:21 UTC - Sijie Guo: sorry I missed your messages yesterday.
where do you put your journal dirs and ledgers dirs on? Are you creating a raid on those 16 x 1.2 TB disks and put journal & ledger there? ---- 2019-12-11 16:19:10 UTC - Brian Doran: trying to get that info for you. Is there a recommended configuration for high throughput? ---- 2019-12-11 16:29:07 UTC - Daniel Ferreira Jorge: Thank you for your help... The problem was that a moron (me) configured something wrong. ---- 2019-12-11 18:46:19 UTC - Sijie Guo: Pulsar (bookkeeper) turns fsync on by default. hence if you put journal and ledger together on same disk, there might be contentions. a couple of options you can consider: 1. disable fsync if your disk is HDD not SSD. 2. if you have 16 x 1.2 TB disks and they are not raided, you can consider using 1~2 disks as the journal disks and the remaining disks as the ledger disks. 3. If these 16 disks are raided, you can try create 8 journal dirs and 8 ledger dirs, so bookies can have mutiple journal threads on flushing out the data. if you can give me more information about your disks, I can provide you better suggestions. ---- 2019-12-11 19:52:15 UTC - Roman Popenov: Hello everyone! So just to make sure I got it right: a segment is not a message. A segment is a block of messages of size X? heavy_check_mark : Sijie Guo ---- 2019-12-11 20:14:44 UTC - Brendan Price: @Brendan Price has joined the channel ---- 2019-12-11 20:16:45 UTC - Brendan Price: hello! i just wanted to check in with pulsar community here to see if anyone else was having problems installing the pulsar client for python via pip? ---- 2019-12-11 20:39:28 UTC - Jared Mackey: Does the standalone server support token auth? (for local dev, testing out auth) ---- 2019-12-11 20:40:36 UTC - Jerry Peng: Yes ---- 2019-12-11 20:40:53 UTC - Sijie Guo: correct thanks : Roman Popenov ---- 2019-12-11 20:40:59 UTC - Jerry Peng: The standalone should support most of the features of a actual deployment ---- 2019-12-11 20:42:04 UTC - Sijie Guo: What kind of issues have you seen? ---- 2019-12-11 20:49:07 UTC - Brendan Price: hey @Sijie Guo thanks for the response. i keep getting this error: ```$ pip3 install pulsar-client== .ERROR: Could not find a version that satisfies the requirement pulsar-client== (from versions: none)``` (note that i omitted the version in the install just to see what versions were available) its as if pypi has no tracked versions for the pulsar client at all. i tried downgrading my version of pip to 19.2.3 but still get the same error. ---- 2019-12-11 20:50:16 UTC - Brendan Price: also im on a mac if that helps provide useful context. ---- 2019-12-11 20:58:52 UTC - Sijie Guo: what is your mac version? ---- 2019-12-11 21:03:39 UTC - Brendan Price: mojave 10.14.6 ---- 2019-12-11 21:20:25 UTC - Sanjeev Kulkarni: What’s your python version ---- 2019-12-11 21:21:40 UTC - Brendan Price: i tried to install using the following versions: ```3.5.8 3.6.4 3.8.0``` (im using pyenv to maintain python versions) i setup a venv for each version and ran into the issue across all 3 versions. ---- 2019-12-11 21:29:29 UTC - Sanjeev Kulkarni: These should all be supported ---- 2019-12-11 21:29:50 UTC - Brendan Price: haha, well i had thought so as well ---- 2019-12-11 21:30:37 UTC - Brendan Price: i'll try and get on another mac real quick and see if i run into this issue. i had originally just wanted to see if anyone else was having any problems at all. i presume you are not? ---- 2019-12-11 21:47:53 UTC - Sanjeev Kulkarni: I just verfied on my mac(python 3.,7.5, mojave) and it worked fine ---- 2019-12-11 21:48:41 UTC - Brendan Price: what version of pip are you running? ---- 2019-12-11 21:48:42 UTC - Sanjeev Kulkarni: Could you see if you can work with 3.7 ---- 2019-12-11 21:49:04 UTC - Sanjeev Kulkarni: pip3 --version pip 19.3.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7) ---- 2019-12-11 21:49:19 UTC - Brendan Price: sure ill setup 3.7 ---- 2019-12-11 22:25:47 UTC - Roman Popenov: Just another quick question: is Ceph offloader officially integrated/supported? ---- 2019-12-11 22:27:10 UTC - Brendan Price: ok interestingly installing the client for 3.7.5 (on the same mac that im having problems) worked. but still does not work for 3.8.0 ---- 2019-12-11 22:27:54 UTC - Alexandre DUVAL: officially don't know but the s3 sdk is compatible so you can use it +1 : Ryan ---- 2019-12-11 22:28:12 UTC - Brendan Price: version 19.2.3 for pip for both python versions ---- 2019-12-11 22:28:45 UTC - Roman Popenov: I saw the master ticket for hdfs/cefs was closed <https://github.com/apache/pulsar/issues/3216> +1 : Ryan ---- 2019-12-11 22:38:59 UTC - Brendan Price: ok just confirmed this from a different mac; the client installs fine for 3.7.5 (also tried 3.6.9) but does not install correctly for 3.8.0 ---- 2019-12-11 22:51:48 UTC - Brendan Price: @Sanjeev Kulkarni can you confirm the version of pulsar that is being installed? i just noticed that the highest tracked version of the client is 2.0.1, which was released about a year and a half ago... ---- 2019-12-11 22:54:24 UTC - Brendan Price: nvm about the above, that was python version 3.6. 3.7 reflects client 2.4.2 ---- 2019-12-11 22:58:20 UTC - Brendan Price: ok so it seems that the problem is strictly limited to python version 3.8. im not sure why python versions 3.5.8 and 3.6.4 were having issues, but upon testing them on another machine they are fine. but 3.8.0 still does not return any tracked version for the cleint in pypi ---- 2019-12-11 23:39:18 UTC - Sanjeev Kulkarni: Looks like we dont support 3.8 yet. ---- 2019-12-11 23:39:31 UTC - Sanjeev Kulkarni: Could you please add a github issue and assign it to me? ---- 2019-12-12 01:21:57 UTC - Jianfeng Qiao: I have a cluster running pulsar 2.3.1, I'm trying to split the bundles of a namespace. After executing the command like 'pulsar-admin namespaces split-bundle --bundle 0xe0000000_0xf0000000 public/test', I check the view of bundles from zk, I notice that there are three bundles there, 0xe0000000_0xe8000000, 0xe8000000_0xf0000000 and 0xe0000000_0xf0000000. Is this correct? ----
