Summary of IRC Meeting in #couchdb-meeting at Wed Jun 3 18:03:50 2015: Attendees: Wohali, michellep, awenkhh, Kxepal, sebastianrothbuc, bigbluehat, jan____, robertkowalski, chewbranca, banjiewen, rnewson
- Preface - CouchDB 2.0 status - Action: Kxepal to investigate into CouchDB benchmarks problem - Open floor - Link: ApacheCon Core Call for Participation: http://events.linuxfoundation.org/events/apachecon-core-europe/program/cfp - Link: ApacheCon BigData Call for Participation: http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp - L10n - Action: Kxepal to check out the L10n possibilities with read the docs service - Link: http://mail-archives.apache.org/mod_mbox/couchdb-l10n/201505.mbox/%3CCADA9kr-RdH8_C0_cF9TLdehCTRiwD3WiXP1T2tfa_iBK5GwYQA%40mail.gmail.com%3E - Open floor IRC log follows: ## Preface ## [Wed Jun 3 18:04:10 2015] <Wohali>: nice to see so many friendly faces here after a long absence [Wed Jun 3 18:04:30 2015] <Kxepal>: Wohali: indeed (: [Wed Jun 3 18:04:36 2015] <Kxepal>: so, let's start from CouchDB 2.0 news ## CouchDB 2.0 status ## [Wed Jun 3 18:05:02 2015] <Kxepal>: ok, so what we have... [Wed Jun 3 18:05:13 2015] <robertkowalski>: 2.0 is 7 times slower? [Wed Jun 3 18:05:25 2015] <Kxepal>: robertkowalski: +/- few times but yes [Wed Jun 3 18:05:34 2015] <chewbranca>: I don't think you can make that statement based off of the test suite [Wed Jun 3 18:05:41 2015] <chewbranca>: or at least you need to qualify it [Wed Jun 3 18:05:52 2015] <chewbranca>: the *test* suite is 7 times slower [Wed Jun 3 18:06:02 2015] <chewbranca>: I know we had to add a number of sleeps in places to make things work [Wed Jun 3 18:06:12 2015] <Kxepal>: chewbranca: it's not CouchDB test suite [Wed Jun 3 18:06:19 2015] <Kxepal>: it's not a question of sleeps [Wed Jun 3 18:06:28 2015] <chewbranca>: that's what you said in #couchdb-dev, maybe I misunderstood [Wed Jun 3 18:06:43 2015] <Kxepal>: it's just about using the same API calls for 1.6.1 and 2.0 [Wed Jun 3 18:07:07 2015] <Kxepal>: anyway, that's the only numbers I have for now, but I agree it's awful to use it as measurement base [Wed Jun 3 18:07:21 2015] <rnewson>: hard to believe. [Wed Jun 3 18:07:22 2015] <Kxepal>: we need proper benchmark test suite results of which everyone could reproduce [Wed Jun 3 18:07:28 2015] <chewbranca>: alright, it would be great to get some proper benchmarks here /cc banjiewen [Wed Jun 3 18:07:30 2015] <robertkowalski>: Kxepal: is it measured on the clustered vs backdoor ports? [Wed Jun 3 18:07:46 2015] <chewbranca>: rnewson: agreed [Wed Jun 3 18:07:46 2015] <banjiewen>: chewbranca: Don't point your fingers at me [Wed Jun 3 18:07:47 2015] <Kxepal>: robertkowalski: cluster. users will not work with backdoor in real. [Wed Jun 3 18:07:50 2015] <banjiewen>: :) [Wed Jun 3 18:08:03 2015] <chewbranca>: banjiewen: I'm just motioning in your direction :p [Wed Jun 3 18:08:51 2015] <Kxepal>: banjiewen: do you working on couchdb benchmarks? [Wed Jun 3 18:09:02 2015] <banjiewen>: Kxepal: No, I'm not. [Wed Jun 3 18:09:22 2015] <Kxepal>: banjiewen: but would like?(; [Wed Jun 3 18:09:26 2015] <banjiewen>: I've been observing the various benchmarking discussions with a curious eye. [Wed Jun 3 18:10:00 2015] <banjiewen>: No, I'm not terribly interested in it atm. chewbranca is trolling... [Wed Jun 3 18:10:08 2015] <Kxepal>: haha, ok [Wed Jun 3 18:10:38 2015] <chewbranca>: I'll keep at him about it ;-) [Wed Jun 3 18:11:19 2015] <chewbranca>: anyways, first step in the performance discussion is get a proper benchmark [Wed Jun 3 18:11:36 2015] <Kxepal>: indeed. measure all the things and then solve the bottlenecks. [Wed Jun 3 18:12:14 2015] <Kxepal>: any ideas what to use for this? [Wed Jun 3 18:12:26 2015] <jan____>: and âmaking the test suite run fasterâ is a non-goal at this point. [Wed Jun 3 18:12:38 2015] <sebastianrothbuc>: good old JMeter? [Wed Jun 3 18:12:48 2015] <rnewson>: let's agree never to say JMeter again. [Wed Jun 3 18:13:10 2015] <Kxepal>: I think about tsung for api and basho_bench for internals [Wed Jun 3 18:13:17 2015] <jan____>: hahaha [Wed Jun 3 18:13:19 2015] <chewbranca>: I've been using wrk2 lately for generating request load lately [Wed Jun 3 18:13:32 2015] <rnewson>: tsung is fiddly, I quite like locust.io for its ease-of-use [Wed Jun 3 18:13:33 2015] <chewbranca>: oops too many "lately" [Wed Jun 3 18:13:56 2015] <chewbranca>: locust is definitely easier to use than wrk2 [Wed Jun 3 18:14:05 2015] <rnewson>: anyway, we can be sure that couchdb 2.0 is not fundamentally slower than 1.x, so at worse we have some regressions relative to what cloudant runs in production [Wed Jun 3 18:14:10 2015] <rnewson>: the lager thing was one part of it [Wed Jun 3 18:14:42 2015] <rnewson>: was the bottleneck in json_nif_compare (iirc) solved yet? [Wed Jun 3 18:14:56 2015] <rnewson>: an optimization in couchdb that ironically made things slower [Wed Jun 3 18:15:08 2015] <Kxepal>: how to know that without the numbers? that's a problem /: [Wed Jun 3 18:15:09 2015] <rnewson>: versus an erlang based solution (that could run on > 1 core) [Wed Jun 3 18:16:03 2015] <chewbranca>: rnewson: it's entirely possible the logging is not a regression, given that twig does the same io_lib formatting on the logging caller side [Wed Jun 3 18:16:49 2015] <chewbranca>: or was there another piece to the Lager problem where it wasn't using async? [Wed Jun 3 18:17:11 2015] <rnewson>: it seems right to me that the formatting time is in the caller's process. [Wed Jun 3 18:18:00 2015] <chewbranca>: oh sure, but robertkowalski noticed a large amount of time in the actual calls to io_lib:format which is a known slow point, I'm just noting that twig does the same thing so it's entirely possible that it's not actually a regression from what we run at Cloudant [Wed Jun 3 18:18:13 2015] <chewbranca>: and rather just a slow thing we need to fix [Wed Jun 3 18:18:33 2015] <rnewson>: sure. [Wed Jun 3 18:18:41 2015] <rnewson>: obviously something is going to be slow. [Wed Jun 3 18:19:58 2015] <jan____>: okay, anything else on this? Any action items? [Wed Jun 3 18:20:38 2015] <Kxepal>: anyone willing to investigate into benchmarks? [Wed Jun 3 18:21:42 2015] <chewbranca>: I'm interested but I don't have the time right now [Wed Jun 3 18:21:46 2015] <jan____>: ACTION steps back [Wed Jun 3 18:22:32 2015] <Kxepal>: #action Kxepal to investigate into CouchDB benchmarks problem [Wed Jun 3 18:22:35 2015] <sebastianrothbuc>: ditto - would love to (using sth from this century) but... [Wed Jun 3 18:22:44 2015] <Kxepal>: ok, I'll take a look onto this [Wed Jun 3 18:22:48 2015] <sebastianrothbuc>: great! [Wed Jun 3 18:22:49 2015] <chewbranca>: maybe the next step is getting a mailing list discussion going about what would be some fruitful benchmarks and then people can tackle those as they have time [Wed Jun 3 18:23:08 2015] <Kxepal>: meanwhile will try locutus, but I'm sceptic of having python for benchmarking [Wed Jun 3 18:23:13 2015] <chewbranca>: for instance querying the same doc repeatedly is not a representative benchmark [Wed Jun 3 18:23:22 2015] <chewbranca>: Kxepal: take a look at wrk2, it's pretty legit [Wed Jun 3 18:23:23 2015] <Kxepal>: yandex.tank doing the same, but they have C-based i/o library [Wed Jun 3 18:23:37 2015] <chewbranca>: https://github.com/giltene/wrk2 [Wed Jun 3 18:23:45 2015] <Kxepal>: chewbranca: I'm not sure how I will communicate with wrk2 from Erlang/Python/whatever [Wed Jun 3 18:23:47 2015] <Kxepal>: stdio? [Wed Jun 3 18:24:03 2015] <chewbranca>: Kxepal: it's an http benchmarking tool [Wed Jun 3 18:24:35 2015] <Kxepal>: chewbranca: I know, but I need 1) prepare environment 2) run benchmark 3) gather the results and format them [Wed Jun 3 18:25:35 2015] <Kxepal>: it's still some scripting process [Wed Jun 3 18:25:50 2015] <chewbranca>: sure [Wed Jun 3 18:26:09 2015] <chewbranca>: also worth noting if people are benchmarking 1.x it should be done with delayed_commits=false [Wed Jun 3 18:26:51 2015] <Kxepal>: chewbranca: that a hacks (: [Wed Jun 3 18:26:56 2015] <sebastianrothbuc>: oh yes - good point [Wed Jun 3 18:26:56 2015] <chewbranca>: what is? [Wed Jun 3 18:27:13 2015] <Wohali>: Kxepal: delayed_commits=true is a hack [Wed Jun 3 18:27:25 2015] <chewbranca>: oh sure, a benchmarking hack even [Wed Jun 3 18:27:31 2015] <chewbranca>: that's enabled by default in 1.x [Wed Jun 3 18:28:16 2015] <jan____>: anything else on benchmarks? [Wed Jun 3 18:28:17 2015] <rnewson>: and off in 2.x [Wed Jun 3 18:28:22 2015] <Wohali>: so this sounds like something we just need to socialize [Wed Jun 3 18:28:34 2015] <Wohali>: document as improving our commitment to data integrity, etc [Wed Jun 3 18:28:58 2015] <Kxepal>: ok...as about other CouchDB 2.0 news: CORS support for chttpd is merged (thanks chewbranca!), Cloudant started to backport various useful changes for replicator, config and fabric. [Wed Jun 3 18:29:07 2015] <jan____>: Wohali: Iâd say leave it be for 1.x, 2.0 has the better setting, possibly remove in 3.0. [Wed Jun 3 18:29:12 2015] <Wohali>: ty cloudant [Wed Jun 3 18:29:14 2015] <Kxepal>: and robertkowalski made first steps for making CouchDB 2.0 faster (: [Wed Jun 3 18:29:19 2015] <Wohali>: jan____: yes i mean document in 2.0 release [Wed Jun 3 18:29:21 2015] <jan____>: ay CORS! [Wed Jun 3 18:29:36 2015] <jan____>: Wohali: Iâd just not mention it :D [Wed Jun 3 18:29:52 2015] <jan____>: maybe in the âchangesâ section [Wed Jun 3 18:30:41 2015] <Kxepal>: Wohali: indeed, it's changes section with backed by migration guide. ## Open floor ## [Wed Jun 3 18:33:47 2015] <jan____>: ApacheCon EU is ramping up again [Wed Jun 3 18:33:56 2015] <jan____>: this year it is split into two events: Core and Big Data [Wed Jun 3 18:34:26 2015] <jan____>: The call for speakers are open for both and we should have a strong CouchDB presence: [Wed Jun 3 18:34:37 2015] <jan____>: - http://events.linuxfoundation.org/events/apachecon-core-europe/program/cfp [Wed Jun 3 18:34:48 2015] <jan____>: - http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp [Wed Jun 3 18:35:14 2015] <jan____>: I personally wonât be able to make it, but Iâd love for us to come up with a full-day CouchDB track like we had last year [Wed Jun 3 18:35:29 2015] <jan____>: #link ApacheCon Core Call for Participation: http://events.linuxfoundation.org/events/apachecon-core-europe/program/cfp [Wed Jun 3 18:35:55 2015] <jan____>: #link ApacheCon BigData Call for Participation: http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp [Wed Jun 3 18:36:19 2015] <jan____>: especially with 2.0 now finally playing in the âbig dataâ leagues, we have a bunch of relevant things to say [Wed Jun 3 18:36:36 2015] <bigbluehat>: FYI those CFP dates are slightly different...keep that in mind when planning :) [Wed Jun 3 18:37:49 2015] <jan____>: Iâd be happy to help come up with a programme, if we find enough speakers ## L10n ## [Wed Jun 3 18:38:41 2015] <Kxepal>: awenkhh: the floor is yours (: [Wed Jun 3 18:38:55 2015] <awenkhh>: As some of you may have seen, I started a thread on L10n@ [Wed Jun 3 18:39:10 2015] <awenkhh>: I tried to get an idea what the status is [Wed Jun 3 18:39:29 2015] <awenkhh>: as expected, not too many people replied but some. [Wed Jun 3 18:40:02 2015] <awenkhh>: I was interrupted then and one bad thing is, that I missed to invite a participant from the CouchDB Day. I still have to invite him [Wed Jun 3 18:40:38 2015] <awenkhh>: On the other hand I am really not sure where the translation activities will lead us to. Because as of today I am still not sure how to bring them into the docs [Wed Jun 3 18:40:46 2015] <jan____>: it happens, donât feel bad about it! [Wed Jun 3 18:41:04 2015] <awenkhh>: and we still do not have a plan if this all will fit together and work out [Wed Jun 3 18:41:21 2015] <awenkhh>: jan____ yeah sure ... but still :( [Wed Jun 3 18:41:42 2015] <jan____>: awenkhh: yeah, I found robertkowalskiâs point the most interesting: stopped because they translations werenât visible anywhere. We need to think about how to make it more of an instant-gratification kind of thing [Wed Jun 3 18:41:47 2015] <jan____>: awenkhh: *hugs* [Wed Jun 3 18:41:47 2015] <Kxepal>: awenkhh: need to make such plan then. good news is RTD service where we host our docs finally got some more-or-less good support for localization. [Wed Jun 3 18:41:54 2015] <awenkhh>: so I would bve happy to hear opinions from others here - especially for 2.0 [Wed Jun 3 18:42:18 2015] <Kxepal>: year ago there was no such [Wed Jun 3 18:42:32 2015] <Wohali>: i am curious to hear what IBM's plans are to help with l10n. i know they assigned resources to this shortly before I left. [Wed Jun 3 18:42:35 2015] <awenkhh>: Kxepal cool - did not know that. I would be happy to see such a plan [Wed Jun 3 18:42:39 2015] <Wohali>: or in general documentation/etc [Wed Jun 3 18:42:44 2015] <awenkhh>: jan____ yes that's a big problem [Wed Jun 3 18:42:50 2015] <Wohali>: they claimed to be working "with the community" but i've seen no evidence of that [Wed Jun 3 18:43:10 2015] <awenkhh>: Wohali happy you mention that because I remeber that there have been plans or at least someone spoke about such plan [Wed Jun 3 18:43:15 2015] <awenkhh>: *plans [Wed Jun 3 18:43:38 2015] <Wohali>: rnewson sits closest to that person but i hesitate to ask him to ask that person to reach out, rnewson has enough on his plate. [Wed Jun 3 18:43:44 2015] <awenkhh>: Wohali and see - nobody participated at l10n so far from IBM (no offense!) [Wed Jun 3 18:43:53 2015] <michellep>: i can ask [Wed Jun 3 18:43:56 2015] <michellep>: who do i ask [Wed Jun 3 18:44:06 2015] <awenkhh>: Wohali sure ... fair point [Wed Jun 3 18:44:10 2015] <rnewson>: I'm unaware of any l10n effort from IBM for couchdb. [Wed Jun 3 18:44:24 2015] <Wohali>: rnewson: documentation people would know best [Wed Jun 3 18:44:33 2015] <awenkhh>: michellep oh super! Would be cool to get feedbck then at L10n@ ... [Wed Jun 3 18:44:59 2015] <Wohali>: i'm bad with names, michellep, or i'd give you one [Wed Jun 3 18:45:02 2015] <michellep>: ok [Wed Jun 3 18:45:06 2015] <awenkhh>: I invite everybody also to have a look at the thread and bring up ideas or suggestions [Wed Jun 3 18:45:14 2015] <Wohali>: some ibm guy in the uk that works on dox, christian would know his name [Wed Jun 3 18:45:28 2015] <michellep>: ok i'll start with him [Wed Jun 3 18:45:38 2015] <Wohali>: awenkhh: does l10n stuff include l10n for docs? [Wed Jun 3 18:45:44 2015] <Wohali>: or just the interface itself? [Wed Jun 3 18:45:53 2015] <Wohali>: because I'm not sure if fauxton has all of its strings externalized yet [Wed Jun 3 18:46:09 2015] <awenkhh>: Wohali it's the translation of docs.couchdb.org [Wed Jun 3 18:46:34 2015] <Wohali>: awenkhh: ahhhh ok, so not the actual software itself, just documentation. Yes, michellep, start with that person. I want to say his name is Adrian but I'm probably completely wrong. [Wed Jun 3 18:47:06 2015] <robertkowalski>: Wohali: it is not externalized fully but it is at least possible since mango for fauxton [Wed Jun 3 18:47:10 2015] <Kxepal>: #link http://mail-archives.apache.org/mod_mbox/couchdb-l10n/201505.mbox/%3CCADA9kr-RdH8_C0_cF9TLdehCTRiwD3WiXP1T2tfa_iBK5GwYQA%40mail.gmail.com%3E [Wed Jun 3 18:47:24 2015] <Kxepal>: ^^^ link with status of translation activities [Wed Jun 3 18:47:36 2015] <awenkhh>: Wohali yes it's just the documentation [Wed Jun 3 18:47:43 2015] <awenkhh>: let's say enduser docs [Wed Jun 3 18:47:55 2015] <awenkhh>: ok that's all from me ... [Wed Jun 3 18:48:01 2015] <Wohali>: robertkowalski: good news, sounds like an excellent second phase to go thru [Wed Jun 3 18:48:08 2015] <robertkowalski>: Wohali: for fauxton the main problem is also that we sometimes just display error messages from couch that come from the erlang core [Wed Jun 3 18:48:11 2015] <jan____>: we have no plans to localize Fauxton yet, although I love the idea. I think the mechanism is going to be different from how we do docs, though [Wed Jun 3 18:48:34 2015] <robertkowalski>: yes, sorry for derailing [Wed Jun 3 18:48:41 2015] <Wohali>: jan____: when I was talking to IBM people they had a single team internally who did all of that [Wed Jun 3 18:48:41 2015] <awenkhh>: jan____ most definitely ... [Wed Jun 3 18:48:51 2015] <Kxepal>: jan____: if Fauxton localization will be used gettext, then it wouldn't be much different, thought testing will be hard /: [Wed Jun 3 18:49:11 2015] <Wohali>: jan____: point being, they have resources, they want to help, but...their workflow is VERY different from ours and the interface will be ugly. someone on the IBM side is going to have to spearhead efforts to get their DITA-based workflow into our stuff. [Wed Jun 3 18:49:23 2015] <jan____>: Wohali: sure, but they will still use different tech for l10n software vs. text [Wed Jun 3 18:49:40 2015] <awenkhh>: isn't it gettext ? (*.po) [Wed Jun 3 18:49:41 2015] <Wohali>: correct [Wed Jun 3 18:49:51 2015] <jan____>: Wohali: thatâs all great, Iâm happt to work with them the second they show up on our door step :) [Wed Jun 3 18:49:51 2015] <awenkhh>: what ARE they using then? [Wed Jun 3 18:49:51 2015] <Kxepal>: awenkhh: yes [Wed Jun 3 18:49:52 2015] <Wohali>: their l10n of text is what I'm talking about now [Wed Jun 3 18:50:05 2015] <Wohali>: UI agreed is a future discussion [Wed Jun 3 18:50:08 2015] <Wohali>: sorry if i confused the waters [Wed Jun 3 18:50:14 2015] <Wohali>: i am on some rather intense painkillers at the moment [Wed Jun 3 18:50:23 2015] <jan____>: no worries [Wed Jun 3 18:50:36 2015] <awenkhh>: yeah got it ... no problem ;-) [Wed Jun 3 18:50:53 2015] <michellep>: wait [Wed Jun 3 18:51:21 2015] <michellep>: should i just ask them the questions in the email thread, and that status? is there anything else i should ask [Wed Jun 3 18:51:37 2015] <Wohali>: if they are willing to help yes [Wed Jun 3 18:51:40 2015] <Wohali>: and if so how [Wed Jun 3 18:51:46 2015] <michellep>: ok [Wed Jun 3 18:52:10 2015] <awenkhh>: yes exactly [Wed Jun 3 18:52:32 2015] <awenkhh>: michellep if it helps, we can outline how we work now ... [Wed Jun 3 18:52:59 2015] <michellep>: ok [Wed Jun 3 18:53:04 2015] <michellep>: that would help [Wed Jun 3 18:53:08 2015] <awenkhh>: michellep as a start: https://cwiki.apache.org/confluence/display/COUCHDB/Translation+Guide [Wed Jun 3 18:53:47 2015] <awenkhh>: but there are not the technical details how we import the files into the Pootle tool ... but I think that's not that important [Wed Jun 3 18:54:04 2015] <awenkhh>: the important fact is, that we use getext [Wed Jun 3 18:54:15 2015] <awenkhh>: * gettext [Wed Jun 3 18:55:48 2015] <Kxepal>: #action Kxepal to check out the L10n possibilities with read the docs service [Wed Jun 3 18:55:58 2015] <Kxepal>: awenkhh: ok, no problem [Wed Jun 3 18:56:49 2015] <Kxepal>: thought I would rather host on docs on CouchDB as attachments. that gives more flexibility for managing them. [Wed Jun 3 18:57:01 2015] <Kxepal>: ACTION just dreaming... [Wed Jun 3 18:57:41 2015] <awenkhh>: Kxepal sure that would be great! [Wed Jun 3 18:58:36 2015] <Kxepal>: awenkhh: yes, but need to figure out how to get into our VM deep inside ASF and configure it to handle docs.couchdb.org requests...quite complicated quest (: [Wed Jun 3 18:58:46 2015] <Kxepal>: thought, will see [Wed Jun 3 18:59:20 2015] <jan____>: letâs not go there right now [Wed Jun 3 19:00:07 2015] <Kxepal>: (: ## Open floor ## [Wed Jun 3 19:00:14 2015] <awenkhh>: yeah I think it is ok to find a way with wath have now [Wed Jun 3 19:00:25 2015] <Kxepal>: ok, anything more else for today? [Wed Jun 3 19:00:36 2015] <robertkowalski>: i am on vacations for 15 days starting this friday! [Wed Jun 3 19:00:46 2015] <robertkowalski>: i will not read my mail! [Wed Jun 3 19:00:47 2015] <awenkhh>: YAY :D lucky you are [Wed Jun 3 19:00:48 2015] <Kxepal>: robertkowalski: hooray! (: [Wed Jun 3 19:00:49 2015] <jan____>: ACTION falls through the open floor into an infinite pit [Wed Jun 3 19:00:57 2015] <jan____>: robertkowalski: yay, happy vacationing! [Wed Jun 3 19:01:03 2015] <awenkhh>: jan____ ha ha :D [Wed Jun 3 19:01:17 2015] <Wohali>: ACTION steals jan____'s portal gun [Wed Jun 3 19:01:42 2015] <Kxepal>: ahaha [Wed Jun 3 19:02:28 2015] <Kxepal>: oh, on dev@ ML GitHub issues proposal is awaiting us [Wed Jun 3 19:02:51 2015] <Kxepal>: so let's end the meeting on this good news (: [Wed Jun 3 19:02:58 2015] <Kxepal>: ASFBot: meeting end Meeting ended at Wed Jun 3 19:02:58 2015
