Re: multiview using bloom filters

2010-09-24 Thread Paul Davis
I would say if you find it somewhere with an EPL header, then that's a good sign. But best to check with the original author that it was his intent. As an interesting aside, I'm not even sure if its possible that someone that's not an employee of Ericson can release something under the EPL. I'm no

Re: multiview using bloom filters

2010-09-24 Thread Norman Barker
Paul, yes, performance is actually much better (for some of our harder queries, so all docs over time with field X (two views), 10x faster), I am testing with docs that in total emit ~100K of keys (following the raindrop megaview). Some of the scalable bloom filter project contained EPL headers,

Re: Simple erlang and erl question

2010-09-24 Thread chongqing xiao
Understand. Anyhow, I figured out the problem. It is because I have to load "couch_stats_collector" first in order to use "couch_file". I am surprised even "couch_file" depends on "couch_stats_collector", the dependency is not explicitly defined in "couch_file". Chong On Fri, Sep 24, 2010 at 2:57

Re: multiview using bloom filters

2010-09-24 Thread Paul Davis
Norman, Just glanced through. Looks better. Any feeling for a performance differences? Also, I glanced at the original files that you linked to. The bit array files didn't have a license, but what you've got there does have EPL headers. We need to make sure we have permission to do so. I would as

multiview using bloom filters

2010-09-24 Thread Norman Barker
Hi, thanks to Paul's excellent suggestion I have rewritten the multiview to use bloom filters, I had a concern that a bloom filter per view would use too much memory but thanks in the main to excellent implementation of bloom filters in erlang (http://sites.google.com/site/scalablebloomfilters/) t

Re: Replacing the _external API

2010-09-24 Thread Filipe David Manana
Paul, you have my +1 cheers On Fri, Sep 24, 2010 at 7:10 PM, Paul Davis wrote: > At CouchCamp there was a bit of discussion on replacing the _external > API with something a bit more modern to give _external processes more > control over their environment. > > The idea was born out of a discussi

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
"I met a real, live Pick user/admin/developer..." http://www.chesnok.com/daily/2010/09/11/couchcamp-2010-yay/ On Fri, Sep 24, 2010 at 3:49 PM, Stephen Prater wrote: > PICK story alert! > > We have that exact implementation for communication with our PICK servers. >  Works pretty well, and since

Re: continuous connection with jquery

2010-09-24 Thread Chris Anderson
continuous and jsonp are incompatible. use longpoll On Tue, Sep 21, 2010 at 10:06 AM, Tom Bichay wrote: > > Hi, > > I am trying to implement a continuous _changes connection to couchdb > (cloudant) with jQuery from a different domain (that's why I am using JSONP). > > With: > > $.ajax({ >      

Re: Simple erlang and erl question

2010-09-24 Thread Robert Newson
A mailing list is not a great medium for learning a new programming language or getting up to speed with a new code base. Try this for the former http://oreilly.com/catalog/9781934356005 and #couchdb for the latter. B. On Fri, Sep 24, 2010 at 8:39 PM, chongqing xiao wrote: > My bad. Now I get t

Re: Replacing the _external API

2010-09-24 Thread Stephen Prater
PICK story alert! We have that exact implementation for communication with our PICK servers. Works pretty well, and since PICK does really horrifying things when you feed it bad data, it provides us another level of input sanity checking. (No, we will not accept your ^Y, thank you.) On

Re: Simple erlang and erl question

2010-09-24 Thread chongqing xiao
My bad. Now I get this 10> couch_file:open("ttt"). ** exception exit: undef in function couch_stats_collector:track_process_count/1 called as couch_stats_collector:track_process_count({couchdb, open_os_files}) in call

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
Dave, My knee-jerk solution would be to write a small Python script that translates HTTP traffic to the current protocol. Ie, present your scripts with the same interface, but not from inside the Erlang VM. I'm not sure if I'm missing any details that would prevent that from working though. HTH,

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
Well, HTTP is implied, but that doesn't require a choice between TCP or Unix domain sockets. Unless there's something weird in Erlang, supporting both transports should be fairly trivial. I'd considered figuring out someway to support arbitrary TCP connections but the initial complexity level conv

Re: Replacing the _external API

2010-09-24 Thread David Hardtke
Hi Robert, Assume you've got a working program that communicates over stdio and want to use it as an external. Or if you're using some antiquated programing language without good http libraries (octave and matlab, for instance). stdio is very easy to use. Dave On 09/24/10 11:46, Robert New

Re: Simple erlang and erl question

2010-09-24 Thread Robert Newson
you need a : where you have a . On Fri, Sep 24, 2010 at 7:48 PM, chongqing xiao wrote: > Hi, > > I was learning erlang and couchdb using the erl and I got the > following error when I tried to use couch_file in erl shell. > Could someone explain what might be the problem? I am guessing > couch_fi

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
On Fri, Sep 24, 2010 at 2:48 PM, Zachary Zolton wrote: > Paul, > > Have you given any thought to having external processes being able to > write to the CouchDB log, or show up in the _active_tasks? > For things managed by the process management, I was considering just merging anything that came f

Re: Replacing the _external API

2010-09-24 Thread Sebastian Cohnen
Does your proxy imply http or tcp? I'm thinking about unix sockets, which would be awesome for many kinds of applications. On 24.09.2010, at 20:10, Paul Davis wrote: > At CouchCamp there was a bit of discussion on replacing the _external > API with something a bit more modern to give _external p

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
David, I think _externals would be deprecated and released after a couple versions. Is there a specific reason for wanting to keep the stdio protocol around? As to Erlang externals, I would probably classify anything running in the same VM as a plugin like what Volker is doing with GeoCouch. Also

Re: Replacing the _external API

2010-09-24 Thread Zachary Zolton
Paul, Have you given any thought to having external processes being able to write to the CouchDB log, or show up in the _active_tasks? For example, it would be neat if I could watch the progress of Lucene indexing along side of building view indexes. That kind of integration makes it more compel

Simple erlang and erl question

2010-09-24 Thread chongqing xiao
Hi, I was learning erlang and couchdb using the erl and I got the following error when I tried to use couch_file in erl shell. Could someone explain what might be the problem? I am guessing couch_file is not the actual module but I can't figure out how to get that. ฀Eshell V5.8.1 (abort with ^G)

Re: Replacing the _external API

2010-09-24 Thread Robert Newson
Who would use stdio when a concurrent and HTTP-based alternative was available? On Fri, Sep 24, 2010 at 7:43 PM, David Hardtke wrote: > Hi Paul, > > Is it possible to support both?  The precedence is query servers, where you > can define either a stdio based query_server or a erlang/otp based > n

Re: Replacing the _external API

2010-09-24 Thread David Hardtke
Hi Paul, Is it possible to support both? The precedence is query servers, where you can define either a stdio based query_server or a erlang/otp based native_query_server. Depending on what you are trying to do, these multiple options come in handy. In fact, it would be really nice to hav

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
On Fri, Sep 24, 2010 at 2:28 PM, Stephen Prater wrote: > As one of the people who wanted the external process management, that's a +1 > from me (if my vote counts.) > Every vote counts, its how we measure community consensus. > But I like the sound of the reverse proxy protocol for externals too

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
On Fri, Sep 24, 2010 at 2:26 PM, Norman Barker wrote: > Paul, > > for us, the process management is critical, we have external process > code and due to this process code size and age it is possible that it > may crash (this is not that unusual!), so I am +1 on process > management and am interest

Re: Replacing the _external API

2010-09-24 Thread Paul Davis
On Fri, Sep 24, 2010 at 2:19 PM, Robert Newson wrote: > Assuming it's straightforward to extend OTP-style process monitoring > to external processes (and I'm assuming that the couchjs processes are > so monitored today) then I like the proposal to add both of these > things. > Yeah, intend for th

Re: Replacing the _external API

2010-09-24 Thread Stephen Prater
As one of the people who wanted the external process management, that's a +1 from me (if my vote counts.) But I like the sound of the reverse proxy protocol for externals too. stephen On Sep 24, 2010, at 1:19 PM, Robert Newson wrote: Assuming it's straightforward to extend OTP-style process

Re: Replacing the _external API

2010-09-24 Thread Norman Barker
Paul, for us, the process management is critical, we have external process code and due to this process code size and age it is possible that it may crash (this is not that unusual!), so I am +1 on process management and am interested in helping. As for reverse proxy is this going to be inside co

Re: Replacing the _external API

2010-09-24 Thread Robert Newson
Assuming it's straightforward to extend OTP-style process monitoring to external processes (and I'm assuming that the couchjs processes are so monitored today) then I like the proposal to add both of these things. My obvious motivation is couchdb-lucene so, with that hat on, would this mechanism o

Replacing the _external API

2010-09-24 Thread Paul Davis
At CouchCamp there was a bit of discussion on replacing the _external API with something a bit more modern to give _external processes more control over their environment. The idea was born out of a discussion with Robert Newson who mentioned that couchdb-lucene really only needs a reverse proxy t