Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread David Lee
In pure generic XQuery there is one output; So if you want 7 outputs you need to either produce a "sequence" of 7 items or a single document encapsulating them. To produce a sequence you don't actually need to collect the values just let them evaluate let $doc = doc("huge.xml") return ( filte

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Ihe Onwuka
I only read the ginormous XML once... I apply the 7 filters to each node read and it gets allocated to one of the 7 output buckets (hows that for a semantically neutral term). On Tue, Jan 14, 2014 at 3:50 AM, Ihe Onwuka wrote: > Input ginormous xml document containing data relating to a cre

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Ihe Onwuka
Input ginormous xml document containing data relating to a credit card application that I only want to read once. Filters. - under 25's, college educated, single mother, bankrupts, 6 figure income, repeat applicants, self-employed. 1 input - 7 outputs All applicants that meet the criteria o

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread David Lee
A Single XQuery invocation has only one "output stream" ... but when your working in a database often you don't need any, you want to put the data back into the database. Functions don't have "streams" really either although the word is overloaded. One way it to produce a "sequence" of results wh

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Ihe Onwuka
If it helps think of an entity that entails a bazillion fields like a credit card application and you have multiple ways you wish to classify it. So you have a filter for each classification scheme and the application "filters" through to the output stream of whichever predicates return true. On

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Ihe Onwuka
take the simplest case. a straight copy of the node provided the predicate passes. On Tue, Jan 14, 2014 at 12:38 AM, David Lee wrote: > Could you define what you mean by "output" ? > > > David A. Lee > d...@calldei.com > http://www.xmlsh.org > > >

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread David Lee
Could you define what you mean by "output" ? David A. Lee d...@calldei.com http://www.xmlsh.org -Original Message- From: Ihe Onwuka [mailto:ihe.onw...@gmail.com] Sent: Monday, January 13, 2014 4:19 PM To: David Lee Cc: talk@x-query.com Subject:

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Ihe Onwuka
On Mon, Jan 13, 2014 at 11:54 PM, David Lee wrote: > If your running in exist then pure XQuery is probably as good or better then > anything else. > Could you expand on your problem ? > I have a collection. Suppose I have 10 different outputs I want and the decision as to which output a node is

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread David Lee
XML Databases tend to be good at this (keeping things cached, avoiding unnecessary parsing and serialization, etc). If it is a NxM problem that may be fine unless it goes too slow. Then you might want to see about ways of optimization ... Generally though (not sure about Exist but I suspect it f

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Adam Retter
> > I don't know exist that well but typically once a document is fetched into > memory in a XML DB it can stay cached, > But if you are loading too many docs the cache will get full and it will have > to reload the docs. > > That is assuming that the size of your documents is bigger than the >

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread David Lee
If your running in exist then pure XQuery is probably as good or better then anything else. Could you expand on your problem ? I don't know exist that well but I cant think off hand of a better solution Unless there is a shortcut to know ahead of time what transforms to apply ... Do make sure you

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Adam Retter
Ihe, I think the XQuery solution is exactly as you described it. A recursive descent, most likely starting with an identity transform, and a sequence of functions that can be combined and applied at each level of the descent. On 13 January 2014 21:12, Ihe Onwuka wrote: > The documents are in an

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Ihe Onwuka
The documents are in an eXist database hence I was expecting and think I need an XQuery solution but am open to other approaches. On Mon, Jan 13, 2014 at 8:40 PM, David Lee wrote: > This is the type of problem xmlsh and XProc were designed for ... > What engine are you using? I personally prefer

[xquery-talk] Ignore ; ) Re: Multiple output via a stream of filters

2014-01-13 Thread David Lee
Sorry replied to wrong email Sent from my iPad (excuse the terseness) David A Lee d...@calldei.com > On Jan 13, 2014, at 12:43 PM, "David Lee" wrote: > > FYI working on build machine and store 2 today ... Let me know when you need > it back. > Maybe we need a store3;) > > > Sent from my iP

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread David Lee
FYI working on build machine and store 2 today ... Let me know when you need it back. Maybe we need a store3;) Sent from my iPad (excuse the terseness) David A Lee d...@calldei.com > On Jan 13, 2014, at 12:40 PM, "David Lee" wrote: > > This is the type of problem xmlsh and XProc were design

Re: [xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread David Lee
This is the type of problem xmlsh and XProc were designed for ... What engine are you using? I personally prefer designing with lots of small programs instead of a monolith. This is practical only if the startup overhead for each is small and preferably if in memory data can be passed between

[xquery-talk] Multiple output via a stream of filters

2014-01-13 Thread Ihe Onwuka
I am running through about a gigabyte worth of xml documents. The ideal processing scenario is to offer each node in the sequence to a list of filters and augment different XML documents (or different branches of one encompassing document) based on the outcome of the filter. If anyone has seen