Hi Stefan, I engineered the changes to be MonetDB4 and even MonetDB5 friendly, and though PF_ROX is a proof of concept branch, the concepts will at a later stage re-appear in a main branch (certainly those on a GDK level).
However, the changes in MonetDB to support it are minimal; it is not a sacrifice to have them in a HEAD branch. I actually think we should just make that little bit of effort to port PF_ROX to the MonetDB[4] HEAD, and take it from there. Part of that step is actually the propagation of my branch changes to the HEAD (as I was assuming would happen semi-automatically). Peter > -----Original Message----- > From: Stefan Manegold [mailto:[EMAIL PROTECTED] > Sent: Wednesday, October 29, 2008 10:35 AM > To: [email protected] > Cc: [EMAIL PROTECTED] > Subject: Re: [Monetdb-checkins] MonetDB4/src/modules/contrib malalgebra.mx, > MonetDB_4-24, 1.8.2.1, 1.8.2.2 > > On Wed, Oct 29, 2008 at 10:27:35AM +0100, Stefan Manegold wrote: > > Gents, > > > > given (1) the inherent incompatibility between the PF_ROX branch (forked > of > > the XQuery_0-24 branch of pathfinder) and the pathfinder development > trunk > > and (hence) the incompatibility between the PF_ROX branch and the > development trunks of MonetDB & MonetDB4 > > > and the facts that (2) PF_ROX (at least for now) is purely for internal > > research purposes and (3) cahnges ("features") in PF_ROX do (e.g.) not > > support updates, i.e., are far from becoming "main stream", > > > > I guess the (only?) proper solution would (have been?) to also fork > PF_ROX > > branches of the MonetDB_1-24 branch (of MonetDB) and the MonetDB_4-24 > branch > > (of MonetDB4) to accomodate Peter's yesterday's changes. > > > > Note though, that this would / will exclude PF_ROX-specific changes in > > MonetDB & MonetDB$ from being tested --- this is already the case for the > > pathfinder PF_ROX branch and will be the case for the MonetDB_1-24 & > > MonetDB_4-24 branches, too, once we start prparing for the next release > no > > later than next week. > > Also, changes from the PF_ROX branches of MonetDB & MonetDB4 would/will > than > NOT be propagated to the respective development trunks; the same will be > the > case for (potential0 changes to the MonetDB_1-24 & MonetDB_4-24 branches > as soon as we start prparing for the next release no later than next week. > > In fact, I would even suggest to not propagate Peter's yesterday's changes > to the development trunk since they are "PF_ROX-specific" (at least for > now). > > Stefan > > > See the results of "Stable" testing (once they are ready later today) to > > judged whether testing is "desired"/"required" or not ... > > > > Stefan > > > > On Tue, Oct 28, 2008 at 11:36:17PM +0100, Sjoerd Mullender wrote: > > > On 2008-10-28 22:37, Peter Boncz wrote: > > > > Hi Sjoerd, > > > > > > > > The one problem that could occur is people trying to check out > MonetDB_4-24 > > > > now, compiling it and trying to use with a precompiled MonetDB_1-24, > without > > > > updating/recompiling that one. So, I agree with you that a problem > could > > > > occur, but I would be surprised if any of our users would really hit > this. > > > > > > Because we are going to abandon this branch I was willing to let the > > > change pass. But in general I am opposed. > > > > > > I agree that in this particular case, it is unlikely that people will > > > encounter any problems. But again, that is because we are going to > > > abandon this branch. > > > > > > > With excuses to pick nits, but contrary to you I believe that API > appends > > > > are not "just as bad" as API modifications. An API modification in > MonetDB, > > > > would force a release in MonetDB5, for instance. An API append does > not do > > > > that (so it is "less bad"). > > > > > > It *is* bad. See the disaster involving the addition of True and False > > > in Python in a minor release. They have sworn to never make that > > > mistake again. People expect API stability in minor releases and > adding > > > something to the API is *not* stability. > > > > > > > PF_ROX is quite different from the HEAD in the pathfinder code due to > bot > > > > branches making different changes to the indexing scheme; afaik this > has > > > > prevented us from keeping it in sync with the HEAD. Various API > changes (not > > > > appends) and even buildtool changes now force it to depend on the > stable > > > > MonetDB/MonetDB4. For PF_ROX itself this is not a problem as it has > no > > > > outside users. However, it might occur that this scenario will repeat > > > > (PF_ROX triggering a change/addition in MonetDB/MonetDB4 that would > trickle > > > > towards the head via MonetDB_[14]-24). > > > > > > I am not talking about the PF_ROX branch. It is not a "stable" branch > > > and you can do with it what you want. But please do consider whether > it > > > is possible to use the development branches of MonetDB and MonetDB4 > with > > > PF_ROX. If you can do that, you can make needed changes to the API > there. > > > > > > > Peter > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: Sjoerd Mullender [mailto:[EMAIL PROTECTED] > > > > Sent: Tuesday, October 28, 2008 9:50 PM > > > > To: [email protected] > > > > Cc: [EMAIL PROTECTED] > > > > Subject: Re: [Monetdb-checkins] MonetDB4/src/modules/contrib > malalgebra.mx, > > > > MonetDB_4-24, 1.8.2.1, 1.8.2.2 > > > > > > > > On 2008-10-28 20:20, Peter Boncz wrote: > > > >> Hi Sjoerd, > > > >> > > > >> Thanks for keeping an eye on developer behavior and effort to > educate. My > > > >> change, however, only introduces new functions in GDK and commands > in MIL; > > > > it > > > >> is only an API append. > > > > > > > > API appends are just as bad as any other API change. If new code > > > > depends on the extra calls, it cannot run with the older version of > the > > > > release. That is bad. It means that the branch is *not* "stable" > anymore. > > > > > > > >> It is checked into the stable branch because afaik PF_ROX only works > with > > > >> that. But I may as often be mistaken. > > > > > > > > I noticed it was for PF_ROX, and that does pose a problem. It seems > > > > that the PF_ROX branch should depend on the current version of > MonetDB > > > > and not on the stable version. > > > > > > > >> Peter > > > >> > > > >>> This type of change is not allowed on a stable branch. Changes of > API > > > >>> are *not* allowed. > > > >>> > > > >>> Since we're going to abandon this stable branch soon, I'll let it > pass. > > > >>> > > > >>> On 2008-10-28 18:49, Peter Boncz wrote: > > > >>>> Update of /cvsroot/monetdb/MonetDB4/src/modules/contrib > > > >>>> In directory > > > >>>> 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv901/src/modules/contrib > > > >>>> > > > >>>> Modified Files: > > > >>>> Tag: MonetDB_4-24 > > > >>>> malalgebra.mx > > > >>>> Log Message: > > > >>>> changes to minimize sampling overhead (mostly in XML text index) > > > >>>> > > > >>>> - introduce bandmergejoin(), and leftmergejoin()/bandmergejoin() > with a > > > >>>> cutoff > > > >>>> > > > >>>> > > > >>>> > > > >>>> U malalgebra.mx > > > >>>> Index: malalgebra.mx > > > >>>> > =================================================================== > > > >>>> RCS file: > /cvsroot/monetdb/MonetDB4/src/modules/contrib/malalgebra.mx,v > > > >>>> retrieving revision 1.8.2.1 > > > >>>> retrieving revision 1.8.2.2 > > > >>>> diff -u -d -r1.8.2.1 -r1.8.2.2 > > > >>>> --- malalgebra.mx 10 Oct 2008 08:42:22 -0000 1.8.2.1 > > > >>>> +++ malalgebra.mx 28 Oct 2008 17:49:34 -0000 1.8.2.2 > > > >>>> @@ -43,7 +43,6 @@ > > > >>>> @:leftjoin(fetch,\nCAUTION: positional matches are assumed not to > be > > > >>>> out-of-bounds!!)@ > > > >>>> @:leftjoin(merge)@ > > > >>>> @:leftjoin(hash)@ > > > >>>> - > > > >>>> .COMMAND leftthetajoin ( BAT[any::1,any::2] left, > BAT[any::2,any::3] > > > >>>> right, int mode) : > > > >>>> BAT[any::1,any::3] = CMDleftthetajoin; > > > >>>> "Hook directly into the 'leftthetajoin' implementation of the > join. > > > >>>> @@ -58,6 +57,22 @@ > > > >>>> Also, for each left tuple, all matching right tuples will > appear in > > > >>>> their order > > > >>>> of appearrance in the right BAT. This property is handy for > XQuery > > > >>>> processing." > > > >>>> > > > >>>> + .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, > BAT[any::2,any::3] > > > >>>> inner, > > > >>>> + any::2 minus, any::2 plus, bit > l_in, bit > > > >>>> h_in) : > > > >>>> + BAT[any::1,any::3] = > CMDbandmergejoin; > > > >>>> + "The bandjoin algorithm, but forced to use (left) mergejoin > rather > > > > than > > > >>>> nested loop" > > > >>>> + > > > >>>> + .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, > BAT[any::2,any::3] > > > >>>> inner, > > > >>>> + any::2 minus, any::2 plus, bit > l_in, bit > > > >>>> h_in, lng limit) : > > > >>>> + BAT[lng,BAT] = > CMDbandmergejoin_limit; > > > >>>> + "A bandjoin that uses the merge algorithm and limits the result > size. > > > >>>> + Result generation is cut off after this point and the > > > >>>> + result [lng,BAT] holds > [estimated_full_resultsize,limited_result]" > > > >>>> + > > > >>>> + .COMMAND leftmergejoin ( BAT[any::1,any::2] left, > BAT[any::2,any::3] > > > >>>> right, lng limit) : > > > >>>> + BAT[lng,BAT] = CMDleftmergejoin_limit; > > > >>>> + "leftmergejoin, but limit the result (by cutting off result > > > > generation) > > > >>>> + return a BAT[lng,BAT] containing > > > > [full_result_estimate,limited_result]" > > > >>>> @= select > > > >>>> .COMMAND [EMAIL PROTECTED] ( BAT[any::1,any::2] b, any::2 low, > > > >>>> any::2 > high) > > > > : > > > >>>> @@ -119,6 +134,7 @@ > > > >>>> #ifndef __MALALGEBRA_H__ > > > >>>> #define __MALALGEBRA_H__ > > > >>>> #include "gdk.h" > > > >>>> +#include "gdk_rangejoin.h" > > > >>>> > > > >>>> /* nothing much */ > > > >>>> > > > >>>> @@ -240,6 +256,40 @@ > > > >>>> return (*result = BATnlthetajoin(l, r, *mode, (size_t) * > estimate)) ? > > > >>>> GDK_SUCCEED : GDK_FAIL; > > > >>>> } > > > >>>> > > > >>>> +int > > > >>>> +CMDbandmergejoin(BAT **result, BAT *left, BAT *right, ptr minus, > ptr > > > > plus, > > > >>>> bit *li, bit *hi) > > > >>>> +{ > > > >>>> + return (*result = BATbandmergejoin(left, right, minus, > plus, > > > > *li, > > > >>>> *hi)) ? GDK_SUCCEED : GDK_FAIL; > > > >>>> +} > > > >>>> + > > > >>>> +int > > > >>>> +CMDbandmergejoin_limit(BAT **result, BAT *left, BAT *right, ptr > minus, > > > > ptr > > > >>>> plus, bit *li, bit *hi, lng* limit) > > > >>>> +{ > > > >>>> + size_t cutoff = *limit; > > > >>>> + *result = BATbandmergejoin_limit(left, right, minus, > plus, *li, > > > >>>> *hi, &cutoff); > > > >>>> + if (*result) { > > > >>>> + bat bid = (*result)->batCacheid; > > > >>>> + *result = BATnew(TYPE_lng, TYPE_bat, 1); > > > >>>> + if (*result) BUNins(*result, &cutoff, &bid, > FALSE); > > > >>>> + BBPunfix(bid); > > > >>>> + } > > > >>>> + return (*result)?GDK_SUCCEED:GDK_FAIL; > > > >>>> +} > > > >>>> + > > > >>>> +int > > > >>>> +CMDleftmergejoin_limit(BAT **result, BAT *left, BAT *right, lng* > limit) > > > >>>> +{ > > > >>>> + size_t cutoff = *limit; > > > >>>> + *result = BATleftmergejoin_limit(left, right, cutoff, > &cutoff); > > > >>>> + if (*result) { > > > >>>> + bat bid = (*result)->batCacheid; > > > >>>> + *result = BATnew(TYPE_lng, TYPE_bat, 1); > > > >>>> + if (*result) BUNins(*result, &cutoff, &bid, > FALSE); > > > >>>> + BBPunfix(bid); > > > >>>> + } > > > >>>> + return (*result)?GDK_SUCCEED:GDK_FAIL; > > > >>>> +} > > > >>>> + > > > >>>> @= selectcmd > > > >>>> int [EMAIL PROTECTED](BAT **result, BAT* b, ptr value) { > > > >>>> return (*result = BAT_select_(b, value, 0, TRUE, TRUE, @2, > FALSE, > > > >>>> TRUE))?GDK_SUCCEED:GDK_FAIL; > > > >>>> > > > >>>> > > > >>>> > > > > --------------------------------------------------------------------- > ---- > > > >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's > > > > challenge > > > >>>> Build the coolest Linux based applications with Moblin SDK & win > great > > > >>>> prizes > > > >>>> Grand prize is a trip for two to an Open Source event anywhere in > the > > > > world > > > >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > >>>> _______________________________________________ > > > >>>> Monetdb-checkins mailing list > > > >>>> [EMAIL PROTECTED] > > > >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > >>> -- > > > >>> Sjoerd Mullender > > > >>> > > > >>> ------------------------------------------------------------------- > ------ > > > >>> This SF.Net email is sponsored by the Moblin Your Move Developer's > > > > challenge > > > >>> Build the coolest Linux based applications with Moblin SDK & win > great > > > > prizes > > > >>> Grand prize is a trip for two to an Open Source event anywhere in > the > > > > world > > > >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > >>> _______________________________________________ > > > >>> Monetdb-checkins mailing list > > > >>> [EMAIL PROTECTED] > > > >>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > >>> > > > >> > > > >> -------------------------------------------------------------------- > ----- > > > >> This SF.Net email is sponsored by the Moblin Your Move Developer's > > > > challenge > > > >> Build the coolest Linux based applications with Moblin SDK & win > great > > > > prizes > > > >> Grand prize is a trip for two to an Open Source event anywhere in > the > > > > world > > > >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > >> _______________________________________________ > > > >> Monetdb-checkins mailing list > > > >> [EMAIL PROTECTED] > > > >> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > > > > > > -- > > > > Sjoerd Mullender > > > > > > > > --------------------------------------------------------------------- > ---- > > > > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > > > > Build the coolest Linux based applications with Moblin SDK & win > great > > > > prizes > > > > Grand prize is a trip for two to an Open Source event anywhere in the > world > > > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > > _______________________________________________ > > > > Monetdb-checkins mailing list > > > > [EMAIL PROTECTED] > > > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > > > > > > --------------------------------------------------------------------- > ---- > > > > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > > > > Build the coolest Linux based applications with Moblin SDK & win > great prizes > > > > Grand prize is a trip for two to an Open Source event anywhere in the > world > > > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > > _______________________________________________ > > > > Monetdb-checkins mailing list > > > > [EMAIL PROTECTED] > > > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > > > -- > > > Sjoerd Mullender > > > > > > ----------------------------------------------------------------------- > -- > > > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > > > Build the coolest Linux based applications with Moblin SDK & win great > prizes > > > Grand prize is a trip for two to an Open Source event anywhere in the > world > > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > _______________________________________________ > > > Monetdb-checkins mailing list > > > [EMAIL PROTECTED] > > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > -- > > | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] | > > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > > | The Netherlands | Fax : +31 (20) 592-4312 | > > -- > | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] | > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > | The Netherlands | Fax : +31 (20) 592-4312 | ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Monetdb-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-developers
