Hi Maurice, valid point.
Unfortunately, we (at least here at CWI) do not have the manpower an/or budget that Microsoft has (recall, our "clients" do not pay for our "product"). Hence, if a dual-path strategy is desired and/or required, please also consider who should be doing all the administration, maintenance, bug fixing, testing, releasing, etc... Stefan On Wed, Oct 17, 2007 at 10:15:17AM +0200, Keulen, M. van (Maurice) wrote: > Hi all, > > As a follow-up to Djoerd's response and my questions about releases, let > me say a little bit more about certain activities and sketch an analogy > which we might mimick. > > We are trying to get many people enthusiastic about XML databases and > XML-IR, as offered by MonetDB/XQuery in particular. To achieve some > visibility for our work, I have for example started several activities > in building production applications on top of MonetDB/XQuery and which > use PF/Tijah. For example, I promised a demonstration in january to > university-wide decision makers of the UT of a publication management > application which uses XML-exports from our E-Prints system including > the full-text of the papers. I hope it will be accepted as a production > application used by all researchers of the UT next to E-Prints, which > will over time demonstrate more advanced features and take over more and > more functionality of E-Prints. As Djoerd pointed out, porting PF/Tijah > to the algebra-backend in the proper way, is more a matter of a year > than of a few months. However, it would be certain death to such > production applications if the support for developing them would stop > (since algebra-only means without PF/Tijah for a long time). > Furthermore, many students following our courses and doing our projects > are very interested in XML databases including XML-IR and probabilistic > XML. A significant portion would be rather discouraged if they would not > be able to grab a stable release to start working and get some bugs fixed. > > Microsoft once had the problem that they had ideas for a fundamentally > different operating system kernel (NT). It was infeasible to make it > fully backward-compatible with Windows 95 (on top of MSDOS), but it > could also not abandon a large part of their userbase if their > applications would not work anymore. Therefore, they had two release > streams: one NT-based and one MSDOS-based. Both knew their releases > until in the end. When most of the applications were ported or at least > working on the new technology, they stopped releasing in the MSDOS-based > stream. > > The algebra-backend is our `new technology', so can't we do something > similar? Have an algebra-based release stream of MonetDB/XQuery > (starting with 0.22) capable of running a significant portion of > applications, and an mps-based release stream needed to serve the > PF/Tijah- and ProbXML-applications. It would mean an extra package under > "Download" on SF. Of course the algebra-based releases will have > priority, but it would make it possible to have a new release for for > example PF/Tijah when it is needed. Of course we will put in the > required effort, but cannot do this alone. > > What do you think of this? > > Maurice. > > Djoerd Hiemstra wrote: > > Dear all, > > > > Very good: no more milprinting! In absence of Jan, please let me add the > > plans for PF/Tijah. To make things slightly more complicated: The PF/Tijah > > team would like to use the algebra release for porting our proprietary > > search extensions to XQuery Full-text, see: > > http://www.w3.org/TR/xquery-full-text/ Some initial design decisions that > > we like to follow have been made by Stefan Klinger at the University of > > Konstanz. The consequences of such a port would be: > > > > 1. XQuery syntax will be extended for text search queries (now only via > > user-defined functions) > > 2. All algebra operations must be score aware (i.e., if a score column > > exists, scores have to be propagated and aggregated) > > > > We are aware that this affects everyone, although, only the extended > > queries will produce a score column, and algebra operations remain exactly > > the same if score columns are not there. The time goal of this release is > > XMAS 2008. In the mean time, we might not port the current PF/Tijah > > extensions to the algebra. > > > > Best, Djoerd. > > > > On Tue, 16 Oct 2007, Martin Kersten wrote: > > > > > >> LS, > >> > >> Good to see this discussion. Although the topic is primarily > >> MonetDB/XQuery it is good > >> to keep an eye on the global picture. Releases include more stuff and > >> requires a lot of > >> time to prepare. (the release itself easily consumes several months of > >> full time work, > >> many delivered by free (weekend) time!!!!) > >> Good planning and long-term internal use are a prerequisite. > >> Something we still have to learn. > >> Once the release cycle starts, the train can not stop and full power > >> is/should > >> be in the hands of a single person. The release manager currently is > >> Sjoerd, > >> one of the project admins! > >> > >> As soon as a RC is available, those with a non-standard platform > >> (windows, sun, > >> mac, vista) should immediately double check to assure it basically works. > >> This holds for all addressed and developers. > >> > >> In this respect, the time frame mentioned below is overly optimistic. > >> Anyone close to > >> the core group should work on the HEAD to help uncover problems from day 1. > >> Code changes should *not* be collected by individual programmers until > >> release time . > >> This is a major source of frustration, because you simply may not expect > >> your team members to spent day and night to fight the 'last' bugs. Let > >> alone to > >> quick-test your latest feature or performance improvement. > >> PLEASE, check-in daily your changes and make sure you have at least ran > >> mtest on your platform. And most of all, know that your changes are > >> scrutinized for potential quality problems by others. It is the best way > >> to avoid many bugs. > >> > >> Furthermore, changes in the interfaces visible to users and the effect > >> on their databases > >> should be kept to a minimum. Backward compatibility is a must as long as > >> it is manageable. > >> Beware that a change in storage structure may help a few extreme users, > >> but may affects > >> hundreds or thousands in their daily use. E.g. one of our users would > >> have to unload/reload > >> a 0.5Tb business critical application. > >> > >> MonetDB/Query and MonetDB/SQL are intertwined. This creates responsibility > >> both ways. > >> > >> Peter Boncz wrote: > >> > >>> Stefan suggested rightly to move this discussion to the mailing list. > >>> I also changed the language into english for that purpose. > >>> > >>> Maurice' original questions were: > >>> > >>> (a) whether the new release would be algebra-only (short answer: > >>> hopefully). > >>> > >> In my opinion this can only be answered if the algebra version, in its > >> full glory (without mps) > >> is used for several months internally, e.g. using students for stress > >> testing. > >> > >>> (b) whether our talk of a new release also refers to submodules, such as > >>> probxml and pftijah (short answer: yes) > >>> > >>> > >> Submodules basically follow their own release cycle. Regardless the > >> source, e.g. datacells > >> geom, .... Keeping in line is the responsibility of their owners, and > >> the core developers > >> should be aware that their changes may break the work of others. Or at > >> least drain a > >> possibly substantial amount of time. > >> > >>> (c) whether all issues on the wiki, see: > >>> http://www.pathfinder-xquery.org/wiki/index.php/Jupiter:_The_Algebra_Release > >>> needed for this action, have already been resolved (short answer: no) > >>> > >>> > >> The pathfinder wiki is a good example of team documentation. > >> > >> I am looking forward to the upcoming release. The next one might be this > >> year, > >> but not necessarily includes everything on the drawing board. > >> > >> regards, Martin > >> > >>> On Tue, Oct 16, 2007 at 05:23:43PM +0200, Peter Boncz wrote: > >>> > >>> > >>>> Hi Maurice, et al, > >>>> > >>>> The release of (hopefully) this week will still be mps based. I also > >>>> anticipate at least one bug-fix release (ie 0.20.2), or even more. > >>>> However, > >>>> the next major release (0.22) should be algebra based, in my opinion. I > >>>> am > >>>> certainly reluctant to keep investing any more effort in mps. I would aim > >>>> for an mps-free release, yes. > >>>> > >>>> The time-goal of that release is "XMAS 2007", as far as we are concerned. > >>>> For the work of built-in functions I have good hope that with the help of > >>>> Lefteris we can come a long way. > >>>> > >>>> I hope and think updates can also be ported by joerd in that time-frame. > >>>> As > >>>> for pftijah, if Jan Flokstra has time, it should be possible to port > >>>> that as > >>>> well in 2-3 months. > >>>> > >>>> There are some hairy issues, one of them being free-form recursion > >>>> (needed > >>>> by you), another one being precompiled modules (needed by XRPC). The > >>>> resolution of these needs assistance from Munich, and maybe related. It > >>>> needs > >>>> a method to compile certain functions into scripted procedures (here MIL) > >>>> and calling them. One algorithm is to do this for all functions in > >>>> modules, > >>>> as well as for all recursive functions. The issue to resolve is the > >>>> representation of the parameters. > >>>> > >>>> Depending on how much time Jan Rittinger has available for this, we will > >>>> have to find creative solutions for this. Indeed, a backup solution > >>>> would be > >>>> to keep handling certain queries for the time being with mps, but that > >>>> surely is ugly. > >>>> > >>>> Peter > >>>> > >>>> -----Original Message----- > >>>> From: Keulen, M. van (Maurice) [mailto:[EMAIL PROTECTED] > >>>> Sent: dinsdag 16 oktober 2007 16:38 > >>>> To: Peter Boncz [CWI]; Stefan Manegold [CWI] > >>>> Cc: Djoerd Hiemstra; Jan Flokstra; Henning Rode; Abdel Kader, R. > >>>> (Riham) > >>>> Subject: Nieuwe releases en mps-backend > >>>> > >>>> > >>>> Hoi, > >>>> > >>>> Sorry dat ik niet bij de Skype-sessie van vandaag kon zijn, maar Riham > >>>> vertelde me dat het onderwerp van nieuwe releases en de mps-backend ter > >>>> sprake is geweest. Het riep wat specifieke vragen bij me op die ze niet > >>>> allemaal kon beantwoorden: > >>>> > >>>> a.. Is het de bedoeling dat een nieuwe release de mps-backend > >>>> helemaal > >>>> niet meer bevat of alleen maar de algebra-backend als default heeft? > >>>> b.. Als jullie praten over "nieuwe release", bedoel je dan alleen een > >>>> package van alle modules van MonetDB/XQuery? Maw zegt het niets over > >>>> eventuele `subreleases' van individuele modules? > >>>> > >>>> c.. Op de wiki hebben we een pagina met open issues voor de > >>>> algebra-release. Wat is de status van die open issues? Ik zie namelijk > >>>> op de > >>>> pagina nergens dat bepaalde issues al opgelost/geïmplementeerd zijn. > >>>> Ik Cc deze e-mail even aan de PF/Tijah-mensen, want hoewel er > >>>> levendige > >>>> discussies zijn hoe PF/Tijah naar de algebra-backend to porten, ik heb > >>>> niet > >>>> het idee dat er oplossingen zijn die op korte termijn realiseerbaar zijn. > >>>> Evenzo het recursie-vraagstuk. Het lijkt me bijvoorbeeld voor de > >>>> zichtbaarheid naar buiten toe onwenselijk als deze features ineens zouden > >>>> wegvallen voor non-insiders. Of andere onwenselijke consequenties van het > >>>> niet meer (sub)releasen van modules die gebaseerd zijn op de mps-backend. > >>>> > >>>> Groetjes, > >>>> Maurice. > >>>> > >>>> -- > >>>> ---------------------------------------------------------------------- > >>>> Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology > >>>> Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands > >>>> Email: [EMAIL PROTECTED], Phone: +31 534893688, Fax: +31 534892927 > >>>> Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen > >>>> > >>>> > >>> > >>> > >> ------------------------------------------------------------------------- > >> This SF.net email is sponsored by: Splunk Inc. > >> Still grepping through log files to find problems? Stop. > >> Now Search log events and configuration files using AJAX and a browser. > >> Download your FREE copy of Splunk now >> http://get.splunk.com/ > >> _______________________________________________ > >> Monetdb-developers mailing list > >> [email protected] > >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers > >> > >> ------------------------------------------------------------------------ > >> > >> ------------------------------------------------------------------------- > >> This SF.net email is sponsored by: Splunk Inc. > >> Still grepping through log files to find problems? Stop. > >> Now Search log events and configuration files using AJAX and a browser. > >> Download your FREE copy of Splunk now >> http://get.splunk.com/ > >> ------------------------------------------------------------------------ > >> > >> _______________________________________________ > >> Monetdb-developers mailing list > >> [email protected] > >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers > >> > > -- > ---------------------------------------------------------------------- > Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology > Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands > Email: [EMAIL PROTECTED], Phone: +31 534893688, Fax: +31 534892927 > Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-developers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 | ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Monetdb-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-developers
