On Wed, Jun 04, 2008 at 10:18:53AM +0200, Stefan Manegold wrote:
> LS,
>
> a possible "HACK" to cure the sympthoms (not necessarily to heal the actual
> problem) would be to add a bit/boolean parameter to logger_del_bat() to
> control the use of BBPdecref() in logger_del_bat() from outside.
Thats not correct. The query isn't committed yet, ie you still need to
be able to abort (aren't transactions lovely ;-).
Niels
>
> ========
> Index: MonetDB/src/gdk/gdk_logger.mx
> ===================================================================
> RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_logger.mx,v
> retrieving revision 1.17.2.1
> diff -u -r1.17.2.1 gdk_logger.mx
> --- MonetDB/src/gdk/gdk_logger.mx 27 May 2008 20:33:39 -0000 1.17.2.1
> +++ MonetDB/src/gdk/gdk_logger.mx 4 Jun 2008 08:14:05 -0000
> @@ -158,7 +158,7 @@
> gdk_export int log_sequence(logger *lg, int seq, lng id);
>
> gdk_export log_bid logger_add_bat(logger *lg, BAT *b, char *name);
> -gdk_export void logger_del_bat(logger *lg, log_bid bid);
> +gdk_export void logger_del_bat(logger *lg, log_bid bid, bit decref);
> gdk_export log_bid logger_find_bat(logger *lg, char *name);
>
> #endif /*_LOGGER_H_*/
> @@ -517,7 +517,7 @@
> log_bid bid = logger_find_bat(lg, la->name);
>
> if (bid)
> - logger_del_bat(lg, bid);
> + logger_del_bat(lg, bid, FALSE);
> }
>
> static int
> @@ -1719,8 +1719,10 @@
> }
>
> void
> -logger_del_bat(logger *lg, log_bid bid)
> +logger_del_bat(logger *lg, log_bid bid, bit decref)
> {
> + if (decref == TRUE)
> + BBPdecref(bid, TRUE);
> BUNdelHead(lg->catalog, &bid, FALSE);
> }
>
> ========
>
> SQL would call loger_del_bat with FALSE to maintain the current behavior,
> while pathfinder would call it with TRUE to enforce the BBPdecref().
>
>
> Stefan
>
>
> On Wed, Jun 04, 2008 at 07:58:50AM +0200, Stefan Manegold wrote:
> > Gents,
> >
> > with my logger patch, the 8 tests fail at server start-up with multiple
> > "
> > !ERROR: BBPdecref: tmp_XXX does not have logical references.
> > "
> >
> > Apparently, with SQL the extra ref-count is taken care of elsewhere!??
> >
> > Hence, we need to look in more detail into the logger and its use in both
> > SQL & XQuery ...
> >
> > Any ideas?
> >
> > Stefan
> >
> >
> > On Wed, Jun 04, 2008 at 07:32:55AM +0200, Stefan Manegold wrote:
> > > Gents,
> > >
> > > while working fine for pathfinder, my patch seems to harm some SQL tests:
> > >
> > > ========
> > > @@ -722,2 +722,2 @@
> > > -03:19:02> src/test/marcin1.sql (<=60,60,180) ... 0.111s
> > > -marcin1.stable.out.FILTERED and marcin1.test.out.FILTERED are equal.
> > > +01:52:39> src/test/marcin1.sql (<=60,60,180) ... 0.136s
> > > +marcin1.stable.out.FILTERED and marcin1.test.out.FILTERED differ
> > > SIGNIFICANTLY!
> > > @@ -1295,2 +1295,2 @@
> > > -03:23:31> src/test/BugTracker/case_in_aggr_bug.SF-1506545.sql
> > > (<=60,60,180) ... 0.156s
> > > -case_in_aggr_bug.SF-1506545.stable.out.FILTERED and
> > > case_in_aggr_bug.SF-1506545.test.out.FILTERED are equal.
> > > +01:57:09> src/test/BugTracker/case_in_aggr_bug.SF-1506545.sql
> > > (<=60,60,180) ... 0.160s
> > > +case_in_aggr_bug.SF-1506545.stable.out.FILTERED and
> > > case_in_aggr_bug.SF-1506545.test.out.FILTERED differ SIGNIFICANTLY!
> > > @@ -1313,2 +1313,2 @@
> > > -03:23:32> src/test/BugTracker/groupby_with_alias.SF-1520575.sql
> > > (<=60,60,180) ... 0.105s
> > > -groupby_with_alias.SF-1520575.stable.out.FILTERED and
> > > groupby_with_alias.SF-1520575.test.out.FILTERED are equal.
> > > +01:57:10> src/test/BugTracker/groupby_with_alias.SF-1520575.sql
> > > (<=60,60,180) ... 0.118s
> > > +groupby_with_alias.SF-1520575.stable.out.FILTERED and
> > > groupby_with_alias.SF-1520575.test.out.FILTERED differ SIGNIFICANTLY!
> > > @@ -1400,2 +1400,2 @@
> > > -03:23:37> src/test/BugTracker/view_creation_using_func.SF-1714814.sql
> > > (<=60,60,180) ... 0.115s
> > > -view_creation_using_func.SF-1714814.stable.out.FILTERED and
> > > view_creation_using_func.SF-1714814.test.out.FILTERED are equal.
> > > +01:57:15> src/test/BugTracker/view_creation_using_func.SF-1714814.sql
> > > (<=60,60,180) ... 0.115s
> > > +view_creation_using_func.SF-1714814.stable.out.FILTERED and
> > > view_creation_using_func.SF-1714814.test.out.FILTERED differ
> > > SIGNIFICANTLY!
> > > @@ -1472,2 +1472,2 @@
> > > -03:23:41> src/test/BugTracker/number_null_first_row.SF-1833286.sql
> > > (<=60,60,180) ... 0.118s
> > > -number_null_first_row.SF-1833286.stable.out.FILTERED and
> > > number_null_first_row.SF-1833286.test.out.FILTERED are equal.
> > > +01:57:20> src/test/BugTracker/number_null_first_row.SF-1833286.sql
> > > (<=60,60,180) ... 0.130s
> > > +number_null_first_row.SF-1833286.stable.out.FILTERED and
> > > number_null_first_row.SF-1833286.test.out.FILTERED differ SIGNIFICANTLY!
> > > @@ -2063,2 +2063,2 @@
> > > -03:32:00> src/test/leaks/temp2.sql (<=60,60,180) ... 0.105s
> > > -temp2.stable.out.Five.FILTERED and temp2.test.out.FILTERED are equal.
> > > +02:05:39> src/test/leaks/temp2.sql (<=60,60,180) ... 0.105s
> > > +temp2.stable.out.Five.FILTERED and temp2.test.out.FILTERED differ
> > > SIGNIFICANTLY!
> > > @@ -2066,2 +2066,2 @@
> > > -03:32:00> src/test/leaks/check3.sql (<=60,60,180) ... 0.104s
> > > -check3.stable.out.Five.FILTERED and check3.test.out.FILTERED are equal.
> > > +02:05:39> src/test/leaks/check3.sql (<=60,60,180) ... 0.105s
> > > +check3.stable.out.Five.FILTERED and check3.test.out.FILTERED differ
> > > SIGNIFICANTLY!
> > > @@ -2075,2 +2075,2 @@
> > > -03:32:01> src/test/leaks/check4.sql (<=60,60,180) ... 0.104s
> > > -check4.stable.out.Five.FILTERED and check4.test.out.FILTERED are equal.
> > > +02:05:39> src/test/leaks/check4.sql (<=60,60,180) ... 0.105s
> > > +check4.stable.out.Five.FILTERED and check4.test.out.FILTERED differ
> > > SIGNIFICANTLY!
> > > @@ -2165 +2165 @@
> > > - 23 out of 650 tests produced SIGNIFICANTLY different output
> > > + 31 out of 650 tests produced SIGNIFICANTLY different output
> > > @@ -2175,0 +2176,417 @@
> > > ========
> > >
> > > I still need to check wha the actual cause/problem is.
> > >
> > > Stefan
> > >
> > >
> > > On Wed, Jun 04, 2008 at 01:32:42AM +0200, Stefan Manegold wrote:
> > > > Niels, Sjoerd, Peter,
> > > >
> > > > after (quite) some time of debugging for
> > > > [ 1976341 ] XQ: leftovers after deleting document
> > > > http://sourceforge.net/tracker/index.php?func=detail&aid=1976341&group_id=56967&atid=482468
> > > >
> > > > I found the --- or at least one possible --- cause for this bug.
> > > >
> > > > In pathfinder, newly shredded read-only documents are added to the
> > > > logger
> > > > using logger_add_bat() --- let's ignore for the time being, that I am a
> > > > bit
> > > > puzzled by this, since I though / was told, pathfinder uses
> > > > check-pointing
> > > > instead of the logger (WAL) for shredding documents.
> > > >
> > > > Obviously, logger_add_bat() increments the ref-count of the logged BATs
> > > > (BBPincref()).
> > > >
> > > > However, logger_del_bat() does not seem to decrements the ref-count,
> > > > again,
> > > > once a BAT is removed from the logger.
> > > > This way, document BATs keep the logger_add_bat()-increased ref-count
> > > > even
> > > > after a pf:del-doc() and stay around until the Mserver is shutdown.
> > > >
> > > > The below patch seem to "fix" this problem.
> > > > However, since I'm everything but familiar with the logger or the usage
> > > > of it
> > > > in either XQuery or SQL, I'd rather have you double-check the patch
> > > > before
> > > > I'd check it in.
> > > > (Of course I'll also run "private" testing of the whole suite overnight
> > > > to
> > > > see whether my patch causes any (obvious) harm.
> > > >
> > > > Thank you very much in advance!
> > > >
> > > > Stefan
> > > >
> > > > ========
> > > > Index: MonetDB/src/gdk/gdk_logger.mx
> > > > ===================================================================
> > > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_logger.mx,v
> > > > retrieving revision 1.17.2.1
> > > > diff -u -r1.17.2.1 gdk_logger.mx
> > > > --- MonetDB/src/gdk/gdk_logger.mx 27 May 2008 20:33:39 -0000
> > > > 1.17.2.1
> > > > +++ MonetDB/src/gdk/gdk_logger.mx 3 Jun 2008 23:27:15 -0000
> > > > @@ -1721,6 +1721,7 @@
> > > > void
> > > > logger_del_bat(logger *lg, log_bid bid)
> > > > {
> > > > + BBPdecref(bid, TRUE);
> > > > BUNdelHead(lg->catalog, &bid, FALSE);
> > > > }
> > > >
> > > > ========
> > > >
> > > > --
> > > > | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] |
> > > > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
> > > > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
> > > > | The Netherlands | Fax : +31 (20) 592-4312 |
> > > >
> > > > -------------------------------------------------------------------------
> > > > Check out the new SourceForge.net Marketplace.
> > > > It's the best place to buy or sell services for
> > > > just about anything Open Source.
> > > > http://sourceforge.net/services/buy/index.php
> > > > _______________________________________________
> > > > Monetdb-developers mailing list
> > > > [email protected]
> > > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers
> > >
> > > --
> > > | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] |
> > > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
> > > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
> > > | The Netherlands | Fax : +31 (20) 592-4312 |
> > >
> > > -------------------------------------------------------------------------
> > > Check out the new SourceForge.net Marketplace.
> > > It's the best place to buy or sell services for
> > > just about anything Open Source.
> > > http://sourceforge.net/services/buy/index.php
> > > _______________________________________________
> > > Monetdb-developers mailing list
> > > [email protected]
> > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers
> >
> > --
> > | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] |
> > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
> > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
> > | The Netherlands | Fax : +31 (20) 592-4312 |
> >
> > -------------------------------------------------------------------------
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://sourceforge.net/services/buy/index.php
> > _______________________________________________
> > Monetdb-developers mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/monetdb-developers
>
> --
> | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] |
> | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
> | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
> | The Netherlands | Fax : +31 (20) 592-4312 |
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> Monetdb-developers mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
--
Niels Nes, Centre for Mathematics and Computer Science (CWI)
Kruislaan 413, 1098 SJ Amsterdam, The Netherlands
room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312
url: http://www.cwi.nl/~niels e-mail: [EMAIL PROTECTED]
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Monetdb-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
