Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
On 03/11/2015 04:24 PM, Steffen Hoffmann wrote: Hm, I'm running past 10k tickets with a lot more than 5 users on SQLite by now, and this is not even latest Trac stable but plugins mostly still using the depreciated Trac 0.11 db API. I think *concurrent* user requests are the issue, not total number of registered users, and ticket count might be not much of an issue in general. Yeah, and with 5 people hammering on it pretty much constantly, we'd get table lock timeouts. Now we've got about 10 heavy users and 15000 tickets, and it's quite nice. -- Matthew Caron, Software Build Engineer Red Lion Controls | www.redlion.net +1 (518) 877-5173 x138 office -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11.03.2015 18:30, tolu...@gmail.com wrote: >> On 03/11/2015 01:24 AM, tol...@gmail.com wrote: >> - There are about 1000 tickets, and about 50 users. > > Eeep... we couldn't get past 5 users on SQLite. SQLite has very coarse > locking, so it would lock whole tables and users would get errors. We > switched to PostgreSQL (not my favorite, but MySQL support was pretty > new at the time) and things got loads better. Hm, I'm running past 10k tickets with a lot more than 5 users on SQLite by now, and this is not even latest Trac stable but plugins mostly still using the depreciated Trac 0.11 db API. I think *concurrent* user requests are the issue, not total number of registered users, and ticket count might be not much of an issue in general. Still I'm approaching SQLite to PostgreSQL migration myself. While I foresee quite a bit more procession overhead for the PostgreSQL backend, it will certainly help with the locking issues, that are nasty with SQLite indeed. Steffen Hoffmann -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlUApJQACgkQ31DJeiZFuHdppgCfZpC79ubGfztrpfGwOds347KM g0gAoLde72klU9pOp2a9xd9AXxXUiuSf =oJ4e -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11.03.2015 20:27, tolu...@gmail.com wrote: > Ok. Here is the debug log output for three pages that took ~10 > seconds to load each. This is on the faster side of loading times. > As you can see, trac has to restart itself everytime it loads a > new page. TracAccountManager-0.4.3 could be upgraded, but it shouldn’t make any difference for your situation. While Trac env reload 'only' takes approx. 5 s of the total request processing time, this is not only much too slow these days, but the cause for the reload should be determined in the first place. This is certainly no near normal Trac behaviour. Did you already try to debug by deactivating plugins one-by-one to find the culprit? Steffen Hoffmann -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlUAoVcACgkQ31DJeiZFuHeNOACfUHyglH645g5h8uPIf7pBWUtu HPcAoKxtrZfjwcQt3ZYaxVWUrjCBLM3g =LmnU -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
Ok. Here is the debug log output for three pages that took ~10 seconds to load each. This is on the faster side of loading times. As you can see, trac has to restart itself everytime it loads a new page. The three pages that I navigated to are, /reports, /admin, and /wiki/test_Tomas - a sandbox page I create wiki pages with. Thanks again. Tomas On Wednesday, March 11, 2015 at 11:13:22 AM UTC-7, RjOllos wrote: > > On Wed, Mar 11, 2015 at 10:56 AM, > wrote: > >> Ok, I think the plugin was installed when my predecessor tried to perform >> the migration. I can try to go through it again. >> >> Any idea why Bitten is slowing things down a lot? I know we have a patch >> that was applied to add some desired functionality to Bitten, like >> keep-alive communication, network timeout retries, etc. >> >> I've attached our patch for reference. >> >> Thanks!! >> >> Tomas >> > > I think there are some Bitten users on this list, but you may have more > success browsing the Bitten mailing list and posting questions about your > issue there. > http://bitten.edgewall.org/wiki/MailingList > > Regarding your earlier statement that it takes "~30 seconds to load a wiki > page", how long does it take after disabling Bitten? It would be useful to > see the DEBUG level log output that results from a single slow-loading > page. The output should be just a dozen or two dozen lines, so you could > manually sanitize it to remove any information you don't want to expose. > > - Ryan > > -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout. trac_debug.log Description: Binary data
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
On Wed, Mar 11, 2015 at 10:56 AM, wrote: > Ok, I think the plugin was installed when my predecessor tried to perform > the migration. I can try to go through it again. > > Any idea why Bitten is slowing things down a lot? I know we have a patch > that was applied to add some desired functionality to Bitten, like > keep-alive communication, network timeout retries, etc. > > I've attached our patch for reference. > > Thanks!! > > Tomas > I think there are some Bitten users on this list, but you may have more success browsing the Bitten mailing list and posting questions about your issue there. http://bitten.edgewall.org/wiki/MailingList Regarding your earlier statement that it takes "~30 seconds to load a wiki page", how long does it take after disabling Bitten? It would be useful to see the DEBUG level log output that results from a single slow-loading page. The output should be just a dozen or two dozen lines, so you could manually sanitize it to remove any information you don't want to expose. - Ryan -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
Ok, I think the plugin was installed when my predecessor tried to perform
the migration. I can try to go through it again.
Any idea why Bitten is slowing things down a lot? I know we have a patch
that was applied to add some desired functionality to Bitten, like
keep-alive communication, network timeout retries, etc.
I've attached our patch for reference.
Thanks!!
Tomas
On Wednesday, March 11, 2015 at 10:35:02 AM UTC-7, RjOllos wrote:
>
> On Wed, Mar 11, 2015 at 10:30 AM, > wrote:
>
>> I'm strongly considering switching to PostgreSQL. Is it a straightforward
>> transition?
>> Can you point me to a link or describe how you performed the switch?
>>
>> A lot of my users are not here anymore. Would it help to delete their
>> user accounts?
>> Would that affect any tickets they're associated with?
>>
>> Thanks!
>>
>
> The TracMigratePlugin, which you show is already installed in your plugin
> list, will help with the migration. I suggest going with PostgreSQL since
> there are fewer caveats involved with how your configure the database. If
> you decide to use MySQL, I suggest upgrading to Trac 1.0.4 first.
>
> http://trac-hacks.org/wiki/TracMigratePlugin
>
> I don't think it will help to delete users in this case. The issues with
> SQLite come down to how much load there is on your system.
>
>
--
You received this message because you are subscribed to the Google Groups "Trac
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to trac-users+unsubscr...@googlegroups.com.
To post to this group, send email to trac-users@googlegroups.com.
Visit this group at http://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.
Index: bitten/admin.py
===
--- bitten/admin.py (revision 1021)
+++ bitten/admin.py (working copy)
@@ -55,6 +55,12 @@
build_all and 'yes' or 'no')
changed = True
+build_order = 'build_order' in req.args
+if build_order != master.build_order:
+self.config['bitten'].set('build_order',
+ build_order and 'yes' or 'no')
+changed = True
+
adjust_timestamps = 'adjust_timestamps' in req.args
if adjust_timestamps != master.adjust_timestamps:
self.config['bitten'].set('adjust_timestamps',
Index: bitten/master.py
===
--- bitten/master.py (revision 1021)
+++ bitten/master.py (working copy)
@@ -56,6 +56,10 @@
"""Whether to request builds of older revisions even if a younger
revision has already been built.""")
+build_order = BoolOption('bitten', 'build_order', False, doc=
+"""Whether to build older revisions first even if a younger revision
+is available.""")
+
stabilize_wait = IntOption('bitten', 'stabilize_wait', 0, doc=
"""The time in seconds to wait for the repository to stabilize before
queuing up a new build. This allows time for developers to check in
@@ -161,6 +165,7 @@
def _process_build_creation(self, req, slave_token):
queue = BuildQueue(self.env, build_all=self.build_all,
+ build_order=self.build_order,
stabilize_wait=self.stabilize_wait,
timeout=self.slave_timeout)
try:
Index: bitten/model.py
===
--- bitten/model.py (revision 1021)
+++ bitten/model.py (working copy)
@@ -546,7 +546,8 @@
fetch = classmethod(fetch)
def select(cls, env, config=None, rev=None, platform=None, slave=None,
- status=None, db=None, min_rev_time=None, max_rev_time=None):
+ status=None, db=None, min_rev_time=None, max_rev_time=None,
+ order=False):
"""Retrieve existing builds from the database that match the specified
criteria.
"""
@@ -573,10 +574,17 @@
else:
where = ""
+# The keyword DESC reverts to descending order
cursor = db.cursor()
-cursor.execute("SELECT id FROM bitten_build %s "
+if order:
+cursor.execute("SELECT id FROM bitten_build %s "
+ "ORDER BY rev_time,config,slave"
+ % where, [wc[1] for wc in where_clauses])
+else:
+cursor.execute("SELECT id FROM bitten_build %s "
"ORDER BY rev_time DESC,config,slave"
% where, [wc[1] for wc in where_clauses])
+
for (id,) in cursor:
yield Build.fetch(env, id)
select = classmethod(select)
Index: bitten/queue.py
===
--- bitten/queue.py (revision 1021)
+++ bitten/queue.py (working copy)
@@ -97,11 +97,12 @@
repository
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
On Wed, Mar 11, 2015 at 10:30 AM, wrote: > I'm strongly considering switching to PostgreSQL. Is it a straightforward > transition? > Can you point me to a link or describe how you performed the switch? > > A lot of my users are not here anymore. Would it help to delete their user > accounts? > Would that affect any tickets they're associated with? > > Thanks! > The TracMigratePlugin, which you show is already installed in your plugin list, will help with the migration. I suggest going with PostgreSQL since there are fewer caveats involved with how your configure the database. If you decide to use MySQL, I suggest upgrading to Trac 1.0.4 first. http://trac-hacks.org/wiki/TracMigratePlugin I don't think it will help to delete users in this case. The issues with SQLite come down to how much load there is on your system. -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
I'm strongly considering switching to PostgreSQL. Is it a straightforward transition? Can you point me to a link or describe how you performed the switch? A lot of my users are not here anymore. Would it help to delete their user accounts? Would that affect any tickets they're associated with? Thanks! On Wednesday, March 11, 2015 at 6:34:48 AM UTC-7, matt.caron wrote: > > > On 03/11/2015 01:24 AM, tol...@gmail.com wrote: > > - I am running an Apache2 web server. I believe the site is running as a > > fast cgi script. Does that also hamper performance? > > Yes. I suggest using WSGI. http://trac.edgewall.org/wiki/TracModWSGI > > > - There are about 1000 tickets, and about 50 users. > > Eeep... we couldn't get past 5 users on SQLite. SQLite has very coarse > locking, so it would lock whole tables and users would get errors. We > switched to PostgreSQL (not my favorite, but MySQL support was pretty > new at the time) and things got loads better. > > -- > Matthew Caron, Software Build Engineer > Red Lion Controls | www.redlion.net > +1 (518) 877-5173 x138 office > -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Trac 1.0 + Bitten + mySql Lite Support
On 03/11/2015 01:24 AM, tolu...@gmail.com wrote: - I am running an Apache2 web server. I believe the site is running as a fast cgi script. Does that also hamper performance? Yes. I suggest using WSGI. http://trac.edgewall.org/wiki/TracModWSGI - There are about 1000 tickets, and about 50 users. Eeep... we couldn't get past 5 users on SQLite. SQLite has very coarse locking, so it would lock whole tables and users would get errors. We switched to PostgreSQL (not my favorite, but MySQL support was pretty new at the time) and things got loads better. -- Matthew Caron, Software Build Engineer Red Lion Controls | www.redlion.net +1 (518) 877-5173 x138 office -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
