Re: [rt-users] Automatically Deleting Ticket Attachments Matching Name

[Nathan Baker]

Have you tried something like this:

rt-shredder --plugin 'Attachments=limit,1000;files_only,1;file,smime.p7s'

That should delete only attachments with the name you mentioned, and will
limit it to 1000 attachments which you can adjust.

On Tue, Jul 28, 2015 at 8:52 AM, Zoey Schutt  wrote:

> Hello,
>
>
>
> RT Instance Info:
>
>
>
> Debian GNU/Linux 7 (wheezy)
>
> Apache/2.2.22 (Debian)
>
> Request Tracker 4.2.11
>
> OpenSSL 1.0.1e 11 Feb 2013
>
> perl 5, version 14, subversion 2 (v5.14.2) built for
> x86_64-linux-gnu-thread-multi
>
>
> --
>
>
>
> I have been searching for a way to do this, but I haven’t been able to
> find a good way to get this working. I’m trying to figure out a way to get
> Request Tracker to automatically delete ticket attachments with a specific
> name, in this case “smime.p7s”. Essentially all outgoing emails in my
> Request Tracker instance are signed, and every email Request Tracker sends
> attaches a “smime.p7s” file to the ticket. I just need a way to get these
> attachments deleted automatically, as they are not needed.
>
>
>
> I’ve attached a screenshot of the attachments pane of a ticket that’s been
> worked on for a few days as an example.
>
>
>
> I was thinking a cron job running rt-shredder might work, but I can’t
> figure out a way to delete attachments by name, only via transaction ID.
>
>
>
> Any assistance would be appreciated!
>
>
>
> Regards,
>
>
>
> Zoey Schutt
>

Re: [rt-users] Upgrade History shows Upgrade Incomplete

[Nathan Baker]

>
> On Thu, May 22, 2014 at 04:33:01PM -0400, Kevin Falcone wrote:
> > oh.
> > You cannot safely upgrade RT like that.
> >
> > Restore from a backup and upgrade cleanly.
> >
> > I wouldn't trust a database that had run the upgrades twice.
>

You're right, that would be the safest.  I've been running it in production
for 1-2 days now though, with no errors or noticeable issues, so I think
I'm just going to leave it.  Thanks to the new "RT upgrade history", I can
see which steps attempted to run twice before it failed the second time.  I
looked at all of those changes, and I don't see anything in there that
would cause any problems with the data or schema.  The steps that ran a
second time are:

Insert from /usr/share/request-tracker4/etc/upgrade/4.0.9/content
Insert from /usr/share/request-tracker4/etc/upgrade/4.0.18/content
Insert from /usr/share/request-tracker4/etc/upgrade/4.0.19/content
Insert from /usr/share/request-tracker4/etc/upgrade/4.1.0/content

So I think I got lucky that it failed right after that before getting any
further :)

On Fri, May 23, 2014 at 6:13 AM, Dominic Hargreaves <
dominic.hargrea...@it.ox.ac.uk> wrote:

> Ah, I had always assumed that updates were idempotent. Sounds like
> we need to adjust the error handling in the Debian package then.
> (What I think happened in Nathan's case was that the upgrade itself
> went okay but something else went wrong in the package postinst).


The first thing that failed during the upgrade process was the database
backup.  Peer authentication for user "postgres" failed.  I'm not sure if
it ever worked on this system, but RT was installed originally using Debian
packages.  I had backed up the database manually before starting just to be
safe.  Running "passwd postgres" and setting a password for that user got
peer authentication working.

During the database upgrade part, I'm not sure why it was failing to
perform the upgrade.  Finally after I selected "retry" and then answered
"No" when asked to use dbconfig to upgrade the database (I was going to do
it manually at that point), it actually started performing the database
upgrade, but from what I can tell it tried to do it twice.

I'm not ruling out that I did something wrong, but if you have anything you
want me to check on this system let me know.  I have the apt term.log yet
which shows what was done, but unfortunately won't show what answers I
chose during the process.  When the RT 4.2.4 package gets to testing I will
try that one and let you know if I have any issues.

Thanks!
Nate
-- 
RT Training - Boston, September 9-10
http://bestpractical.com/training

Re: [rt-users] Upgrade History shows Upgrade Incomplete

[Nathan Baker]

After looking at all of the schema.Pg files after 4.1.1, it looks like all
of the changes have already been applied.  The subsequent changes in the
4.1.1 script have also been applied already (Disabled column was added,
Stage and Queue columns were removed), so I don't think it would work to
patch my 4.1.1 file and run the upgrade again.

>From what I can tell, the upgrade was successful the first time and failed
the second time, which is probably for the best anyway.  I'm not sure what
caused that, but I did notify Dominic about it.  I guess unless I notice
any problems, I'm just going to leave it and consider it to be normal/clean.

Thanks,
Nate


On Wed, May 21, 2014 at 4:23 PM, Nathan Baker  wrote:

>
> On Wed, May 21, 2014 at 1:22 PM, Kevin Falcone 
> wrote:
>
>> On Wed, May 21, 2014 at 11:33:20AM -0400, Nathan Baker wrote:
>> > Hello everyone,
>> >
>> > I just upgraded from 4.0.6 to 4.2.3 using the Debian packages.  During
>> the
>> > database upgrade, I received a few errors:
>>
>> I'm guessing you set up a new machine, installed request-tracker4 from
>> testing, restored your database and then did the upgrade?
>>
>
> Actually this is on a system that was running 4.0.6 previously.  I just
> did apt-get install request-tracker4 (using the testing repository) and it
> upgraded all the packages.
>
>
>> You have an unclean database with 4.2 tables in it, and you're
>> tripping over some of the code we added to help RT handle that more
>> gracefully.
>>
>
> What I'm wondering is how I can tell if the database is "unclean" or if
> it's okay.  The "upgrade history" section in System Configuration shows
> that it did "Upgrade from 4.0.19 to 4.2.3" once without errors, and then it
> did it again and it says "(Incomplete)".  Maybe that doesn't actually mean
> it tried twice though, I'm not sure.
>
>
>>
>> I've pushed
>>
>> https://github.com/bestpractical/rt/commit/0ac64855f479b4133f4fe838af6b8b2dec8f5d18.patch
>> and it should be in 4.2.5.  You may be able to apply the patch
>> manually to /usr/share/request-tracker4/etc/upgrade/4.1.1/schema.Pg
>> or just clean out your DB by hand before letting the debian upgrade
>> scripts go.
>>
>
> Can I safely just drop  the table, drop the sequence, create the sequence,
> and create the table?  I won't lose anything critical by doing that?
>
> I'm guessing there's a bunch of stuff it was supposed to do after that,
> but it probably stopped at that point.  I'm just thrown off by it being in
> the "upgrade history" twice, the first time it looks like it did a whole
> bunch of stuff successfully, including this step and lot after this step.
>
> -kevin
>>
>> >  Processing 4.1.1
>> >
>> > Now populating database schema.
>> >
>> > [12453] [Wed May 21 03:19:36 2014] [critical]: DBD::Pg::st execute
>> failed:
>> > ERROR:  cannot drop sequence objectscrips_id_seq because other objects
>> > depend on it
>> >
>> > DETAIL:  default for table objectscrips column id depends on sequence
>> > objectscrips_id_seq
>> >
>> > HINT:  Use DROP ... CASCADE to drop the dependent objects too. at
>> > /usr/share/request-tracker4/lib/RT/Handle.pm line 529.
>> > (/usr/share/request-tracker4/lib/RT.pm:394)
>> >
>> > DBD::Pg::st execute failed: ERROR:  cannot drop sequence
>> > objectscrips_id_seq because other objects depend on it
>> >
>> > DETAIL:  default for table objectscrips column id depends on sequence
>> > objectscrips_id_seq
>> >
>> > HINT:  Use DROP ... CASCADE to drop the dependent objects too. at
>> > /usr/share/request-tracker4/lib/RT/Handle.pm line 529.
>> >
>> > error encountered processing
>> > /usr/share/dbconfig-common/scripts/request-tracker4/upgrade/pgsql/4.2.3:
>> >
>> > /usr/share/dbconfig-common/scripts/request-tracker4/upgrade/pgsql/4.2.3
>> > exited with non-zero status
>> >
>> >
>> > There could be an issue with the Debian packages (they are still in the
>> > testing branch), but I'm trying to figure out what state the database
>> is in
>> > now.  The system is working fine, and shows version 4.2.3.  If I
>> navigate
>> > to Admin -> Tools -> System Configuration, then in the "RT Upgrade
>> History"
>> > section it looks like it might have tried to upgrade the database twice,
>> > and the second time it failed (understandably)

Re: [rt-users] Upgrade History shows Upgrade Incomplete

[Nathan Baker]

On Wed, May 21, 2014 at 1:22 PM, Kevin Falcone wrote:

> On Wed, May 21, 2014 at 11:33:20AM -0400, Nathan Baker wrote:
> > Hello everyone,
> >
> > I just upgraded from 4.0.6 to 4.2.3 using the Debian packages.  During
> the
> > database upgrade, I received a few errors:
>
> I'm guessing you set up a new machine, installed request-tracker4 from
> testing, restored your database and then did the upgrade?
>

Actually this is on a system that was running 4.0.6 previously.  I just did
apt-get install request-tracker4 (using the testing repository) and it
upgraded all the packages.


> You have an unclean database with 4.2 tables in it, and you're
> tripping over some of the code we added to help RT handle that more
> gracefully.
>

What I'm wondering is how I can tell if the database is "unclean" or if
it's okay.  The "upgrade history" section in System Configuration shows
that it did "Upgrade from 4.0.19 to 4.2.3" once without errors, and then it
did it again and it says "(Incomplete)".  Maybe that doesn't actually mean
it tried twice though, I'm not sure.


>
> I've pushed
>
> https://github.com/bestpractical/rt/commit/0ac64855f479b4133f4fe838af6b8b2dec8f5d18.patch
> and it should be in 4.2.5.  You may be able to apply the patch
> manually to /usr/share/request-tracker4/etc/upgrade/4.1.1/schema.Pg
> or just clean out your DB by hand before letting the debian upgrade
> scripts go.
>

Can I safely just drop  the table, drop the sequence, create the sequence,
and create the table?  I won't lose anything critical by doing that?

I'm guessing there's a bunch of stuff it was supposed to do after that, but
it probably stopped at that point.  I'm just thrown off by it being in the
"upgrade history" twice, the first time it looks like it did a whole bunch
of stuff successfully, including this step and lot after this step.

-kevin
>
> >  Processing 4.1.1
> >
> > Now populating database schema.
> >
> > [12453] [Wed May 21 03:19:36 2014] [critical]: DBD::Pg::st execute
> failed:
> > ERROR:  cannot drop sequence objectscrips_id_seq because other objects
> > depend on it
> >
> > DETAIL:  default for table objectscrips column id depends on sequence
> > objectscrips_id_seq
> >
> > HINT:  Use DROP ... CASCADE to drop the dependent objects too. at
> > /usr/share/request-tracker4/lib/RT/Handle.pm line 529.
> > (/usr/share/request-tracker4/lib/RT.pm:394)
> >
> > DBD::Pg::st execute failed: ERROR:  cannot drop sequence
> > objectscrips_id_seq because other objects depend on it
> >
> > DETAIL:  default for table objectscrips column id depends on sequence
> > objectscrips_id_seq
> >
> > HINT:  Use DROP ... CASCADE to drop the dependent objects too. at
> > /usr/share/request-tracker4/lib/RT/Handle.pm line 529.
> >
> > error encountered processing
> > /usr/share/dbconfig-common/scripts/request-tracker4/upgrade/pgsql/4.2.3:
> >
> > /usr/share/dbconfig-common/scripts/request-tracker4/upgrade/pgsql/4.2.3
> > exited with non-zero status
> >
> >
> > There could be an issue with the Debian packages (they are still in the
> > testing branch), but I'm trying to figure out what state the database is
> in
> > now.  The system is working fine, and shows version 4.2.3.  If I navigate
> > to Admin -> Tools -> System Configuration, then in the "RT Upgrade
> History"
> > section it looks like it might have tried to upgrade the database twice,
> > and the second time it failed (understandably).
> >
> >
> > Is there a way I can verify that the database schema is completely up to
> > date?
> >
> >
> > Here is what the "RT Upgrade History" section shows: (Sorry for the HTML,
> > wasn't sure how else to show it)
> > RT (Current version: 4.2.3) ActionDateElapsedRT VersionInsert from
> > /usr/share/request-tracker4/etc/upgrade/4.0.9/contentTue May 20 23:17:45
> > 20141 second4.2.3Insert from
> > /usr/share/request-tracker4/etc/upgrade/4.0.18/contentTue May 20 23:17:53
> > 20141 second4.2.3Insert from
> > /usr/share/request-tracker4/etc/upgrade/4.0.19/contentTue May 20 23:17:55
> > 20140 seconds4.2.3 <http://rt/Admin/Tools/Configuration.html#>Upgrade
> from
> > 4.0.19 to 4.2.3Tue May 20 23:17:56 20142 minutes4.2.3Insert from
> > /usr/share/request-tracker4/etc/upgrade/4.0.9/contentTue May 20 23:19:29
> > 20140 seconds4.2.3Insert from
> > /usr/share/request-tracker4/etc/upgrade/4.0.18/contentTue May 20 23:19:34
> > 20140 seconds4.2.3Insert from
> > /usr/share/requ

[rt-users] Upgrade History shows Upgrade Incomplete

[Nathan Baker]

Hello everyone,

I just upgraded from 4.0.6 to 4.2.3 using the Debian packages.  During the
database upgrade, I received a few errors:

 Processing 4.1.1

Now populating database schema.

[12453] [Wed May 21 03:19:36 2014] [critical]: DBD::Pg::st execute failed:
ERROR:  cannot drop sequence objectscrips_id_seq because other objects
depend on it

DETAIL:  default for table objectscrips column id depends on sequence
objectscrips_id_seq

HINT:  Use DROP ... CASCADE to drop the dependent objects too. at
/usr/share/request-tracker4/lib/RT/Handle.pm line 529.
(/usr/share/request-tracker4/lib/RT.pm:394)

DBD::Pg::st execute failed: ERROR:  cannot drop sequence
objectscrips_id_seq because other objects depend on it

DETAIL:  default for table objectscrips column id depends on sequence
objectscrips_id_seq

HINT:  Use DROP ... CASCADE to drop the dependent objects too. at
/usr/share/request-tracker4/lib/RT/Handle.pm line 529.

error encountered processing
/usr/share/dbconfig-common/scripts/request-tracker4/upgrade/pgsql/4.2.3:

/usr/share/dbconfig-common/scripts/request-tracker4/upgrade/pgsql/4.2.3
exited with non-zero status


There could be an issue with the Debian packages (they are still in the
testing branch), but I'm trying to figure out what state the database is in
now.  The system is working fine, and shows version 4.2.3.  If I navigate
to Admin -> Tools -> System Configuration, then in the "RT Upgrade History"
section it looks like it might have tried to upgrade the database twice,
and the second time it failed (understandably).


Is there a way I can verify that the database schema is completely up to
date?


Here is what the "RT Upgrade History" section shows: (Sorry for the HTML,
wasn't sure how else to show it)
RT (Current version: 4.2.3) ActionDateElapsedRT VersionInsert from
/usr/share/request-tracker4/etc/upgrade/4.0.9/contentTue May 20 23:17:45
20141 second4.2.3Insert from
/usr/share/request-tracker4/etc/upgrade/4.0.18/contentTue May 20 23:17:53
20141 second4.2.3Insert from
/usr/share/request-tracker4/etc/upgrade/4.0.19/contentTue May 20 23:17:55
20140 seconds4.2.3 Upgrade from
4.0.19 to 4.2.3Tue May 20 23:17:56 20142 minutes4.2.3Insert from
/usr/share/request-tracker4/etc/upgrade/4.0.9/contentTue May 20 23:19:29
20140 seconds4.2.3Insert from
/usr/share/request-tracker4/etc/upgrade/4.0.18/contentTue May 20 23:19:34
20140 seconds4.2.3Insert from
/usr/share/request-tracker4/etc/upgrade/4.0.19/contentTue May 20 23:19:35
20140 seconds4.2.3 Upgrade from
4.0.19 to 4.2.3 (Incomplete)Tue May 20 23:19:36 20144.2.3Upgrade from
4.0.19 to 4.1.0Tue May 20 23:19:36 20140 seconds4.2.3Insert from
/usr/share/request-tracker4/etc/upgrade/4.1.0/contentTue May 20 23:19:36
20140 seconds4.2.3Upgrade from 4.1.0 to 4.1.1 (Incomplete)Tue May 20
23:19:36 20144.2.3Schema updates from
/usr/share/request-tracker4/etc/upgrade/4.1.1 (Incomplete)Tue May 20
23:19:36 20144.2.3
Thanks,
Nate
-- 
RT Training - Boston, September 9-10
http://bestpractical.com/training

Re: [rt-users] Apache Threads hanging & not gracefully exiting

[Nathan Baker]

I switched from mod_perl to mod_fcgid and along with the memory usage
decreasing by about 75%, the problem seems to have disappeared.  I'm not
sure if there is a problem with the code and mod_fcgid is just handling it
better, or what the deal is, but everything is working fine now.

Judging by the user reviews of mod_perl (
http://cpanratings.perl.org/dist/mod_perl) it seems like mod_perl should be
the less preferred option, and mod_fastcgi or mod_fcgid should be used if
possible.  Is this the general consensus?  If so, it might be helpful to
add that recommendation on
http://bestpractical.com/docs/rt/latest/web_deployment.html.

Thanks for your suggestions though Kevin, if I do see any further issues
I'll try using strace.

For anyone else that comes across this, here are some apache mod_perl
documents about debugging mod_perl applications, using strace and other
methods:
http://perl.apache.org/docs/1.0/guide/debug.html
http://perl.apache.org/docs/2.0/user/troubleshooting/troubleshooting.html

-Nate



On Wed, Jan 15, 2014 at 11:21 AM, Kevin Falcone
wrote:

> On Tue, Jan 14, 2014 at 01:43:59PM -0500, Nathan Baker wrote:
> >(gracefully finishing). Those threads will never exit unless I kill
> the processes manually. My
> >guess would be that one of my customizations are causing this, but
> does anyone have any tips
> >for how to find out what the problem is?
>
> strace/dtruss?
>
> >- custom Scrip that uses Filesys::SmbClient to copy attachments to
> the user's computer when
> >they "Take" the ticket
>
> That sounds like the biggest suspect.
>
> -kevin
>

[rt-users] Apache Threads hanging & not gracefully exiting

[Nathan Baker]

Hello everyone,

I currently have an instance of RT 4.0.17 running that we're using for
processing incoming faxes.  The faxes come in as attachments, and I have
some customizations to allow our staff to process these attachments (PDFs)
faster.

Everything seems to work great for a while, and the memory usage gradually
increases, and eventually all of the Apache threads (using event mpm) are
stuck in the "W" state (sending reply).  If I try to gracefully restart
apache, all of those threads are stuck in the "G" state (gracefully
finishing). Those threads will never exit unless I kill the processes
manually.  My guess would be that one of my customizations are causing
this, but does anyone have any tips for how to find out what the problem is?

I tried this:
http://perl.apache.org/docs/general/control/control.html#All_RAM_Consumed
And all I get is some "No object mapping for field::Name" warnings when
people login (we are using the ExternalAuth module for Active Directory
integration).

I also came across these two things saying to try adding the
PERL_DESTRUCT_LEVEL environment variable to avoid the perl_destruct() call
before exiting, and that could let the threads exit.  Is this safe to try
with RT?

Here's a brief list of what was customized:
- ExternalAuth module using Active Directory
- modified Ticket/Elements/ShowAttachments (locally) to show PDF attachment
preview
-
using Callbacks/AddAttachmentsColumn/Elements/RT__Ticket/ColumnMap/Default
callback to add "Attachments" column to dashboard for quickly viewing
attachments without going into ticket
- custom Scrip that uses Filesys::SmbClient to copy attachments to the
user's computer when they "Take" the ticket

I would be happy to share any of the custom code, I just want to make sure
nothing is wrong with it before posting it to the Wiki.  Any tip on
troubleshooting this would be greatly appreciated!

Thanks,
Nate

Re: [rt-users] Resolve without Stealing

[Nathan Baker]

Tom,

We got rid of the "On Correspond Open Tickets" Scrip to avoid tickets being
reopened because of those type of responses.  Is that an option for you, or
would you like your tickets to re-open?

-Nate


On Fri, Jan 10, 2014 at 1:33 PM, Tom Leslein  wrote:

>
>
> I am trying to find a way to allow my Tier 1 team to resolve tickets
> without stealing tickets.
>
>
>
> At this point if someone works on a ticket for two days, or 1 hour, and
> completes the ticket, the client usually replies back with a thank you or
> “Yup, everything is good to go.”
>
>
>
> I do not want another Tier 1 person to have to Steal the ticket to resolve
> it.
>
>
>
> One caveat about this request is that I know there is a permission out
> there, “Edit Ticket”, but that gives them a lot more features which I do
> not want them to have.  Specifically, being able to reply to any ticket
> even if they are not the owner.
>
>
>
>
>
> Kind of long winded but hopefully someone can help.
>
>
>

Re: [rt-users] Change Subject When Resolving

[Nathan Baker]

That makes sense.  I might try to create a Scrip to update the ticket
subject, since it would be very convenient for this particular application.

Thanks!


On Thu, Jan 9, 2014 at 4:16 PM, Tim Wiley  wrote:

> On 01/09/2014 01:10 PM, Nathan Baker wrote:
>
>> Hello Everyone,
>>
>> We recently noticed that if you change the subject of a ticket while you
>> are resolving it, the change does not take effect.  I've noticed this
>> for RT 4.0.6 and 4.0.17, although I haven't been able to try it on 4.2.x
>> yet.  Can anyone confirm if this is a bug or if it's something
>> mis-configured?
>>
>> Thanks,
>> Nate
>>
>
> As far as I understand it, changing the subject line when resolving,
> commenting & replying will only set it for that transaction.  To change it
> permanently you'll want to change it via the ticket details.
>

[rt-users] Change Subject When Resolving

[Nathan Baker]

Hello Everyone,

We recently noticed that if you change the subject of a ticket while you
are resolving it, the change does not take effect.  I've noticed this for
RT 4.0.6 and 4.0.17, although I haven't been able to try it on 4.2.x yet.
 Can anyone confirm if this is a bug or if it's something mis-configured?

Thanks,
Nate

Re: [rt-users] RT 4 Upgrade Slow Performance - CustomFields?

[Nathan Baker]

Ruslan,

I agree with your recommendation in general for most installations,
especially ones larger than ours.  I don't think increasing the
KeepAliveTimeout is necessary anymore now that I fixed the swapping issue,
because the initial page load does not take a long time anymore.  However,
for our environment I know that there will never be more than 5 people
accessing the site at once, so we will never run into the scenario you were
giving as an example.  The reason I left the KeepAliveTimeout at 60 was
because I'd like to have it as fast as possible once a user starts doing
something, and I think a user could look at a ticket for more than 15
seconds (the default KeepAliveTimeout in my apache configuration) and then
continue on, but it's not as likely they would take longer than 60 seconds
in between clicks.  For our environment it would be fine to have 5 apache
processes dedicated for 5 users at once.

Thanks for the information on the lightweight front-end, it looks like it
would help a lot, although I think it might be overkill for our relatively
small installation.

-Nate

On Sat, Jun 2, 2012 at 6:00 PM, Ruslan Zakirov wrote:

> It shouldn't be necessary if you know how to fit things in.
>
> You don't want KeepAliveTimeout to be very high. Keep alive at 60
> seconds means that user when touched apache process holds it from
> serving other users for 60 seconds even if he doesn't do anything. 10
> users hit the server within a minute - you need 11 apache processes to
> serve next user. Your deployment is not configured for such values.
>
> For big keep alive values you need two step processing with light
> frontend and heavy backend. Frontend keep connections open and can
> hold many of them with low footprint. For example take a look at the
> following blog post:
> http://blog.webfaction.com/a-little-holiday-present, especially memory
> footprint chart.
>
> As the backend you either use FCGI server running RT, your current
> apache setup or something else.
>
> Take a look at the following extension:
>
>
> http://search.cpan.org/~ruz/RT-Extension-Nginx-0.02/lib/RT/Extension/Nginx.pm#FEATURES
>
> It generates config for nginx where a few features of the server and
> knowledge of RT are used to lower memory footprint, increase
> concurrency, lower page load times.
>
> --
> Best regards, Ruslan.
>

Re: [rt-users] RT 4 Upgrade Slow Performance - CustomFields?

[Nathan Baker]

Thanks Kenn, I checked and didn't see any permissions globally set for
everyone, except the Create Ticket right is set for Everyone on each of our
queues.

I made a few more changes though and am considering the problem fixed at
this point.  I found that the system was doing a lot of memory swapping,
even though I increased the memory from 512MB with RT 3.8.8 (and MySQL) to
2GB with 4.0.5 (and Postgresql).  I disabled all debugging and heavy
logging, and adjusted the Apache configuration to increase the
KeepAliveTimeout to 60 and reduce the MinSpareServers and MaxSpareServers.
 The apache processes were using between 60-100MB each (because of modperl
I think), so if you have 15 apache processes running that's potentially
1.5GB.  After making that change the system is "lightning fast" again.  I
still might add 1-2GB of memory just to be safe, I just didn't think that
much should be necessary.

I also have rt-clean-sessions running every night, which should help some.

Thank you everyone that helped!

Re: [rt-users] RT 4.0.5 in search don´t have content option

[Nathan Baker]

I did it recently.  There's some good instructions on this older thread:

http://www.gossamer-threads.com/lists/rt/users/103305

Specifically post #13

On Thu, May 31, 2012 at 5:49 AM, Juanjo  wrote:

>
>
> OK, thanks.
>>
>> Now works as the old RT.
>> But i think that i have to migrate RT 4.0.5 in mysql to postgre.
>>
>> Anybody do it?
>>
>>
>>
>>
>> 2012/5/31 Tim Cutts 
>>
>>>
>>> On 31 May 2012, at 10:06, Juanjo wrote:
>>>
>>> > Thanks for your reply.
>>> >
>>> > Yes, i read the full_text_indexing.pod
>>> > But, are the only option to search in content activate the indiexing
>>> with a compatible database?
>>> >
>>> > I don´t want to migrate to postgre, i think is very hard to my
>>> knowledge.
>>>
>>> You can do it with MySQL, if you enable sphinx (which is also fairly
>>> difficult, but not as hard as a Postgres migration).
>>>
>>> I think you can also switch it on without the indexing, which will
>>> restore 3.x like behaviour, with very slow full content searches.  I think
>>> for this, you want something like this in your RT site config:
>>>
>>> Set( %FullTextSearch,
>>>Enable => 1,
>>>Indexed=> 0,
>>> );
>>>
>>> Regards,
>>>
>>>
>>> Tim
>>>
>>> --
>>>  The Wellcome Trust Sanger Institute is operated by Genome Research
>>>  Limited, a charity registered in England with number 1021457 and a
>>>  company registered in England with number 2742969, whose registered
>>>  office is 215 Euston Road, London, NW1 2BE.
>>>
>>
>>
>>
>> --
>> Un saludo.
>> Juanjo Corral
>>
>
>
>
> --
> Un saludo.
> Juanjo Corral
>

Re: [rt-users] RT 4 Upgrade Slow Performance - CustomFields?

[Nathan Baker]

Okay...just an update.  This is definitely directly tied to the Apache
KeepAliveTimeout setting.  The default was 15 seconds for my installation,
and if I change it to 10 seconds or 60 seconds that is exactly how long of
a wait is required to make it "slow" again.

So from here it looks like the options are:
1) Something is rebuilding after the KeepAliveTimeout expires that
shouldn't be
or
2) Something should be rebuilding and my server is not doing that fast
enough.

The server I'm using has one Xeon 2.4GHz CPU and 2GB Memory (it is
virtual).  Any pointers on what's happening when the KeepAliveTimout
expires would be greatly appreciated.

-Nate

Re: [rt-users] RT 4 Upgrade Slow Performance - CustomFields?

[Nathan Baker]

Ruslan,

I guess what I was getting at is I don't think the SQL queries are the
problem here.  The sum (by using your perl code) was 0.324813 seconds, much
less than 4 seconds.

I'm still trying different Apache settings, Postgresql settings, etc., but
here's a different way to explain the issue:

I can click on Home, then click on a ticket, then Home, then a ticket,
etc., for as long as I want and it is very fast (page load time is about
0.2-0.3 seconds).

If I wait for about 5 seconds, I can continue this without any change.

If I wait for even 10 seconds, or 30 seconds, or anything longer, when I
click on either Home or a ticket, the page takes about 2-4 seconds to load.
This will happen for both the Home page and a ticket Display page the first
time they are each displayed, and after that they are very fast again.
 This is why it seems to me it has to be an Apache or Mason cache/timeout
issue.  It's almost like it's compiling or building something again, which
is why I originally was drawn to the Javascript minifier suggestion.

I really appreciate your advice on this.

-Nate

On Wed, May 30, 2012 at 3:49 PM, Ruslan Zakirov wrote:

> On Wed, May 30, 2012 at 10:15 PM, Nathan Baker  wrote:
> > Ruslan,
> >
> > I wasn't aware that sessions had to be cleared, but now that you
> mentioned
> > it I looked and there were almost 10k sessions in our table.  I cleared
> that
> > out and it does not seem to be slow in that section anymore.  I've also
> > added that command to crontab to run daily.
> >
> > It seems much better after doing that, I don't see any pages taking over
> 3-4
> > seconds to load anymore.  I do still see some taking about 2-3 seconds to
> > load.  That is more usable, although I would like to get it better if
> > possible.  I turned on logging of SQL statements in RT only for now, I'm
> not
> > as familiar with Postgresql as I am with MySQL, but I could look into
> > logging in there if necessary.
> >
> > We only have about 5 staff using the RT web interface.  The longest
> queries
> > I see are still fairly quick I think, but here are a couple examples of
> the
> > longer ones for Display.html:
> >
> > [Wed May 30 17:52:44 2012] [debug]: SQL(0.136027s): SELECT  * FROM Groups
> > WHERE LOWER(Domain) = LOWER(?) AND LOWER(Type) = LOWER(?);  [ bound
> values:
> > 'SystemInternal' 'Privileged' ]
> > (/usr/share/request-tracker4/lib/RT/Interface/Web.pm:1115)
>
> This one is sort of "known issue". We have code paths where LOWER is
> used and is not used, but indexes for Pg on Groups table don't account
> this. Proper way would be to collect as many queries as possible that
> select or join Groups table, do analysis and come up with set of
> indexes for the table. I would love to take a look at such log.
>
> For this particular query index on LOWER(Domain) would be enough or
> (LOWER(Domain), LOWER(Type)) pair.
>
> > [Wed May 30 17:49:20 2012] [debug]: SQL(0.387888s): SELECT DISTINCT
> main.*
> > FROM Users main JOIN Principals Principals_1  ON ( Principals_1.id =
> main.id
> > ) JOIN CachedGroupMembers CachedGroupMembers_2  ON (
> > CachedGroupMembers_2.MemberId = Principals_1.id )  WHERE
> > (Principals_1.Disabled = '0') AND (CachedGroupMembers_2.GroupId =
> '25472')
> > AND (CachedGroupMembers_2.Disabled = '0') AND
> > (LOWER(Principals_1.PrincipalType) = 'user')  ORDER BY main.Name ASC ;
> > (/usr/share/request-tracker4/lib/RT/Interface/Web.pm:1115)
>
> Hard to tell anything without EXPLAIN ANALYZE... This query selects
> users who are not disabled and recursive members of a group. Select
> from Groups by id to see how important this query is.
>
> > Here is an example summary of a page load (with SQL logging on and Mason
> > profiling off) for Display.html:
> > 48 Queries
> > 4.1 Seconds page load
> > Longest query:
> > [Wed May 30 17:56:19 2012] [debug]: SQL(0.118095s): SELECT  * FROM Groups
> > WHERE LOWER(Domain) = LOWER(?) AND LOWER(Type) = LOWER(?);  [ bound
> values:
> > 'SystemInternal' 'Unprivileged' ]
> > (/usr/share/request-tracker4/lib/RT/Interface/Web.pm:1115)
> >
> > All other queries < 0.1 seconds, most are < 0.01 seconds
>
> Use the following command to calculate how much time all 48 queries took:
>
> cat part.of.rt.log | perl -ne '$res += (/SQL\(([0-9.]+s)\)/)[0] || 0;
> END { print "$res\n"}'
>
> Question is how close the sum to 4 seconds.
>
> > Thanks,
> > Nate
>
> --
> Best regards, Ruslan.
>

Re: [rt-users] RT 4 Upgrade Slow Performance - CustomFields?

[Nathan Baker]

Ruslan,

I wasn't aware that sessions had to be cleared, but now that you mentioned
it I looked and there were almost 10k sessions in our table.  I cleared
that out and it does not seem to be slow in that section anymore.  I've
also added that command to crontab to run daily.

It seems much better after doing that, I don't see any pages taking over
3-4 seconds to load anymore.  I do still see some taking about 2-3 seconds
to load.  That is more usable, although I would like to get it better if
possible.  I turned on logging of SQL statements in RT only for now, I'm
not as familiar with Postgresql as I am with MySQL, but I could look into
logging in there if necessary.

We only have about 5 staff using the RT web interface.  The longest queries
I see are still fairly quick I think, but here are a couple examples of the
longer ones for Display.html:

[Wed May 30 17:52:44 2012] [debug]: SQL(0.136027s): SELECT  * FROM Groups
WHERE LOWER(Domain) = LOWER(?) AND LOWER(Type) = LOWER(?);  [ bound values:
'SystemInternal' 'Privileged' ]
(/usr/share/request-tracker4/lib/RT/Interface/Web.pm:1115)

[Wed May 30 17:49:20 2012] [debug]: SQL(0.387888s): SELECT DISTINCT main.*
FROM Users main JOIN Principals Principals_1  ON ( Principals_1.id =
main.id) JOIN CachedGroupMembers CachedGroupMembers_2  ON (
CachedGroupMembers_2.MemberId = Principals_1.id )  WHERE
(Principals_1.Disabled = '0') AND (CachedGroupMembers_2.GroupId = '25472')
AND (CachedGroupMembers_2.Disabled = '0') AND
(LOWER(Principals_1.PrincipalType) = 'user')  ORDER BY main.Name ASC ;
(/usr/share/request-tracker4/lib/RT/Interface/Web.pm:1115)

Here is an example summary of a page load (with SQL logging on and Mason
profiling off) for Display.html:
48 Queries
4.1 Seconds page load
Longest query:
[Wed May 30 17:56:19 2012] [debug]: SQL(0.118095s): SELECT  * FROM Groups
WHERE LOWER(Domain) = LOWER(?) AND LOWER(Type) = LOWER(?);  [ bound values:
'SystemInternal' 'Unprivileged' ]
(/usr/share/request-tracker4/lib/RT/Interface/Web.pm:1115)

All other queries < 0.1 seconds, most are < 0.01 seconds

Thanks,
Nate

On Wed, May 30, 2012 at 12:09 PM, Ruslan Zakirov wrote:

> On Wed, May 30, 2012 at 7:53 PM, Nathan Baker  wrote:
> > Ruslan,
> >
> > I actually started the other thread, so my details are in the first post.
> >  That thread was sort of hijacked (no big deal) and getting messy, so I
> > wanted to separate them.  I'm using Postgresql, not MySQL, and had
> already
> > turned on the SQL statement log and all queries seemed to be completing
> > quickly.
>
> I know that your thread was hijacked, but this one lacks history and
> for me it's hard
> to jump between threads to figure out background. Anyway, see comments
> below.
>
> Do you log SQL statements on RT side or in Pg? In your case I would
> recommend
> to enable both method for a while.
>
> > I think I might have been too quick to assume it's the custom fields
> though,
> > I've been testing it for a few hours now and I don't see that part being
> > slow anymore.  To be honest the only part I see being consistently slow
> > right now is:
> >
> > =Mason= localhost - /Elements/SetupSessionCookie {{{
> > =Mason= localhost - /Elements/SetupSessionCookie }}} 1.9105
> >
> > That seems to take about 2 seconds on about 30% of page loads.  Is that
> > normal?
>
> Nope. It's either big session size or you don't clean sessions table from
> old
> records with rt-clean-sessions.
>
> SELECT COUNT(1) FROM sessions;
>
> Above should say how many records you have in the table.
>
> Also, sessions are locked and if you click a link or refresh while page
> is still loading then time on SetupSessionCookie can contain time it
> was waiting for lock to be released by previous request.
>
> I'm not sure how you're testing. If your DB is actively used by many RT
> users then it's hard to tell what's slow.
>
> > I will try to turn off caching in Postgresql though to see if that
> verifies
> > your theory about slow queries.
>
> Ah. You migrated to Pg. Caching in Pg is different from mysql.
>
> > Thanks!
> > Nate
>
> --
> Best regards, Ruslan.
>

Re: [rt-users] RT 4 Upgrade Slow Performance - CustomFields?

[Nathan Baker]

Ruslan,

I actually started the other thread, so my details are in the first post.
 That thread was sort of hijacked (no big deal) and getting messy, so I
wanted to separate them.  I'm using Postgresql, not MySQL, and had already
turned on the SQL statement log and all queries seemed to be completing
quickly.

I think I might have been too quick to assume it's the custom fields
though, I've been testing it for a few hours now and I don't see that part
being slow anymore.  To be honest the only part I see being consistently
slow right now is:

=Mason= localhost - /Elements/SetupSessionCookie {{{
=Mason= localhost - /Elements/SetupSessionCookie }}} 1.9105

That seems to take about 2 seconds on about 30% of page loads.  Is that
normal?

I will try to turn off caching in Postgresql though to see if that verifies
your theory about slow queries.

Thanks!
Nate

On Wed, May 30, 2012 at 11:37 AM, Ruslan Zakirov wrote:

> On Wed, May 30, 2012 at 6:37 PM, Nathan Baker  wrote:
> > I'm going to try and separate this thread since my issue doesn't seem to
> be
> > related to the Search page or the SelectOwner field.  Using the Mason
> > Profiler did give me some info though, it looks like it might be due to
> some
> > custom fields:
> >
> > =Mason= localhost -
> > /Ticket/Elements/ShowCustomFields {{{
> > =Mason= localhost -
> > /Elements/ShowCustomFields {{{
> > =Mason= localhost -
> > /Elements/ShowCustomFields }}} 2.0221
> > =Mason= localhost -
> > /Ticket/Elements/ShowCustomFields }}} 2.0261
> > ...
> > =Mason= localhost - /autohandler }}} 3.2073
> >
> > Is it normal that having 1-2 custom fields with about 10-20 options for
> each
> > would cause this extra delay.  I know 2 seconds doesn't seem like much,
> but
> > it can sometimes be longer.  Can anyone give me any tips as to what the
> > issue could be?  Is Mason maybe caching that differently?
>
> Nate,
>
> Separating threads is correct, but skipping details about your system
> is not so good.
>
> I think here it's either slow one query or many semi-fast
> queries. If you collect all queries for this page, it wouldn't be too
> hard to isolate those that executed by ShowCustomFields.
>
> The fact that subsequent requests take less time suggests that queries
> end up in mysql query cache and are fast until cache is flushed. To
> confirm you can disable cache in mysql for some time and check if
> problem is repeatable on every request.
>
> >
> > Thanks,
> > Nate
>
>
>
> --
> Best regards, Ruslan.
>

[rt-users] RT 4 Upgrade Slow Performance - CustomFields?

[Nathan Baker]

I'm going to try and separate this thread since my issue doesn't seem to be
related to the Search page or the SelectOwner field.  Using the Mason
Profiler did give me some info though, it looks like it might be due to
some custom fields:

=Mason= localhost -
/Ticket/Elements/ShowCustomFields {{{
=Mason= localhost -
/Elements/ShowCustomFields {{{
=Mason= localhost -
/Elements/ShowCustomFields }}} 2.0221
=Mason= localhost -
/Ticket/Elements/ShowCustomFields }}} 2.0261
...
=Mason= localhost - /autohandler }}} 3.2073

Is it normal that having 1-2 custom fields with about 10-20 options for
each would cause this extra delay.  I know 2 seconds doesn't seem like
much, but it can sometimes be longer.  Can anyone give me any tips as to
what the issue could be?  Is Mason maybe caching that differently?

Thanks,
Nate

Re: [rt-users] RT 4 Upgrade Slow Performance

[Nathan Baker]

Ruslan,

Thank you, that provided some great info.  I'm wondering if my issue (or
one of them) is from our custom fields.  There is one spot where it
consistently takes a couple seconds:

=Mason= localhost -
/Ticket/Elements/ShowCustomFields {{{
=Mason= localhost -
/Elements/ShowCustomFields {{{
=Mason= localhost -
/Elements/ShowCustomFields }}} 2.0221
=Mason= localhost -
/Ticket/Elements/ShowCustomFields }}} 2.0261
...
=Mason= localhost - /autohandler }}} 3.2073

We only have 1 or 2 custom fields (depending on the Queue), and there are
less than 20 options for each.  Any ideas what could cause that, or is that
normal for custom fields?

Thanks,
Nate

On Tue, May 29, 2012 at 4:59 PM, Ruslan Zakirov wrote:

> Hi,
>
> Probably next step would be Mason profiler. It's described in
> RT_Config.pm. Once you know where WebUI is slow return back.
>
> On Wed, May 30, 2012 at 12:32 AM, Nathan Baker  wrote:
> > Hello Everyone,
> >
> > We've upgraded from RT 3.8.8 with MySQL to RT 4.0.5 (debian package) with
> > Postgresql, and are having some performance issues with the web
> interface.
> >  I've searched the list archives and Google, and haven't been able to
> find
> > the issue.  I'm hoping someone can help point me in the right direction.
> >  Basically, the first time I visit any page (Home, Display, etc.) it
> takes
> > 3-5 seconds to load, and sometimes 10-15 seconds.  If I keep clicking on
> > other links as soon as the pages load, it's fast and loads pages in 0.2 -
> > 0.3 seconds.  If I wait for about 15-30 seconds and then click a link
> again,
> > it takes 3-5 seconds to load again.  Updating tickets seems to
> consistently
> > take 3-5 seconds.
> >
> > Here's some info about our installation:
> > - About 6,000 tickets, 5 active users
> > - Increased memory to 2GB, running on VMWare
> > - Increased Postgresql shared_buffers and effective_cache_size
> > (http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server)
> > - Changed JS Minifier to jsmin as suggested
> > here: http://www.gossamer-threads.com/lists/rt/users/102703
> >
> > Our config seems fairly basic compared so some examples I've seen, I can
> > post it if necessary though.  We are using FullTextSearch and
> CommandByMail,
> > although I've tried disabling them both.  I turned on debugging to a log
> > file, and I don't see anything unusual.  I also turned on the SQL
> statement
> > log, and didn't see any queries taking a long time.  The longest was 0.12
> > seconds.  I tried disabling the Javascript Minifier in JS.pm like I've
> seen
> > in other posts, and that didn't seem to make a difference.
> >
> > Any help would be greatly appreciated.
> >
> > Thanks,
> > Nate
>
>
>
> --
> Best regards, Ruslan.
>

[rt-users] RT 4 Upgrade Slow Performance

[Nathan Baker]

Hello Everyone,

We've upgraded from RT 3.8.8 with MySQL to RT 4.0.5 (debian package) with
Postgresql, and are having some performance issues with the web interface.
 I've searched the list archives and Google, and haven't been able to find
the issue.  I'm hoping someone can help point me in the right direction.
 Basically, the first time I visit any page (Home, Display, etc.) it takes
3-5 seconds to load, and sometimes 10-15 seconds.  If I keep clicking on
other links as soon as the pages load, it's fast and loads pages in 0.2 -
0.3 seconds.  If I wait for about 15-30 seconds and then click a link
again, it takes 3-5 seconds to load again.  Updating tickets seems to
consistently take 3-5 seconds.

Here's some info about our installation:
- About 6,000 tickets, 5 active users
- Increased memory to 2GB, running on VMWare
- Increased Postgresql shared_buffers and effective_cache_size (
http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server)
- Changed JS Minifier to jsmin as suggested here:
http://www.gossamer-threads.com/lists/rt/users/102703

Our config seems fairly basic compared so some examples I've seen, I can
post it if necessary though.  We are using FullTextSearch and
CommandByMail, although I've tried disabling them both.  I turned on
debugging to a log file, and I don't see anything unusual.  I also turned
on the SQL statement log, and didn't see any queries taking a long time.
 The longest was 0.12 seconds.  I tried disabling the Javascript Minifier
in JS.pm like I've seen in other posts, and that didn't seem to make a
difference.

Any help would be greatly appreciated.

Thanks,
Nate