Hi all,
Please find attached a short description on how I migrated our 26K
bugzilla tickets to RT.
Any comments or fixes will be welcomed.
Environment:
- OS: Debian Etch 2.6.18-6-486
- RT version: 3.8.2
- MySQL version: 5.0.32-Debian_7etch8-log
Cheers,
Javier Garcia
Systems Engineer at Snow Valley
-----Original Message-----
From: rt-users-boun...@lists.bestpractical.com
[mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Javier
Garcia
Sent: 23 April 2009 14:03
To: Keith A McDermott; Ken Crocker
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] Migration
Hi,
I've finally finished the migration.
We have to test everything but I'm reasonably happy about it.
I will send an update with my code changes asap.
Cheers,
Javier
-----Original Message-----
From: Keith A McDermott [mailto:kmcde...@purdue.edu]
Sent: 23 April 2009 13:56
To: Ken Crocker
Cc: Javier Garcia; rt-users@lists.bestpractical.com
Subject: Re: [rt-users] Migration
Kenn,
I have not, however I am definitely not too good at programming - let
alone perl (my brain is hard-coded to procedural not object-oriented
programming). Sounds interesting though as a possible side-project as
we're not going to be live with RT for some time.
Keith
Ken Crocker wrote:
> Kieth,
>
> Have you thought about writing a program to copy all the Bugzilla data
> into flatfiles that can then be used as input to another program that
> uses API to create tickets, etc? We just converted an old legacy
> ticket system into RT doing just that, comments and email included.
> Converted 17,000 tickets. Of course, I had to take RT down and then
> turn OFF /all/ scrips, etc (didn't need 17,00 emails going out).
> Afterwards, I did a little bulk updating for some custom fields and
> viola! All done. Tirn system on, Turn on scrips. ready to go.
>
>
> Kenn
> LBNL
>
> On 4/22/2009 5:48 AM, Keith A McDermott wrote:
>> I'd like to see this if it does eventually work. I gave up on
converting
>> our bugzilla database many months ago. Although we have 12,000
bugzilla
>> tickets, it wasn't worth the time and pain for us to try to fix the
script.
>>
>> Thanks,
>> Keith
>>
>> Javier Garcia wrote:
>>
>>> OK. I've found out that my initialdata file is wrong. There's
nothing
>>> on the @Groups array.
>>>
>>> I've downloaded a new initialdata file (the syntax is as follows):
>>>
>>> @Groups = (
>>>
>>> { Name => '',
>>>
>>> Type => 'Everyone', # loc
>>>
>>> Domain => 'SystemInternal',
>>>
>>> Instance => '',
>>>
>>> Description => 'Pseudogroup for internal use', # loc
>>>
>>> },
>>>
>>> But then I get a new error:
>>>
>>> Odd number of elements in hash assignment at
>>> /opt/rt3/lib/RT/Group_Overlay.pm line 405.
>>>
>>> /opt/rt3/lib/RT/Group_Overlay.pm line 405:
>>>
>>> my %args = (
>>>
>>> Name => undef,
>>>
>>> Description => undef,
>>>
>>> Domain => undef,
>>>
>>> Type => undef,
>>>
>>> Instance => '0',
>>>
>>> InsideTransaction => undef,
>>>
>>> _RecordTransaction => 1,
>>>
>>> @_
>>>
>>> );
>>>
>>> I'm not an expert Perl developer, but I have tried modifying the
>>> @Groups array, adding more fields or deleting others, but nothing,
>>> same error.
>>>
>>> *From:* rt-users-boun...@lists.bestpractical.com
>>> [mailto:rt-users-boun...@lists.bestpractical.com] *On Behalf Of
>>> *Javier Garcia
>>> *Sent:* 21 April 2009 10:50
>>> *To:* Jo Rhett
>>> *Cc:* rt-users@lists.bestpractical.com
>>> *Subject:* Re: [rt-users] Migration
>>>
>>> Hi all,
>>>
>>> I have found the scripts here:
>>>
>>>
http://www.gossamer-threads.com/lists/rt/users/53073?do=post_view_thread
ed#53073
>>>
>>> But they are not working. After following the README I ran "sh
>>> migrate.sh" and got the following:
>>>
>>> Could not load Everyone group on user creation.
>>> (/opt/rt3/lib/RT/User_Overlay.pm:222)
>>>
>>> The error is on the following block:
>>>
>>> my $everyone = RT::Group->new($self->CurrentUser);
>>>
>>> $everyone->LoadSystemInternalGroup('Everyone');
>>>
>>> unless ($everyone->id) {
>>>
>>> $RT::Logger->crit("Could not load Everyone group on user
creation.");
>>>
>>> $RT::Handle->Rollback();
>>>
>>> return ( 0, $self->loc('Could not create user') );
>>>
>>> }
>>>
>>> And we are not able to solve it. There's no information about it on
>>> the Internet. Any ideas?
>>>
>>> I have a hunch that it has to do with the relationship between the
>>> principals, groups and ACL table.
>>>
>>> Any help would be greatly appreciated, otherwise we can continue
>>> writing bespoke scripts to fit our needs.
>>>
>>> We are currently running RT 3.8.2 under Debian Etch.
>>>
>>> Cheers,
>>>
>>> Javier
>>>
>>> *From:* Jo Rhett [mailto:jrh...@netconsonance.com]
>>> *Sent:* 20 April 2009 17:34
>>> *To:* Javier Garcia
>>> *Cc:* rt-users@lists.bestpractical.com
>>> *Subject:* Re: [rt-users] Migration
>>>
>>> Yes. Search the mailing list archives or Google. Both of which
return
>>> the script you need quickly.
>>>
>>> On Apr 20, 2009, at 8:59 AM, Javier Garcia wrote:
>>>
>>> We are manually migrating all our Bugzilla databases to RT using
>>> Perl, but we are having issues with some of them and I can't
find
>>> anything on the Internet.
>>>
>>> Is there a documented way on How to migrate Bugzilla to RT?
>>>
>>> Does anybody have a script for that?
>>>
>>> PS: Both servers are running on Debian and using MySQL.
>>>
>>> Thanks,
>>>
>>> Javier Garcia
>>>
>>> Systems Engineer
>>>
>>> javier.gar...@snowvalley.com
<mailto:javier.gar...@snowvalley.com>
>>>
>>> 10 Emerald Street, London, WC1N 3QA
>>>
>>> Website: http://www.snowvalley.com <http://www.snowvalley.com/>
>>>
>>> _______________________________________________
>>> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>>>
>>> Community help: http://wiki.bestpractical.com
>>> Commercial support: sa...@bestpractical.com
>>> <mailto:sa...@bestpractical.com>
>>>
>>>
>>> Discover RT's hidden secrets with RT Essentials from O'Reilly
Media.
>>> Buy a copy at http://rtbook.bestpractical.com
>>>
>>> --
>>>
>>> Jo Rhett
>>>
>>> Net Consonance : consonant endings by net philanthropy, open source
>>> and other randomness
>>>
>>>
------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>>>
>>> Community help: http://wiki.bestpractical.com
>>> Commercial support: sa...@bestpractical.com
>>>
>>>
>>> Discover RT's hidden secrets with RT Essentials from O'Reilly Media.
>>> Buy a copy at http://rtbook.bestpractical.com
>>>
>> _______________________________________________
>> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>>
>> Community help: http://wiki.bestpractical.com
>> Commercial support: sa...@bestpractical.com
>>
>>
>> Discover RT's hidden secrets with RT Essentials from O'Reilly Media.
>> Buy a copy at http://rtbook.bestpractical.com
>>
>>
_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
Community help: http://wiki.bestpractical.com
Commercial support: sa...@bestpractical.com
Discover RT's hidden secrets with RT Essentials from O'Reilly Media.
Buy a copy at http://rtbook.bestpractical.com
**************************
* Bugzilla RT migration
* Javier Garcia
* Snow Valley
* javier(dot)garcia(at)snowvalley(dot)com
* 24/04/2009
**************************
Environment:
- OS: Debian Etch 2.6.18-6-486
- RT version: 3.8.2
- MySQL version: 5.0.32-Debian_7etch8-log
I. Download and untar migration files
-----------------------------------
tar -xvzf bugzilla-rt-convert.tar.gz
II. Read README file
-----------------------------------
Please, pay attention to the README file inside bugzilla-rt-convert folder
III. Replace initial data file
-----------------------------------
The original /opt/rt3/etc/initialdata file on RT3.8 has no groups
information. We have to insert this information in our DB in order to avoid
future issues loading tickets and queues.
@Groups = (
{ Name => '',
Type => 'Everyone',
Domain => 'SystemInternal',
Instance => '',
Description => 'Pseudogroup for internal use',
InsideTransaction => undef,
},
{ Type => 'Privileged',
Domain => 'SystemInternal',
Instance => '',
Name => '',
Description => 'Pseudogroup for internal use',
InsideTransaction => undef,
},
{ Name => '',
Type => 'Unprivileged',
Domain => 'SystemInternal',
Instance => '',
Description => 'Pseudogroup for internal use',
InsideTransaction => undef,
},
{ Name => '',
Type => 'Owner',
Domain => 'RT::System-Role',
Instance => '',
Description => 'SystemRolegroup for internal use',
InsideTransaction => undef,
},
{ Name => '',
Type => 'Requestor',
Domain => 'RT::System-Role',
Instance => '',
Description => 'SystemRolegroup for internal use',
InsideTransaction => undef,
},
{ Name => '', Type => 'Cc',
Domain => 'RT::System-Role',
Instance => '',
Description => 'SystemRolegroup for internal use',
InsideTransaction => undef,
},
{ Name => '',
Type => 'AdminCc',
Domain => 'RT::System-Role',
Instance => '',
Description => 'Pseudogroup for internal use',
InsideTransaction => undef,
},
);
IV. print_prod.pl
-----------------------------------
This script loads tickets information from bugzilla in a file. There are issues
in some of the loaded data, as lots of creation date fields contain the string
now() instead of a valid date. The best way to fix this is to replace the
following lines in the script:
### sub print_prod
my $created = 'now()';
...
print "\t Description\t=> $desc,\n";
print "\t Created\t=> $created,\n";
With these ones (NOTE: we are adding a random created date. Some people will
need to modify this as they have to use the current date and time. Perl
provides lots of time functions for this):
### sub print_prod
my $created = 'now()';
...
print "\t Description\t=> $desc,\n";
#
### ADDED TO AVOID now() DATES WHICH CAUSE LOADING ERRORS ON RT
if ($created eq 'now()') {
print "\t Created\t=> '2009-04-22 11:00:00',\n";
}else {
print "\t Created\t=> $created,\n";
}
V. print_bugs.pl
-----------------------------------
This script loads bugs information from bugzilla in a file. We are in
the same situation again, as there a lots of corrupted data. The best way to
fix this is to replace the following lines in the script:
### sub print_bugs
my $created = 'now()';
...
print "\t Type\t=> 'ticket',\n";
print "\t Created\t=> $created,\n";
With these ones (NOTE: we are adding a random created date. Some people will
need to modify this as they have to use the current date and time. Perl
provides lots of time functions for this):
### sub print_bugs
my $created = 'now()';
...
print "\t Type\t=> 'ticket',\n";
#
### ADDED TO AVOID now() DATES WHICH CAUSE LOADING ERRORS ON RT
if ($created eq 'now()') {
print "\t Created\t=> '2009-04-22 11:00:00',\n";
}else {
print "\t Created\t=> $created,\n";
}
VI. migrate.sh
-----------------------------------
Modify create_user function adding a line for the DB initialization. This
initialization is needed as Nobody and System users are not automatically
added by the original script.
create_user(){
passwd=$1
perl $setup --action drop --dba root --dba-password $passwd
# We have to initialize the DB, as we need to insert Nobody and
# System users, which are required and the original script
doesn't
# carry this out
perl $setup --action init --dba root --dba-password $passwd
# don't insert scrip here
perl bugimport --action init --dba root --dba-password $passwd
perl print_users_groups.pl $passwd > data.pl
perl $setup --action insert --datafile ./data.pl --dba root
--dba-password $passwd
perl insert_ids_map.pl $passwd
}
VII. Stop emailing
-----------------------------------
In order to avoid thousands of emails to be sent automatically, every time a
new ticket is created, we have to stop our mail client
VIII. Run migration script
-----------------------------------
The best way to run the script is redirecting the standard error (which
contains also Perl warning
and informational messages) to a file
sh migrate.sh 2> output
IX. Delete queued emails
-----------------------------------
Before restarting the email client we have to delete all the queued emails:
rm -f /var/spool/mqueue/* /var/spool/mqueue-client/*
NOTE: If there are too much pending emails, rm wont work and you will need to
use the following command instead:
cd /var/spool/mqueue
find . -name '*' -print0 | xargs -0 rm
cd /var/spool/mqueue-client
find . -name '*' -print0 | xargs -0 rm
X. Restart email client
-----------------------------------
_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
Community help: http://wiki.bestpractical.com
Commercial support: sa...@bestpractical.com
Discover RT's hidden secrets with RT Essentials from O'Reilly Media.
Buy a copy at http://rtbook.bestpractical.com
