On Fri, Jun 12, 2009 at 10:36 AM, Frances Dean McNamara<[email protected]> wrote: > I believe NONE of the records loaded got rows. Perhaps trying the > bib_ids.list for ranges of records to see if it works. You say that would be > a really long thing? Does that do more thand the metabib.rec_descriptor > inserts? >
Right. I gave you the more complicated SQL so you could grab a fresh set of just the remaining to-be-fixed records if you have to stop the process or it dies. It re-ingests the entire record, basically rewriting it. This, of course, assumes that the mrd portion of the ingest process is not broken in some way and that re-ingesting will fix things. you can test on one record to find out, of course. > Or we could use your first suggestion. We actually do have the original > input files broken down to about 59 files I think. I will talk to Dale when > he is in. If we figure out which way to do it we can probably script it. > This will be faster, but more involved. Pop in here if you run into trouble. --miker > Thanks. > > Frances > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Mike > Rylander > Sent: Friday, June 12, 2009 9:03 AM > To: Evergreen Development Discussion List > Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading > bibs. > > On Fri, Jun 12, 2009 at 9:47 AM, Frances Dean > McNamara<[email protected]> wrote: >> Ah, I see, it is pulling out stuff like type of date, date1, date2 from the >> fixed field, bib lvl, enc lvl, looks like mainly 008 type of data? Is there >> a script we could run for a while to build this? We have over 5 million >> bibs. Thanks. Frances >> > > You can use the stock ingest scripts, and avoid intermediate files by > doing something like: > > $ marc2bre.pl --marctype=xml [... other options] | direct_ingest.pl | > grep '"__c":"mrd"' > mrd.ingest > > for 1.4. If you're working with 1.2, you'll need to change the grep, > but that will get you the mrd data. > > Another other option is to get a list of all the bibs that lack 'mrd's > in a file and re-ingest them. In psql: > > \t > \o /tmp/bib_ids.list > SELECT b.id FROM biblio.record_entry b LEFT JOIN > metabib.rec_descriptor d ON (b.id = d.record) WHERE r.id IS NULL b.id >> 0 AND NOT b.deleted; > \o > \t > > Then as the opensrf user, at the command line: > > $ for i in `cat /tmp/bib_ids.list`; do echo "request open-ils.ingest > open-ils.ingest.full.biblio.record $i"; done | srfsh > > That will take, um, a really long time to complete. If anything > happens to stop that, just start over with a fresh bib_ids.list file > -- the SQL above will give you only te records that are missing an > 'mrd'. > > --miker > >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Frances >> Dean McNamara >> Sent: Friday, June 12, 2009 8:37 AM >> To: Evergreen Development Discussion List >> Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading >> bibs. >> >> Dale has not replied to this yet, but I don't think we have kept the interim >> files due to file space (big database). >> >> Are their scripts needed to regenerate that table from what is in the db >> now? What is in that table anyhow? >> >> Frances McNamara >> University of Chicago >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Mike >> Rylander >> Sent: Friday, June 12, 2009 7:12 AM >> To: Evergreen Development Discussion List >> Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading >> bibs. >> >> On Thu, Jun 11, 2009 at 7:23 PM, Dale Arntson<[email protected]> wrote: >>> Hi All, >>> >>> After loading our bib records into evergreen, I got zero hits on searches in >>> the evergreen client. I traced the problem back to the fact that there are >>> no records in the metabib.rec_descriptor table. The other metabib tables >>> seem fully populated. Here are the flags I used in parallel_pg_loader. >>> >>> perl parallel_pg_loader.pl -order bre -order mrd -order mfr -order mtfe >>> -order mafe -order msfe -order mkfe -order msefe -autoprimary mrd >>> -autoprimary mfr -autoprimary mtfe -autoprimary mafe -autoprimary msfe >>> -autoprimary mkfe -autoprimary msefe >>> >>> Any ideas what I did wrong? Any ideas how to fix it? The bibs took a long >>> time to load. I would rather not redo it, if I don't have to. >> >> If you have the intermediate bib processing files, particularly the >> output of direct_ingest, make sure that file contains 'mrd' rows. For >> OpenSRF 0.9 (Evergreen 1.2.x), those will start with '/*--S mrd--*/', >> and in OpenSRF 1.0.x they will start with '[{"__c":"mrd","__p"'. >> >> If you can find that file and it contains 'mrd' lines, then we can >> simply regenerate that table's worth of data. Otherwise you'll need to >> reprocess the bib records in order to get that data, but you won't >> need to reload the entire dataset. >> >> -- >> Mike Rylander >> | VP, Research and Design >> | Equinox Software, Inc. / The Evergreen Experts >> | phone: 1-877-OPEN-ILS (673-6457) >> | email: [email protected] >> | web: http://www.esilibrary.com >> > > > > -- > Mike Rylander > | VP, Research and Design > | Equinox Software, Inc. / The Evergreen Experts > | phone: 1-877-OPEN-ILS (673-6457) > | email: [email protected] > | web: http://www.esilibrary.com > -- Mike Rylander | VP, Research and Design | Equinox Software, Inc. / The Evergreen Experts | phone: 1-877-OPEN-ILS (673-6457) | email: [email protected] | web: http://www.esilibrary.com
