I believe NONE of the records loaded got rows. Perhaps trying the bib_ids.list for ranges of records to see if it works. You say that would be a really long thing? Does that do more thand the metabib.rec_descriptor inserts?
Or we could use your first suggestion. We actually do have the original input files broken down to about 59 files I think. I will talk to Dale when he is in. If we figure out which way to do it we can probably script it. Thanks. Frances -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Mike Rylander Sent: Friday, June 12, 2009 9:03 AM To: Evergreen Development Discussion List Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading bibs. On Fri, Jun 12, 2009 at 9:47 AM, Frances Dean McNamara<[email protected]> wrote: > Ah, I see, it is pulling out stuff like type of date, date1, date2 from the > fixed field, bib lvl, enc lvl, looks like mainly 008 type of data? Is there > a script we could run for a while to build this? We have over 5 million > bibs. Thanks. Frances > You can use the stock ingest scripts, and avoid intermediate files by doing something like: $ marc2bre.pl --marctype=xml [... other options] | direct_ingest.pl | grep '"__c":"mrd"' > mrd.ingest for 1.4. If you're working with 1.2, you'll need to change the grep, but that will get you the mrd data. Another other option is to get a list of all the bibs that lack 'mrd's in a file and re-ingest them. In psql: \t \o /tmp/bib_ids.list SELECT b.id FROM biblio.record_entry b LEFT JOIN metabib.rec_descriptor d ON (b.id = d.record) WHERE r.id IS NULL b.id > 0 AND NOT b.deleted; \o \t Then as the opensrf user, at the command line: $ for i in `cat /tmp/bib_ids.list`; do echo "request open-ils.ingest open-ils.ingest.full.biblio.record $i"; done | srfsh That will take, um, a really long time to complete. If anything happens to stop that, just start over with a fresh bib_ids.list file -- the SQL above will give you only te records that are missing an 'mrd'. --miker > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Frances > Dean McNamara > Sent: Friday, June 12, 2009 8:37 AM > To: Evergreen Development Discussion List > Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading > bibs. > > Dale has not replied to this yet, but I don't think we have kept the interim > files due to file space (big database). > > Are their scripts needed to regenerate that table from what is in the db now? > What is in that table anyhow? > > Frances McNamara > University of Chicago > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Mike > Rylander > Sent: Friday, June 12, 2009 7:12 AM > To: Evergreen Development Discussion List > Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading > bibs. > > On Thu, Jun 11, 2009 at 7:23 PM, Dale Arntson<[email protected]> wrote: >> Hi All, >> >> After loading our bib records into evergreen, I got zero hits on searches in >> the evergreen client. I traced the problem back to the fact that there are >> no records in the metabib.rec_descriptor table. The other metabib tables >> seem fully populated. Here are the flags I used in parallel_pg_loader. >> >> perl parallel_pg_loader.pl -order bre -order mrd -order mfr -order mtfe >> -order mafe -order msfe -order mkfe -order msefe -autoprimary mrd >> -autoprimary mfr -autoprimary mtfe -autoprimary mafe -autoprimary msfe >> -autoprimary mkfe -autoprimary msefe >> >> Any ideas what I did wrong? Any ideas how to fix it? The bibs took a long >> time to load. I would rather not redo it, if I don't have to. > > If you have the intermediate bib processing files, particularly the > output of direct_ingest, make sure that file contains 'mrd' rows. For > OpenSRF 0.9 (Evergreen 1.2.x), those will start with '/*--S mrd--*/', > and in OpenSRF 1.0.x they will start with '[{"__c":"mrd","__p"'. > > If you can find that file and it contains 'mrd' lines, then we can > simply regenerate that table's worth of data. Otherwise you'll need to > reprocess the bib records in order to get that data, but you won't > need to reload the entire dataset. > > -- > Mike Rylander > | VP, Research and Design > | Equinox Software, Inc. / The Evergreen Experts > | phone: 1-877-OPEN-ILS (673-6457) > | email: [email protected] > | web: http://www.esilibrary.com > -- Mike Rylander | VP, Research and Design | Equinox Software, Inc. / The Evergreen Experts | phone: 1-877-OPEN-ILS (673-6457) | email: [email protected] | web: http://www.esilibrary.com
