Thanks Stuart, that did the trick. We are now working through the invalid 
records.

For reference though, we had to enable debugging on the dspace packages as well:
log4j.logger.org.dspace=DEBUG, A1

cheers,
Steve


On 01/11/2010, at 4:27 PM, Stuart Lewis wrote:

> Hi Steve,
> 
> See:
> 
> - 
> https://wiki.duraspace.org/display/DSPACE/TechnicalFaq#TechnicalFaq-SettingloggingleveluptoDEBUG
> 
> If you set your logging to DEBUG, they will be *very* verbose, but should 
> show you the item that is failing, and the SQL queries in particular that are 
> causing the problem.
> 
> Thanks,
> 
> 
> Stuart Lewis
> IT Innovations Analyst and Developer
> Te Tumu Herenga The University of Auckland Library
> Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand
> Ph: +64 (0)9 373 7599 x81928
> 
> 
> 
> On 1/11/2010, at 4:59 PM, Steve Swinsburg wrote:
> 
>> Hi,
>> 
>> We've created a new dump with -E UNICODE added to the dump command, imported 
>> it into a clean UTF-8 database, upgraded via the 15-16 sql update script 
>> which completes successfully, but as soon as we run the index-init script 
>> again, the same issue occurs. It would be helpful if the exception told us 
>> which item was causing the issue.
>> 
>> We ran the checks and everything is UTF-8 so I have no idea why it's still 
>> failing.
>> 
>> We are now looking at different way of migrating our data: export and then 
>> import the collections. This is going to take quite some time which is not 
>> ideal, but seems to be the only way we can get our data migrated.
>> 
>> Thanks for you help,
>> Steve
>> 
>> 
>> On 29/10/2010, at 2:05 PM, Kim Shepherd wrote:
>> 
>>> Hi Steve,
>>> 
>>> As well as Keith's suggestion of inspecting your locale, I'd try a redump 
>>> of the database with -E UNICODE passed to 'pg_dump', otherwise the dumped 
>>> tables will still be in their original encoding.
>>> 
>>> http://www.postgresql.org/docs/8.4/static/app-pgdump.html
>>> 
>>> Cheers,
>>> 
>>> Kim
>>> 
>>> On 28 October 2010 10:46, Steve Swinsburg <steve.swinsb...@gmail.com> wrote:
>>> Hi all,
>>> 
>>> We've completed another restore but are still having the same issue. This 
>>> time we created a new database with the UNICODE encoding as per the 
>>> installation manual:
>>> createdb -U dspace -E UNICODE dspace
>>> We then took an export of the current production database:
>>> pg_dump dspace > dspace-prod.dump
>>> and imported it into the new database:
>>> psql -d dspace -f dspace-prod.dump
>>> 
>>> We then upgraded the database to 1.6:
>>> 
>>> psql -U dspace -f 
>>> /dspace-1.6-2-src-release/dspace/etc/postgres/database_schema_15_16.sql 
>>> dspace
>>> 
>>> 
>>> And ran the index-init script:
>>> [dspace]/bin/dspace index-init
>>> But still the same issue as before.
>>> 
>>>>> dspace index-init -v
>>>>> Exception: org.postgresql.util.PSQLException: ERROR: invalid multibyte 
>>>>> character for locale
>>>>> org.dspace.browse.BrowseException: org.postgresql.util.PSQLException: 
>>>>> ERROR: invalid multibyte character for locale
>>>>>       at 
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.insertDistinctRecord(BrowseCreateDAOPostgres.java:691)
>>>>>       at 
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.getDistinctID(BrowseCreateDAOPostgres.java:566)
>>>>>       at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:485)
>>>>>       at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>>>>>       at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>>>>>       at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>>>>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>       at 
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>       at 
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>       at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>       at 
>>>>> org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>>> 
>>> 
>>> Is there anything wrong with the above process? The next step we are going 
>>> to attempt is checking the actual SQL dump for the bad characters but it is 
>>> massive (1.5 Gb)
>>> 
>>> 
>>> thanks,
>>> Steve
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 26/10/2010, at 10:10 AM, Steve Swinsburg wrote:
>>> 
>>>> Hi Kim,
>>>> 
>>>> We took a *copy* of the database and refreshed it back to dev, so it is 
>>>> identical to the production one, then upgraded it in dev. So I'll need to 
>>>> find out if the original database from production was created that way.
>>>> 
>>>> Perhaps we should take a dump of the production database instead, and then 
>>>> import that into a fresh UTF8 created database? I think the issue is that 
>>>> the original database was never created as UTF8.
>>>> 
>>>> Thanks,
>>>> Steve
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 26/10/2010, at 9:17 AM, Kim Shepherd wrote:
>>>> 
>>>>> Hi Steve,
>>>>> 
>>>>> Just to check, has your new/dev database been created with UTF8 character 
>>>>> encoding? (pass -E UNICODE or -E UTF8 to createdb)
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Kim
>>>>> 
>>>>> On 25 October 2010 14:57, Steve Swinsburg <steve.swinsb...@gmail.com> 
>>>>> wrote:
>>>>> Hi all,
>>>>> 
>>>>> We are having a problem upgrading an instance of dspace from 1.5.x-1.6.2. 
>>>>> We built the new 1.6.2 code and took a snapshot of our old production 
>>>>> database and migrated in back to our dev environment (along with the 
>>>>> assets). We then ran the db upgrade script.
>>>>> 
>>>>> Now, when running the index-init script we get this:
>>>>> 
>>>>> ./dspace index-init -v
>>>>> Exception: org.postgresql.util.PSQLException: ERROR: invalid multibyte 
>>>>> character for locale
>>>>> org.dspace.browse.BrowseException: org.postgresql.util.PSQLException: 
>>>>> ERROR: invalid multibyte character for locale
>>>>>       at 
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.insertDistinctRecord(BrowseCreateDAOPostgres.java:691)
>>>>>       at 
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.getDistinctID(BrowseCreateDAOPostgres.java:566)
>>>>>       at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:485)
>>>>>       at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>>>>>       at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>>>>>       at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>>>>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>       at 
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>       at 
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>       at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>       at 
>>>>> org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>>>>> 
>>>>> We are on Postgres:
>>>>> PostgreSQL 8.1.21 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 
>>>>> 4.1.2 20080704 (Red Hat 4.1.2-48)
>>>>> 
>>>>> Any tips?
>>>>> 
>>>>> thanks,
>>>>> Steve
>>>>> ------------------------------------------------------------------------------
>>>>> Nokia and AT&T present the 2010 Calling All Innovators-North America 
>>>>> contest
>>>>> Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
>>>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in 
>>>>> marketing
>>>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
>>>>> http://p.sf.net/sfu/nokia-dev2dev
>>>>> _______________________________________________
>>>>> DSpace-tech mailing list
>>>>> DSpace-tech@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>>> 
>>>> 
>>> 
>>> 
>>> ------------------------------------------------------------------------------
>>> Nokia and AT&T present the 2010 Calling All Innovators-North America contest
>>> Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
>>> http://p.sf.net/sfu/nokia-dev2dev
>>> _______________________________________________
>>> DSpace-tech mailing list
>>> DSpace-tech@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>> 
>>> 
>> 
>> <ATT00001..txt><ATT00002..txt>
> 
> 

------------------------------------------------------------------------------
Achieve Improved Network Security with IP and DNS Reputation.
Defend against bad network traffic, including botnets, malware, 
phishing sites, and compromised hosts - saving your company time, 
money, and embarrassment.   Learn More! 
http://p.sf.net/sfu/hpdev2dev-nov
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Reply via email to