Thanks Stuart, that did the trick. We are now working through the invalid
records.
For reference though, we had to enable debugging on the dspace packages as well:
log4j.logger.org.dspace=DEBUG, A1
cheers,
Steve
On 01/11/2010, at 4:27 PM, Stuart Lewis wrote:
> Hi Steve,
>
> See:
>
> -
> https://wiki.duraspace.org/display/DSPACE/TechnicalFaq#TechnicalFaq-SettingloggingleveluptoDEBUG
>
> If you set your logging to DEBUG, they will be *very* verbose, but should
> show you the item that is failing, and the SQL queries in particular that are
> causing the problem.
>
> Thanks,
>
>
> Stuart Lewis
> IT Innovations Analyst and Developer
> Te Tumu Herenga The University of Auckland Library
> Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand
> Ph: +64 (0)9 373 7599 x81928
>
>
>
> On 1/11/2010, at 4:59 PM, Steve Swinsburg wrote:
>
>> Hi,
>>
>> We've created a new dump with -E UNICODE added to the dump command, imported
>> it into a clean UTF-8 database, upgraded via the 15-16 sql update script
>> which completes successfully, but as soon as we run the index-init script
>> again, the same issue occurs. It would be helpful if the exception told us
>> which item was causing the issue.
>>
>> We ran the checks and everything is UTF-8 so I have no idea why it's still
>> failing.
>>
>> We are now looking at different way of migrating our data: export and then
>> import the collections. This is going to take quite some time which is not
>> ideal, but seems to be the only way we can get our data migrated.
>>
>> Thanks for you help,
>> Steve
>>
>>
>> On 29/10/2010, at 2:05 PM, Kim Shepherd wrote:
>>
>>> Hi Steve,
>>>
>>> As well as Keith's suggestion of inspecting your locale, I'd try a redump
>>> of the database with -E UNICODE passed to 'pg_dump', otherwise the dumped
>>> tables will still be in their original encoding.
>>>
>>> http://www.postgresql.org/docs/8.4/static/app-pgdump.html
>>>
>>> Cheers,
>>>
>>> Kim
>>>
>>> On 28 October 2010 10:46, Steve Swinsburg <steve.swinsb...@gmail.com> wrote:
>>> Hi all,
>>>
>>> We've completed another restore but are still having the same issue. This
>>> time we created a new database with the UNICODE encoding as per the
>>> installation manual:
>>> createdb -U dspace -E UNICODE dspace
>>> We then took an export of the current production database:
>>> pg_dump dspace > dspace-prod.dump
>>> and imported it into the new database:
>>> psql -d dspace -f dspace-prod.dump
>>>
>>> We then upgraded the database to 1.6:
>>>
>>> psql -U dspace -f
>>> /dspace-1.6-2-src-release/dspace/etc/postgres/database_schema_15_16.sql
>>> dspace
>>>
>>>
>>> And ran the index-init script:
>>> [dspace]/bin/dspace index-init
>>> But still the same issue as before.
>>>
>>>>> dspace index-init -v
>>>>> Exception: org.postgresql.util.PSQLException: ERROR: invalid multibyte
>>>>> character for locale
>>>>> org.dspace.browse.BrowseException: org.postgresql.util.PSQLException:
>>>>> ERROR: invalid multibyte character for locale
>>>>> at
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.insertDistinctRecord(BrowseCreateDAOPostgres.java:691)
>>>>> at
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.getDistinctID(BrowseCreateDAOPostgres.java:566)
>>>>> at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:485)
>>>>> at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>>>>> at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>>>>> at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>>>
>>>
>>> Is there anything wrong with the above process? The next step we are going
>>> to attempt is checking the actual SQL dump for the bad characters but it is
>>> massive (1.5 Gb)
>>>
>>>
>>> thanks,
>>> Steve
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 26/10/2010, at 10:10 AM, Steve Swinsburg wrote:
>>>
>>>> Hi Kim,
>>>>
>>>> We took a *copy* of the database and refreshed it back to dev, so it is
>>>> identical to the production one, then upgraded it in dev. So I'll need to
>>>> find out if the original database from production was created that way.
>>>>
>>>> Perhaps we should take a dump of the production database instead, and then
>>>> import that into a fresh UTF8 created database? I think the issue is that
>>>> the original database was never created as UTF8.
>>>>
>>>> Thanks,
>>>> Steve
>>>>
>>>>
>>>>
>>>>
>>>> On 26/10/2010, at 9:17 AM, Kim Shepherd wrote:
>>>>
>>>>> Hi Steve,
>>>>>
>>>>> Just to check, has your new/dev database been created with UTF8 character
>>>>> encoding? (pass -E UNICODE or -E UTF8 to createdb)
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Kim
>>>>>
>>>>> On 25 October 2010 14:57, Steve Swinsburg <steve.swinsb...@gmail.com>
>>>>> wrote:
>>>>> Hi all,
>>>>>
>>>>> We are having a problem upgrading an instance of dspace from 1.5.x-1.6.2.
>>>>> We built the new 1.6.2 code and took a snapshot of our old production
>>>>> database and migrated in back to our dev environment (along with the
>>>>> assets). We then ran the db upgrade script.
>>>>>
>>>>> Now, when running the index-init script we get this:
>>>>>
>>>>> ./dspace index-init -v
>>>>> Exception: org.postgresql.util.PSQLException: ERROR: invalid multibyte
>>>>> character for locale
>>>>> org.dspace.browse.BrowseException: org.postgresql.util.PSQLException:
>>>>> ERROR: invalid multibyte character for locale
>>>>> at
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.insertDistinctRecord(BrowseCreateDAOPostgres.java:691)
>>>>> at
>>>>> org.dspace.browse.BrowseCreateDAOPostgres.getDistinctID(BrowseCreateDAOPostgres.java:566)
>>>>> at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:485)
>>>>> at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>>>>> at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>>>>> at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>>>>>
>>>>> We are on Postgres:
>>>>> PostgreSQL 8.1.21 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC)
>>>>> 4.1.2 20080704 (Red Hat 4.1.2-48)
>>>>>
>>>>> Any tips?
>>>>>
>>>>> thanks,
>>>>> Steve
>>>>> ------------------------------------------------------------------------------
>>>>> Nokia and AT&T present the 2010 Calling All Innovators-North America
>>>>> contest
>>>>> Create new apps & games for the Nokia N8 for consumers in U.S. and Canada
>>>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in
>>>>> marketing
>>>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
>>>>> http://p.sf.net/sfu/nokia-dev2dev
>>>>> _______________________________________________
>>>>> DSpace-tech mailing list
>>>>> DSpace-tech@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Nokia and AT&T present the 2010 Calling All Innovators-North America contest
>>> Create new apps & games for the Nokia N8 for consumers in U.S. and Canada
>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
>>> http://p.sf.net/sfu/nokia-dev2dev
>>> _______________________________________________
>>> DSpace-tech mailing list
>>> DSpace-tech@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>
>>>
>>
>> <ATT00001..txt><ATT00002..txt>
>
>
------------------------------------------------------------------------------
Achieve Improved Network Security with IP and DNS Reputation.
Defend against bad network traffic, including botnets, malware,
phishing sites, and compromised hosts - saving your company time,
money, and embarrassment. Learn More!
http://p.sf.net/sfu/hpdev2dev-nov
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
