Problem solved after upgrading to postgres to 8.4 and re-ran the indexer!
Cheers,
Osama Alkadi
On 03/11/2010, at 1:57 PM, Steve Swinsburg wrote:
> Thanks Stuart, that did the trick. We are now working through the invalid
> records.
>
> For reference though, we had to enable debugging on the dspace packages as
> well:
> log4j.logger.org.dspace=DEBUG, A1
>
> cheers,
> Steve
>
>
> On 01/11/2010, at 4:27 PM, Stuart Lewis wrote:
>
>> Hi Steve,
>>
>> See:
>>
>> -
>> https://wiki.duraspace.org/display/DSPACE/TechnicalFaq#TechnicalFaq-SettingloggingleveluptoDEBUG
>>
>> If you set your logging to DEBUG, they will be *very* verbose, but should
>> show you the item that is failing, and the SQL queries in particular that
>> are causing the problem.
>>
>> Thanks,
>>
>>
>> Stuart Lewis
>> IT Innovations Analyst and Developer
>> Te Tumu Herenga The University of Auckland Library
>> Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand
>> Ph: +64 (0)9 373 7599 x81928
>>
>>
>>
>> On 1/11/2010, at 4:59 PM, Steve Swinsburg wrote:
>>
>>> Hi,
>>>
>>> We've created a new dump with -E UNICODE added to the dump command,
>>> imported it into a clean UTF-8 database, upgraded via the 15-16 sql update
>>> script which completes successfully, but as soon as we run the index-init
>>> script again, the same issue occurs. It would be helpful if the exception
>>> told us which item was causing the issue.
>>>
>>> We ran the checks and everything is UTF-8 so I have no idea why it's still
>>> failing.
>>>
>>> We are now looking at different way of migrating our data: export and then
>>> import the collections. This is going to take quite some time which is not
>>> ideal, but seems to be the only way we can get our data migrated.
>>>
>>> Thanks for you help,
>>> Steve
>>>
>>>
>>> On 29/10/2010, at 2:05 PM, Kim Shepherd wrote:
>>>
>>>> Hi Steve,
>>>>
>>>> As well as Keith's suggestion of inspecting your locale, I'd try a redump
>>>> of the database with -E UNICODE passed to 'pg_dump', otherwise the dumped
>>>> tables will still be in their original encoding.
>>>>
>>>> http://www.postgresql.org/docs/8.4/static/app-pgdump.html
>>>>
>>>> Cheers,
>>>>
>>>> Kim
>>>>
>>>> On 28 October 2010 10:46, Steve Swinsburg <steve.swinsb...@gmail.com>
>>>> wrote:
>>>> Hi all,
>>>>
>>>> We've completed another restore but are still having the same issue. This
>>>> time we created a new database with the UNICODE encoding as per the
>>>> installation manual:
>>>> createdb -U dspace -E UNICODE dspace
>>>> We then took an export of the current production database:
>>>> pg_dump dspace > dspace-prod.dump
>>>> and imported it into the new database:
>>>> psql -d dspace -f dspace-prod.dump
>>>>
>>>> We then upgraded the database to 1.6:
>>>>
>>>> psql -U dspace -f
>>>> /dspace-1.6-2-src-release/dspace/etc/postgres/database_schema_15_16.sql
>>>> dspace
>>>>
>>>>
>>>> And ran the index-init script:
>>>> [dspace]/bin/dspace index-init
>>>> But still the same issue as before.
>>>>
>>>>>> dspace index-init -v
>>>>>> Exception: org.postgresql.util.PSQLException: ERROR: invalid multibyte
>>>>>> character for locale
>>>>>> org.dspace.browse.BrowseException: org.postgresql.util.PSQLException:
>>>>>> ERROR: invalid multibyte character for locale
>>>>>> at
>>>>>> org.dspace.browse.BrowseCreateDAOPostgres.insertDistinctRecord(BrowseCreateDAOPostgres.java:691)
>>>>>> at
>>>>>> org.dspace.browse.BrowseCreateDAOPostgres.getDistinctID(BrowseCreateDAOPostgres.java:566)
>>>>>> at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:485)
>>>>>> at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>>>>>> at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>>>>>> at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>> at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>>>>
>>>>
>>>> Is there anything wrong with the above process? The next step we are going
>>>> to attempt is checking the actual SQL dump for the bad characters but it
>>>> is massive (1.5 Gb)
>>>>
>>>>
>>>> thanks,
>>>> Steve
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 26/10/2010, at 10:10 AM, Steve Swinsburg wrote:
>>>>
>>>>> Hi Kim,
>>>>>
>>>>> We took a *copy* of the database and refreshed it back to dev, so it is
>>>>> identical to the production one, then upgraded it in dev. So I'll need to
>>>>> find out if the original database from production was created that way.
>>>>>
>>>>> Perhaps we should take a dump of the production database instead, and
>>>>> then import that into a fresh UTF8 created database? I think the issue is
>>>>> that the original database was never created as UTF8.
>>>>>
>>>>> Thanks,
>>>>> Steve
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 26/10/2010, at 9:17 AM, Kim Shepherd wrote:
>>>>>
>>>>>> Hi Steve,
>>>>>>
>>>>>> Just to check, has your new/dev database been created with UTF8
>>>>>> character encoding? (pass -E UNICODE or -E UTF8 to createdb)
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Kim
>>>>>>
>>>>>> On 25 October 2010 14:57, Steve Swinsburg <steve.swinsb...@gmail.com>
>>>>>> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> We are having a problem upgrading an instance of dspace from
>>>>>> 1.5.x-1.6.2. We built the new 1.6.2 code and took a snapshot of our old
>>>>>> production database and migrated in back to our dev environment (along
>>>>>> with the assets). We then ran the db upgrade script.
>>>>>>
>>>>>> Now, when running the index-init script we get this:
>>>>>>
>>>>>> ./dspace index-init -v
>>>>>> Exception: org.postgresql.util.PSQLException: ERROR: invalid multibyte
>>>>>> character for locale
>>>>>> org.dspace.browse.BrowseException: org.postgresql.util.PSQLException:
>>>>>> ERROR: invalid multibyte character for locale
>>>>>> at
>>>>>> org.dspace.browse.BrowseCreateDAOPostgres.insertDistinctRecord(BrowseCreateDAOPostgres.java:691)
>>>>>> at
>>>>>> org.dspace.browse.BrowseCreateDAOPostgres.getDistinctID(BrowseCreateDAOPostgres.java:566)
>>>>>> at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:485)
>>>>>> at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>>>>>> at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>>>>>> at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>> at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>>>>>>
>>>>>> We are on Postgres:
>>>>>> PostgreSQL 8.1.21 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC)
>>>>>> 4.1.2 20080704 (Red Hat 4.1.2-48)
>>>>>>
>>>>>> Any tips?
>>>>>>
>>>>>> thanks,
>>>>>> Steve
>>>>>> ------------------------------------------------------------------------------
>>>>>> Nokia and AT&T present the 2010 Calling All Innovators-North America
>>>>>> contest
>>>>>> Create new apps & games for the Nokia N8 for consumers in U.S. and
>>>>>> Canada
>>>>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in
>>>>>> marketing
>>>>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
>>>>>> http://p.sf.net/sfu/nokia-dev2dev
>>>>>> _______________________________________________
>>>>>> DSpace-tech mailing list
>>>>>> DSpace-tech@lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>>>>
>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Nokia and AT&T present the 2010 Calling All Innovators-North America
>>>> contest
>>>> Create new apps & games for the Nokia N8 for consumers in U.S. and Canada
>>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in
>>>> marketing
>>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
>>>> http://p.sf.net/sfu/nokia-dev2dev
>>>> _______________________________________________
>>>> DSpace-tech mailing list
>>>> DSpace-tech@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>>
>>>>
>>>
>>> <ATT00001..txt><ATT00002..txt>
>>
>>
>
> ------------------------------------------------------------------------------
> Achieve Improved Network Security with IP and DNS Reputation.
> Defend against bad network traffic, including botnets, malware,
> phishing sites, and compromised hosts - saving your company time,
> money, and embarrassment. Learn More!
> http://p.sf.net/sfu/hpdev2dev-nov_______________________________________________
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
