Hi,
I am running into the same issue below.
I am using ERXJDBCUtilities.executeUpdateScriptFromResourceNamed() to insert
few rows in the DB. The DB is using UTF-8 Encoding. File containing the SQL
statements is using UTF-8 as well. The problem is that a value like "Québec" is
not inserted properly using the
ERXJDBCUtilities.executeUpdateScriptFromResourceNamed(). The same fine inserts
the data properly if executed directly using Postgres command line.
I tried to modify the file.encoding Property with no luck.
I tried the UTF-16 encoding: I change the file encoding (made sure to tell
Eclipse about the UTF-16 encoding). Now, the migration fails
ERXJDBCUtilities.executeUpdateScriptFromResourceNamed() throws an exception:
Caused by: java.lang.RuntimeException: Failed to execute 'ˇ˛INSERT INTO
entity_status(status_name) values ('ACTIVE')'.
at
er.extensions.jdbc.ERXJDBCUtilities.executeUpdateScript(ERXJDBCUtilities.java:660)
at
er.extensions.jdbc.ERXJDBCUtilities.executeUpdateScript(ERXJDBCUtilities.java:614)
at
er.extensions.jdbc.ERXJDBCUtilities.executeUpdateScriptFromResourceNamed(ERXJDBCUtilities.java:734)
at ca.fusiondev.migration.Peripheral0.upgrade(Peripheral0.java:581)
at
er.extensions.migration.ERXMigrationDatabase$Migration.upgrade(ERXMigrationDatabase.java:444)
at
er.extensions.migration.ERXMigrator$ERXMigrationAction.doPerform(ERXMigrator.java:473)
... 16 more
Caused by: org.postgresql.util.PSQLException: ERROR: insufficient data left in
message
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:299)
at
er.extensions.jdbc.ERXJDBCUtilities.executeUpdateScript(ERXJDBCUtilities.java:656)
... 21 more
However setting the JVM parameter file.encoding=UTF-8 in the command line
solves my issue and data are inserted properly.
I am wondering if somebody knows about a better solution for this issue.
Thanks,
Mahdi
Begin forwarded message:
> From: Johann Werner <[email protected]>
> Date: July 20, 2009 1:02:17 PM GMT+03:00
> To: WebObjects-Dev Mailing List List <[email protected]>
> Subject: Re: Migrations encoding
>
> Hi,
>
> I just came across the same problem. I have an UTF-8 encoded file that should
> be executed during migrations. On my development machine everything works as
> expected but on the deployment machine unicode characters get garbled. After
> some debugging I found the difference between both machines (development is
> 10.5.7 Client, deployment 10.5.7 Server): the java system property
> file.encoding is UTF-8 on my development machine and on the server it is set
> to MacRoman. I don't know why this setting is different nor how I can change
> this default.
>
> By adding a -Dfile.encoding=UTF-8 to the JVM arguments everything is ok but
> this means that either I have to add this in JavaMonitor or add it to the
> generated WOA-startup script. Both things you can easily forget and screw up
> your migrations. So another solution I tried is to add
>
> System.setProperty("file.encoding", "UTF-8"); (1)
>
> to the application constructor. This seems to remedy the situation but
> unfortunately the server showed the same garbled string :( By stepping
> through the code the line
>
> return new String(ERXFileUtilities.bytesFromInputStream(in)); (2)
>
> in ERXStringUtilities.stringFromInputStream calls
>
> String->StringCoding.decode->Converters.getDefaultEncodingName
>
> which returns again MacRoman. Could it be that setting the system property
> (1) does not alter the value globally? How are you assuring that the file
> encoding defaults to UTF-8 in your WO applications? Is there a special trick?
> Is changing the startup script in the ant build the way to go (so should it
> possibly be made the default in WOLips)?
>
> A temporary patch would be to change the line (2) to
>
> return new String(ERXFileUtilities.bytesFromInputStream(in),
> Charset.defaultCharset());
>
> jw
>
>
> Am 15.06.2009 um 20:45 schrieb Tusker:
>
>> Yup. I made sure it was using bbedit. I read in the sql file
>> manually(using FileInputStream with UTF-16 encoding) and inserted it into
>> the database via editingcontext with no problem at all. How else can I make
>> sure it's encoded properly?
>>
>>
>> On Jun 14, 2009, at 12:20 PM, Chuck Hill wrote:
>>
>>>
>>> On Jun 12, 2009, at 10:59 AM, Tusker wrote:
>>>
>>>> Hi,
>>>>
>>>> Same problem. I'm actually using a *.sql file within the Resources
>>>> folder. It's been working great until I got to unicode characters.
>>>>
>>>> I'm using this to call the *.sql. I've made sure that it is encoded
>>>> (UTF-16). Also tried (UTF-8)
>>>
>>>
>>> You made sure it was encoded in UTF-16 or you just told Eclipse that it
>>> was? I don't think that Eclipse will re-encode the file if you do that.
>>>
>>>
>>> Chuck
>>>
>>>
>>>>
>>>> ERXJDBCUtilities.executeUpdateScriptFromResourceNamed(channel,
>>>> "mysqltest.sql", null);
>>>>
>>>> I tried changing the name to "name". Works fine until I introduce
>>>> unicode characters.
>>>>
>>>> Somewhere between reading the file to execution, the encoding is not right.
>>>>
>>>> This is what ends up in the database:
>>>>
>>>> Belgium/België/Belgique
>>>>
>>>> Thanks
>>>>
>>>>
>>>> On Jun 12, 2009, at 10:34 AM, Jon Nolan wrote:
>>>>
>>>>> Tusker wrote:
>>>>>> I changed the file to be UTF-16 but I get an error now. I use the same
>>>>>> sql file and manually run it in frontbase, it works. Example of the
>>>>>> statement:
>>>>>>
>>>>>> update Country set name = 'Belgium/België/Belgique' where country_id = 2;
>>>>>>
>>>>> "name" is a reserved word in Frontbase although FBManager (not sure about
>>>>> sql92) can often/usually/always handle non-quoted reserved words
>>>>> depending on how you're using them. I've always had to double-quote
>>>>> reserved words when working in Migrations though. i.e.
>>>>>
>>>>> ERXJDBCUtilities.executeUpdate(database.adaptorChannel(), "update SITE
>>>>> set \"TYPE\" = 1 where \"TYPE\" = 10");
>>>>>
>>>>>
>>>>> Of course it might still be an encoding issue but let's rule out this
>>>>> potential problem first. Try..
>>>>>
>>>>>
>>>>> update Country set "name" = 'Belgium/België/Belgique' where country_id =
>>>>> 2;
>>>>> _______________________________________________
>>>>> Do not post admin requests to the list. They will be ignored.
>>>>> Webobjects-dev mailing list ([email protected])
>>>>> Help/Unsubscribe/Update your Subscription:
>>>>> http://lists.apple.com/mailman/options/webobjects-dev/tusker%40me.com
>>>>>
>>>>> This email sent to [email protected]
>>>>
>>>> _______________________________________________
>>>> Do not post admin requests to the list. They will be ignored.
>>>> Webobjects-dev mailing list ([email protected])
>>>> Help/Unsubscribe/Update your Subscription:
>>>> http://lists.apple.com/mailman/options/webobjects-dev/chill%40global-village.net
>>>>
>>>> This email sent to [email protected]
>>>
>>> --
>>> Chuck Hill Senior Consultant / VP Development
>>>
>>> Come to WOWODC'09 in San Fran this June!
>>> http://www.wocommunity.org/wowodc09/
>>>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Webobjects-dev mailing list ([email protected])
> Help/Unsubscribe/Update your Subscription:
> http://lists.apple.com/mailman/options/webobjects-dev/mmankai%40protocolis.ca
>
> This email sent to [email protected]
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list ([email protected])
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to [email protected]
