[
http://issues.apache.org/jira/browse/DERBY-877?page=comments#action_12364296 ]
Kathey Marsden commented on DERBY-877:
--------------------------------------
Here is a summary if this issue. I will submit a patch next week. Please read
and comment if you have time. To repro on windows with jdk 1.5:
java -Dfile.encoding=UTF-16 -DConsole.encoding=Cp1252 TestEnc
PROBLEM:
Encoding issues with getDate(), getTime, and getTimeStamp() when the JVM
encoding does not match server encoding. Failures are manifested with various
exceptions indicating the data is not formatted properly.
BACKGROUND:
In DRDA, the rule for data encoding for result set data values is that the
receiver makes right. So if the server sends a column with Ccsid 1208 (UTF-8),
the client is responsible for handling that format correctly. Alternately if
the client sends a parameter, the server is responsible for handling the
encoding properly.
Network Server always sends data in UTF-8 (CCSID- 1208) format, so the client
really only needs to handle that for result set data.
Note: DDM String values are always EBCDIC (but that seems to be handled
properly)
PROBLEM:
Methods in org.apache.derby.client.am.DateTime.java make the assumption that
the default encoding of the JVM matches the encoding of the date/time/timestamp
value for the characters sent in those values. For example, the method
*timeStampBytesToTimestamp* takes a byte[] buffer as sent by the server and
returns a java.sql.Timestamp, computing values like this:
String date = new String(buffer, offset,
DateTime.dateRepresentationLength);
[snip]
int zeroBase = ((int) '0');
year =
1000 * (((int) date.charAt(yearIndx)) - zeroBase) + ...
When the default encoding for the JVM is not compatible with the encoding sent
by Network Server, (for example Cp500 or Cp037 are EBCDIC), zero base is
caluculated incorectly, thus the resulting values are garbage.
Other methods like *timestampToTimestampBytes* go the exact reverse. They
build up a character array and then turn it into a String and do getBytes()
without specifying the encoding. It seems that these would affect
setTimestamp/setTime/setDate calls, but I will file this as a separate issue
once I have a repro.
One other issue regarding DateTime.java is that it uses deprecated methods
such as Date.setYear() deprecated since jdk 1.1.
FIX
As a fix for 10.1 I am thinking the solution is to pass the encoding of the
column into the methods in DateTime.java that create Strings and then create
the string with the column encoding.
e.g
*timeStampBytesToTimestamp* will have
String date = new String(buffer, offset,
DateTime.dateRepresentationLength,encoding);
Then the math will be correct.
I think longer term we should reconsider DateTime.java altogether and consider
how much all of this character arithmetic is buying us, especially give it is
relying on deprecated methods etc.
TESTING
I can use Deepa's trick with jdk 1.5 to reproduce the issue, but hope we can
find a more generic way to test these encoding issues so that they can be
encluded in the nightly regressions.
java -Dfile.encoding=UTF-16 -DConsole.encoding=Cp1252 TestEnc
OTHER AREAS OF CONCERN
I looked briefly at the code for additional potential encoding problems. I saw
these possible issues.
DateTime.java
Need to look into setTime/setDate/setTimestamp cases. I have not seen errors
so maybe they are ok.
Decimal.java, FloatingPoint.java
These converters need closer examination.
Clob.java
Has an encoding_ field but it looks like it is not used.
NetConnectionReply.java
has a getBytes() call that needs consideration.
Cursor.java
In getVARCHAR there is code like this that I don't quite understand. If the
server uses Ccsid 1208, why do we call getStringWithoutConvert for 1200?
if(ccsid_[column-1] == 1200)
return getStringWithoutConvert (columnDataPosition_[column-1] + 2,
columnDataComputedLength_[column-1] - 2, fdocaLength_[column-1]);
> zOS - with DerbyClient getDate(#) fails with IllegalArgumentException -
> unsupported date format - resultset.java
> ----------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-877
> URL: http://issues.apache.org/jira/browse/DERBY-877
> Project: Derby
> Type: Bug
> Components: Network Client
> Versions: 10.1.1.0
> Environment: OS/390 with classic (IBM) jvm142
> Reporter: Myrna van Lunteren
> Assignee: Kathey Marsden
> Attachments: TestEnc.java
>
> The test lang/resultset.java fails with DerbyNetClient on zOS
> because ResultSet.getDate(#) fails with an
> java.lang.IllegalArgumentException - unsupported date format.
> This is the stack trace with 10.2 debug version (but it fails
> with 10.1 also):
> ------------------
> ....
> getBytes(dt) got exception
> Data Conversion SQLException
> FAIL -- unexpected exception:
> java.lang.IllegalArgumentException: Unsupported date format!
> java.lang.IllegalArgumentException: Unsupported date format!
> at
> org.apache.derby.client.am.DateTime.dateBytesToDate(DateTime.java:63)
> at
> org.apache.derby.client.am.Cursor.getDATE(Cursor.java:400)
> at
> org.apache.derby.client.am.Cursor.getDate(Cursor.java:712)
> at
> org.apache.derby.client.am.ResultSet.getDate(ResultSet.java:687)
> at
> org.apache.derbyTesting.functionTests.tests.jdbcapi.resultset.main(Unknown
> Source)
> ------------------
> Note: does not fail with jcc.
> Also, test lang/updatableResultSet.java failed with e.g.:
> - instead of 'Got expected exception : Illegal Conversion' :
> 'Got expected exception : Unsupported date format!' .
> - instead of 'Got expected exception : Illegal Conversion' :
> 'Got expected exception : nanos > 99999999999 or < 0' .
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
