Re: [classlib][nio] How to fix read-only problem? (Re: [classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel)
On 10/3/06, Andrew Zhang [EMAIL PROTECTED] wrote:
There are two ways to fix this problem:
1. Add a read-only flag in FileDescriptor. The default value is false. Set
the flag to true in FileInputStream and RandomeAccessFile(r).
2. Determine the fd in native code whether the fd is read-only.
Solution 1 judges the fd in java code, but has to remove native
from FileDescriptor#sync() signature. While solution 2 seems nature, and
solves the problem thoroughly, but requires more codes. If we adopt solution
2, we should add a new portable native mehtod like isReadOnly(int fd). Where
should we put the method? in port? or OSFileSystemWin32/Linux? Any
suggestions and comments?
Andrew,
I prefer to solution 2. IMHO, we may put the method in portlib if necessary.
Best regards,
Richard
Thanks!
On 9/19/06, Andrew Zhang [EMAIL PROTECTED] wrote:
I just found another bug of sync. Harmony throws SyncFailedException when
fd is read-only while RI returns silently. Spec doesn't explictly describe
the behaviour in such case[1]. But, it seems intended behaviour of RI,
because it requires additional check before invoke os sync. Following test
reproduces the bug:
public void testSyncReadOnly() throws Exception {
String TESTFILE = tempFile;
try {
FileOutputStream fos = new FileOutputStream(TESTFILE);
fos.write(something.getBytes());
fos.close();
RandomAccessFile raf = new RandomAccessFile(TESTFILE, rw);
raf.getFD().sync();
raf.close();
FileInputStream fis = new FileInputStream(TESTFILE);
fis.getFD().sync();
fis.close();
} finally {
new File(TESTFILE).delete();
}
}
I'll file a JIRA to record this bug soon!
[1] SyncFailedException - Thrown when the buffers cannot be flushed, or
because the system cannot guarantee that all the buffers have been
synchronized with physical media.
On 9/19/06, Richard Liang [EMAIL PROTECTED] wrote:
On 9/18/06, Andrew Zhang [EMAIL PROTECTED] wrote:
On 9/18/06, Richard Liang [EMAIL PROTECTED] wrote:
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which
is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write (ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
Interesting. I think we'd better follow RI although it's
implementation
dependent. Otherwise, it breaks existing application.
To make test more interesting, I wrote a similar test:
public void testFile() throws Exception {
File derbyLog = File.createTempFile(test, log);
derbyLog.deleteOnExit();
RandomAccessFile fos = new RandomAccessFile(derbyLog, rws);
for (int i = 0; i 1000; i++) {
fos.write(0x41);
assertEquals(1 + i, derbyLog.length());
}
}
Run it and you'll be surprised. :-)
Wow! It tooks RI 0.381 seconds, while 21.761 seconds for Harmony. We
shall improve our performance!
Let's have a more further study about this issue.
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
--
Best regards,
Andrew
Re: [classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel
I just found another bug of sync. Harmony throws SyncFailedException when fd
is read-only while RI returns silently. Spec doesn't explictly describe the
behaviour in such case[1]. But, it seems intended behaviour of RI, because
it requires additional check before invoke os sync. Following test
reproduces the bug:
public void testSyncReadOnly() throws Exception {
String TESTFILE = tempFile;
try {
FileOutputStream fos = new FileOutputStream(TESTFILE);
fos.write(something.getBytes());
fos.close();
RandomAccessFile raf = new RandomAccessFile(TESTFILE, rw);
raf.getFD().sync();
raf.close();
FileInputStream fis = new FileInputStream(TESTFILE);
fis.getFD().sync();
fis.close();
} finally {
new File(TESTFILE).delete();
}
}
I'll file a JIRA to record this bug soon!
[1] SyncFailedException - Thrown when the buffers cannot be flushed, or
because the system cannot guarantee that all the buffers have been
synchronized with physical media.
On 9/19/06, Richard Liang [EMAIL PROTECTED] wrote:
On 9/18/06, Andrew Zhang [EMAIL PROTECTED] wrote:
On 9/18/06, Richard Liang [EMAIL PROTECTED] wrote:
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write(ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
Interesting. I think we'd better follow RI although it's implementation
dependent. Otherwise, it breaks existing application.
To make test more interesting, I wrote a similar test:
public void testFile() throws Exception {
File derbyLog = File.createTempFile(test, log);
derbyLog.deleteOnExit();
RandomAccessFile fos = new RandomAccessFile(derbyLog, rws);
for (int i = 0; i 1000; i++) {
fos.write(0x41);
assertEquals(1 + i, derbyLog.length());
}
}
Run it and you'll be surprised. :-)
Wow! It tooks RI 0.381 seconds, while 21.761 seconds for Harmony. We
shall improve our performance!
Let's have a more further study about this issue.
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
Re: [classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel
On 9/18/06, Richard Liang [EMAIL PROTECTED] wrote:
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write(ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
Richard, we're fooled by derbyLog.length(). :-) That's the root of evil!
Harmony uses FindFirstFile to get file attribute, which may not be latest
information of file. MSND points out In rare cases, file attribute
information on NTFS file systems may not be current at the time you call
this function. That's why the test failed against Harmony. If using
GetFileAttributeEx instead, the test passes against Harmony. :-) But I
don't think the test is theoretically stable, since the file is not opened
with sync flag. :-) We could use RandomAccessFile(file,rwd(s)) for
test. Let's file a JIRA and fix it!
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
Re: [classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel
On 9/19/06, Andrew Zhang [EMAIL PROTECTED] wrote:
On 9/18/06, Richard Liang [EMAIL PROTECTED] wrote:
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write(ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
Richard, we're fooled by derbyLog.length(). :-) That's the root of evil!
Harmony uses FindFirstFile to get file attribute, which may not be latest
information of file. MSND points out In rare cases, file attribute
information on NTFS file systems may not be current at the time you call
this function. That's why the test failed against Harmony. If using
GetFileAttributeEx instead, the test passes against Harmony. :-) But I
don't think the test is theoretically stable, since the file is not opened
with sync flag. :-) We could use RandomAccessFile(file,rwd(s)) for
test. Let's file a JIRA and fix it!
Thank you very much, Andrew. I have raised a JIRA [1] for this issue.
[1] https://issues.apache.org/jira/browse/HARMONY-1497
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel
On 9/20/06, Richard Liang [EMAIL PROTECTED] wrote:
On 9/19/06, Andrew Zhang [EMAIL PROTECTED] wrote:
On 9/18/06, Richard Liang [EMAIL PROTECTED] wrote:
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write(ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
Richard, we're fooled by derbyLog.length(). :-) That's the root of
evil!
Harmony uses FindFirstFile to get file attribute, which may not be
latest
information of file. MSND points out In rare cases, file attribute
information on NTFS file systems may not be current at the time you call
this function. That's why the test failed against Harmony. If using
GetFileAttributeEx instead, the test passes against Harmony. :-) But
I
don't think the test is theoretically stable, since the file is not
opened
with sync flag. :-) We could use RandomAccessFile(file,rwd(s)) for
test. Let's file a JIRA and fix it!
Thank you very much, Andrew. I have raised a JIRA [1] for this issue.
Patch updated. :-)
[1] https://issues.apache.org/jira/browse/HARMONY-1497
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
[classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write(ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel
On 9/18/06, Richard Liang [EMAIL PROTECTED] wrote:
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write(ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
Interesting. I think we'd better follow RI although it's implementation
dependent. Otherwise, it breaks existing application.
To make test more interesting, I wrote a similar test:
public void testFile() throws Exception {
File derbyLog = File.createTempFile(test, log);
derbyLog.deleteOnExit();
RandomAccessFile fos = new RandomAccessFile(derbyLog, rws);
for (int i = 0; i 1000; i++) {
fos.write(0x41);
assertEquals(1 + i, derbyLog.length());
}
}
Run it and you'll be surprised. :-)
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
Re: [classlib][io][nio] Sync issue of java.io.FileOutputStream and java.nio.channels.FileChannel
On 9/18/06, Andrew Zhang [EMAIL PROTECTED] wrote:
On 9/18/06, Richard Liang [EMAIL PROTECTED] wrote:
Hello,
One Apache Derby test[1] fails on Harmony. It seems that RI always
sync the FileOutputStream and FileChannel after each write, which is
different from Harmony. But there is no explicit description in Java
Spec. Shall we follow RI? Thanks a lot.
The following test cases could demonstrate this issue.
public void testFile() {
File derbyLog = new File(d:\\, derby1.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
fos.write(0x41);
assertEquals(1, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
public void testFileChannel() {
File derbyLog = new File(d:\\, derby2.log);
try {
FileOutputStream fos = new FileOutputStream(derbyLog);
FileChannel fc = fos.getChannel();
fc.write(ByteBuffer.wrap(new byte[]{0x41, 0x42}));
assertEquals(2, derbyLog.length());
} catch (Exception e) {
e.printStackTrace();
}
}
Interesting. I think we'd better follow RI although it's implementation
dependent. Otherwise, it breaks existing application.
To make test more interesting, I wrote a similar test:
public void testFile() throws Exception {
File derbyLog = File.createTempFile(test, log);
derbyLog.deleteOnExit();
RandomAccessFile fos = new RandomAccessFile(derbyLog, rws);
for (int i = 0; i 1000; i++) {
fos.write(0x41);
assertEquals(1 + i, derbyLog.length());
}
}
Run it and you'll be surprised. :-)
Wow! It tooks RI 0.381 seconds, while 21.761 seconds for Harmony. We
shall improve our performance!
Let's have a more further study about this issue.
[1]
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/logStream.java?view=co
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Andrew Zhang
China Software Development Lab, IBM
--
Richard Liang
China Development Lab, IBM
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
