Re: [io] Tailer returning partial lines returned when EOF before newline
Hi Frank,
What I suggest you is following the usual workflow, I mean opening an
Issue on ASF Jira - see Niall's link - and build a patch against the
/trunk and attach it to the filled Issue.
Otherwise it will be very hard that IO maintainers could notice and
apply a textual patch inside the ML.
HTH, have a nice day!
Simo
http://people.apache.org/~simonetripodi/
http://www.99soft.org/
On Tue, May 24, 2011 at 10:39 PM, frankgrimes97 frankgrime...@gmail.com wrote:
The following is a patch against 2.0.1 in SVN which seems to address the
limitation:
Index: src/main/java/org/apache/commons/io/input/Tailer.java
===
--- src/main/java/org/apache/commons/io/input/Tailer.java (revision 1127267)
+++ src/main/java/org/apache/commons/io/input/Tailer.java (working copy)
@@ -335,12 +335,56 @@
* @throws java.io.IOException if an I/O error occurs.
*/
private long readLines(RandomAccessFile reader) throws IOException {
- String line = reader.readLine();
+ String line = readLine(reader);
while (line != null) {
listener.handle(line);
- line = reader.readLine();
+ line = readLine(reader);
}
return reader.getFilePointer();
}
+ /**
+ * Copied from RandomAccessFile.readLine() but returns null and resets
file pointer on EOF.
+ * @param reader
+ * @return
+ * @throws IOException
+ */
+ private final String readLine(RandomAccessFile reader) throws
IOException {
+ long start = reader.getFilePointer();
+ StringBuffer input = new StringBuffer();
+ int c = -1;
+ boolean eol = false;
+ boolean eof = false;
+
+ while (!eol !eof) {
+ switch (c = reader.read()) {
+ case -1:
+ eof = true;
+ break;
+ case '\n':
+ eol = true;
+ break;
+ case '\r':
+ eol = true;
+ long cur = reader.getFilePointer();
+ if ((reader.read()) != '\n') {
+ reader.seek(cur);
+ }
+ break;
+ default:
+ input.append((char)c);
+ break;
+ }
+ }
+
+ if ((c == -1) (input.length() == 0)) {
+ return null;
+ }
+ if (eof) {
+ reader.seek(start);
+ return null;
+ }
+
+ return input.toString();
+ }
}
Index: src/test/java/org/apache/commons/io/input/TailerTest.java
===
--- src/test/java/org/apache/commons/io/input/TailerTest.java (revision
1127267)
+++ src/test/java/org/apache/commons/io/input/TailerTest.java (working copy)
@@ -45,6 +45,38 @@
protected void tearDown() throws Exception {
FileUtils.deleteDirectory(getTestDirectory());
}
+
+ public void testTailerEof() throws Exception {
+ // Create start the Tailer
+ long delay = 50;
+ final File file = new File(getTestDirectory(), tailer2-test.txt);
+ createFile(file, 0);
+ final TestTailerListener listener = new TestTailerListener();
+ final Tailer tailer = new Tailer(file, listener, delay, false);
+ final Thread thread = new Thread(tailer);
+ thread.start();
+
+ // Write some lines to the file
+ FileWriter writer = null;
+ try {
+ writeString(file, Line);
+
+ Thread.sleep(delay * 2);
+ ListString lines = listener.getLines();
+ assertEquals(1 line count, 0, lines.size());
+
+ writeString(file, one\n);
+ Thread.sleep(delay * 2);
+ lines = listener.getLines();
+
+ assertEquals(1 line count, 1, lines.size());
+ assertEquals(1 line 1, Line one, lines.get(0));
+
+ listener.clear();
+ } finally {
+ IOUtils.closeQuietly(writer);
+ }
+ }
public void testTailer() throws Exception {
@@ -142,6 +174,17 @@
IOUtils.closeQuietly(writer);
}
}
+
+ /** Append a string to a file */
+ private void writeString(File file, String string) throws Exception {
+ FileWriter writer = null;
+ try {
+ writer = new FileWriter(file, true);
+ writer.write(string);
+ } finally {
+ IOUtils.closeQuietly(writer);
+ }
+ }
public void testStopWithNoFile() throws Exception {
final File file = new File(getTestDirectory(),nosuchfile);
On Tue, May 24, 2011 at 1:32 PM, frankgrimes97 frankgrime...@gmail.comwrote:
Hi All,
We are using org.apache.commons.io.input.Tailer to process log files for
insertion into a database.
What we are seeing is that occasionally a line fails to process because it
is incomplete.
In reviewing the code, it appears that Tailer.readLines delegates to
java.io.RandomAccessFile.readLine which returns a partial line if EOF is
reached.
Shouldn't Tailer be providing a
Re: [beanutils] Issue with ContextClassLoaderLocal class.
Thanks for the responses.
I had been thinking along those lines that the registration should be done
within the parsing module. But I am a bit curious as to know the root cause
of this problem. Is my application design flawed in registering all custom
converters during application startup or is it a result of some limitation of
beanutils?
And also how it manages to work properly in WebSphere?
Thanks.
-Niall Pemberton niall.pember...@gmail.com wrote: -
To: Commons Users List user@commons.apache.org
From: Niall Pemberton niall.pember...@gmail.com
Date: 05/24/2011 08:00PM
Subject: Re: [beanutils] Issue with ContextClassLoaderLocal class.
On Tue, May 24, 2011 at 1:55 PM, Rammohan Natarajan
rammohan.natara...@tcs.com wrote:
Hi,
We are working on migrating a J2EE application from WAS6.1 to JBoss EAP 5.0.
For unmarshalling of some custom XML we make use of commons-beanutils-1.8.3
within this application; and in the process, we have defined a custom type
converter for one of the XML tags.
The registration of this converter happens in one EJB module within our EAR,
whereas theactual XML parsing occurs in another EJB module within the same
EAR.
The org.apache.commons.beanutils.ContextClassLoaderLocal class ties 'global'
variables to a classloader instance in a map to ensure data isolation even if
the variable is referenced by multiple components running within a container.
Here is the relevant code snippet from its get() method:
ClassLoader contextClassLoader =
Thread.currentThread().getContextClassLoader();
if (contextClassLoader != null)
{
Object value = valueByClassLoader.get(contextClassLoader);
if ((value == null)
!valueByClassLoader.containsKey(contextClassLoader))
{
value = initialValue();
valueByClassLoader.put(contextClassLoader, value);
}
return value;
}
After all that background information, here is the problem:
During the converter registration process, a BeanUtilsBean instance is
maintained in the ContextClassLoaderLocal map against the classloader of the
registration EJB module.
During the parsing process, this map is queried to retreive the same
BeanUtilsBean instance, using the classloader of the parsing EJB module as
the key. In Websphere Application server, the classloader instance for the
registration module and the parser module are both same; the correct
BeanUtilsBean instance is retreived and processing proceeds as expected.
However, in JBoss EAP 5.0, the two classloader instances are different so
that the parser module fails to retreive the value from the map.
Is this a problem with beanutils' design or an issue with JBoss?
I think you need to register the converter in the module its used in -
i.e. your parsing EJB module
Niall
Please let me know if I am missing something or if additional details are
required.
Thanks in advance.
N.Rammohan.
=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org
-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org
=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org
Re: [io] Tailer returning partial lines returned when EOF before newline
FYI, I have created the following bug report in JIRA for this issue: https://issues.apache.org/jira/browse/IO-274 Frank Grimes On Tue, May 24, 2011 at 1:32 PM, frankgrimes97 frankgrime...@gmail.comwrote: Hi All, We are using org.apache.commons.io.input.Tailer to process log files for insertion into a database. What we are seeing is that occasionally a line fails to process because it is incomplete. In reviewing the code, it appears that Tailer.readLines delegates to java.io.RandomAccessFile.readLine which returns a partial line if EOF is reached. Shouldn't Tailer be providing a guarantee of complete lines? Should we create a bug report for this? FYI, we are using 1.6.0_15 on Linux. Thanks, Frank Grimes
Re: [io] Tailer returning partial lines returned when EOF before newline
Hi, Simone Tripodi wrote: Great, that's the way to go, well done and thanks for your contribution! actually I think, it's not the way to go. The Javadoc in the patch implies that Frank simply took the code from the JDK and adjusted it a bit. This is not possible! We cannot simply relicense Oracle's IP. - Jörg - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [io] Tailer returning partial lines returned when EOF before newline
I didn't check the patch, I was referring to the process. sorry for the misunderstanding http://people.apache.org/~simonetripodi/ http://www.99soft.org/ On Wed, May 25, 2011 at 3:12 PM, Jörg Schaible joerg.schai...@scalaris.com wrote: Hi, Simone Tripodi wrote: Great, that's the way to go, well done and thanks for your contribution! actually I think, it's not the way to go. The Javadoc in the patch implies that Frank simply took the code from the JDK and adjusted it a bit. This is not possible! We cannot simply relicense Oracle's IP. - Jörg - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
