[
https://issues.apache.org/jira/browse/EDGENT-306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645258#comment-15645258
]
Dale LaBossiere commented on EDGENT-306:
----------------------------------------
I suspect there are still some misunderstandings as there is no absolute
requirement that directoryWatcher()/textFileReader() can only be used for files
created with textFileWriter().
At this time textFileReader() only supports files whose content is static.
i.e., textFileReader() only makes a single pass reading a file's content and
converting it to tuples. As noted in its javadoc "A file is not subsequently
monitored for additional lines". So, perhaps your code ended up providing
textFileReader() with a pathname of a file whose contents was not yet
"complete" (as it is in the "pre-existing" case)?
I thought there might already be a jira for a "hot" textFileReader() capability
but didn't notice one - i.e. a textFileReader() that can function like "tail -f
...", watching and reporting a file's content that's being incrementally added.
In the absence of hot-textFileReader() once could use
Connectors.CommandStreams (e.g., CommandStreams.generate(...)) to create a
long-running "tail -f" command and convert its output to tuples.
directoryWatcher() generates a file name in its output stream only once -- when
the file is first noticed. This is subject to platform's JRE
java.nio.file.WatchService behavior as noted in the directoryWatcher() javadoc.
As noted in the javadoc, directoryWatcher() ignores "hidden" files --
File.isHidden()==true.
textFileWriter() / FileWriterPolicy creates files in a manner that works with
the above. i.e., by default, a file that's under construction, an "active
file", has a "hidden file" name. Only once the file's content is "complete" is
the file "cycled" and renamed to a non-hidden file that directoryWatcher() will
see/report and hence when textFileReader() receives the pathname the file is
complete and will be fully processed.
I hope that helps clarify things.
In light of that information should/can this jira be closed as a "usage
mistake" or do you believe there is a bug present?
> With edgent Tester, failed to read file content from the newly appeared files
> from a directory
> -----------------------------------------------------------------------------------------------
>
> Key: EDGENT-306
> URL: https://issues.apache.org/jira/browse/EDGENT-306
> Project: Edgent
> Issue Type: Bug
> Components: Test
> Affects Versions: 0.4.0
> Reporter: Rui Shen
> Attachments: CopyOfFileStreamsTest.java, FileReaderApp.java, M2.png
>
>
> I got a trouble when using edgent Tester. The scenario that I am trying to
> test is:
> 1) There is a empty directory (e.g "testdata")
> 2) Use FileStreams.directoryWatcher to watch the directory "testdata" and use
> FileStreams.textFileReader to read content of newly appeared files in the
> directory.
>
> I am trying to use edgent Tester to write a unit test of this scenario (I
> checked org.apache.edgent.test.connectors.file.FileStreamsTest and didn't
> find such scenario). But I found my unit test always failed at reading the
> content of newly appeared files. But if the files are pre-existing, it works.
>
> I attached two files. To reproduce this issue, you can run
> testDirectoryWatcherOrder in CopyOfFileStreamsTest.java. And you will see the
> following error. FileReaderApp.java is a standard alone version of the above
> test scenario - it works well (once you start it, generate some files with
> content in the directory, you can see the file content are printed in
> console).
>
> So could you please let me know if this is a defect of edgent Tester or my
> usage mistake?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
