[
https://issues.apache.org/jira/browse/FLUME-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269499#comment-13269499
]
[email protected] commented on FLUME-1174:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5046/#review7628
-----------------------------------------------------------
Hi Alexander,
Since the goal of FLUME-1174 is to attempt to build Flume NG on Windows, it
looks like there are 2 issues:
(1) There's no 'cat' command on Windows (if we only consider DOS commands, not
cygwin)
(2) There's no /etc/password on Windows
It seems that the patch is intended to address only #2, so I'll focus on that
issue. Instead of 'cat /etc/password', we can try substituting 'cat' with
'echo' so that we don't need to access any data files on the filesystem at all
during the test, and we'd still be testing the functionality of ExecSource. For
example:
final Queue<String> input = Lists.newLinkedList();
input.add("event1");
input.add("event2");
input.add("event3");
final String cmdArg = Joiner.on("\\n").join(input);
context.put("command", "echo -ne " + cmdArg);
...and then in the loop we can have something like this:
//Read all events from channel. Expect exactly 3 events.
for (int i = 0; i < 3; i++) {
Event event = c.take();
assertNotNull("Channel is missing at least one event", event);
assertTrue("Channel contains more events than expected",
!input.isEmpty());
assertEquals("Input event doesn't match output event", input.remove(),
new String(event.getBody(), Charsets.UTF_8));
}
assertTrue(c.take() == null);
assertTrue(input.isEmpty());
Thoughts?
To address #1, one might try just using the "echo" command (w/o args) since
echo is also a DOS cmd, and the reason we'd need to take-out the "-ne" options
is because the Windows echo DOS cmd doesn't have those options. This means that
we'd be echoing only a single line which would result in only one event, and
that wouldn't test the event-splitting newline-delimited nature of ExecSource.
But at least the proposed patch would address issue #2.
Cheers,
Will
- Will
On 2012-05-06 13:21:54, Alexander Alten-Lorenz wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5046/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-06 13:21:54)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. FLUME-1174 describes an error when Flume is built on Windows machines. The
error occurs through hardcoded test cases which assume a /etc/passwd file and a
hardcoded /tmp path. As revealed in the Jira that could be an security issue.
On top it makes it impossible to build Flume on non standard Linux operating
systems (including Busy Box, grml, Windows etc).
bq.
bq.
bq. This addresses bug FLUME-1174.
bq. https://issues.apache.org/jira/browse/FLUME-1174
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java
615f2a3
bq.
bq. Diff: https://reviews.apache.org/r/5046/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. [INFO]
------------------------------------------------------------------------
bq. [INFO] Reactor Summary:
bq. [INFO]
bq. [INFO] Apache Flume ...................................... SUCCESS [1.575s]
bq. [INFO] Flume NG SDK ...................................... SUCCESS
[20.774s]
bq. [INFO] Flume NG Configuration ............................ SUCCESS [0.261s]
bq. [INFO] Flume NG Core ..................................... SUCCESS
[1:45.541s]
bq. [INFO] Flume NG Sinks .................................... SUCCESS [0.052s]
bq. [INFO] Flume NG HDFS Sink ................................ SUCCESS
[34.842s]
bq. [INFO] Flume NG IRC Sink ................................. SUCCESS [0.220s]
bq. [INFO] Flume NG Channels ................................. SUCCESS [0.204s]
bq. [INFO] Flume NG JDBC channel ............................. SUCCESS
[41.959s]
bq. [INFO] Flume NG Node ..................................... SUCCESS
[18.442s]
bq. [INFO] Flume NG file-based channel ....................... SUCCESS [0.756s]
bq. [INFO] Flume NG file backed Memory channel ............... SUCCESS
[22.213s]
bq. [INFO] Flume legacy Sources .............................. SUCCESS [0.045s]
bq. [INFO] Flume legacy Avro source .......................... SUCCESS [1.678s]
bq. [INFO] Flume legacy Thrift Source ........................ SUCCESS [0.716s]
bq. [INFO] Flume NG Clients .................................. SUCCESS [0.039s]
bq.
bq.
bq. Thanks,
bq.
bq. Alexander
bq.
bq.
> Attempt to build Flume v1.1.0 on Windows 7 x64 fails
> ----------------------------------------------------
>
> Key: FLUME-1174
> URL: https://issues.apache.org/jira/browse/FLUME-1174
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.1.0
> Environment: Windows 7 - 64 Bit - jdk1.7u4 - mvn 3.0.4
> Reporter: Jeff Lord
> Assignee: Alexander Lorenz-Alten
>
> It looks as if the src is expecting to be built on Linux.
> Attempting to build on Windows 7 fails with the below error that references a
> missing /etc/passwd file.
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.156 sec
> Running org.apache.flume.TestCounterGroup
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 sec
> Results :
> Failed tests: testRestart(org.apache.flume.source.TestExecSource)
> Tests in error:
> test(org.apache.flume.serialization.TestBodyTextEventSerializer): Unable to
> de
> lete file: src\test\resources\events.txt
> testProcess(org.apache.flume.source.TestExecSource): \etc\passwd (The
> system c
> annot find the path specified)
> Tests run: 114, Failures: 1, Errors: 2, Skipped: 2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
