[
https://issues.apache.org/jira/browse/FLUME-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481973#comment-13481973
]
Brock Noland commented on FLUME-1661:
-------------------------------------
Guys, this stuff is very brittle... For example, the following tests various
commands which just by looking at them, you'd think would all output the same
thing. They are executed the same way we start the command in exec source:
{code}
public static void main(String[] args) throws Exception {
test("/bin/sh -c echo `date +%Y%m%d%H`");
test("echo `date +%Y%m%d%H`");
test("date +%Y%m%d%H");
test("/bin/sh -c date +%Y%m%d%H");
}
static void test(String cmd) throws Exception {
ProcessBuilder builder = new ProcessBuilder(cmd.split("\\s+"));
Process process = builder.start();
BufferedReader out = new BufferedReader(
new InputStreamReader(process.getInputStream()));
BufferedReader err = new BufferedReader(
new InputStreamReader(process.getErrorStream()));
Thread.sleep(10L);
System.out.println("cmd = " + cmd);
System.out.println("out = '" + readAll(out) + "'");
System.out.println("err = '" + readAll(err) + "'");
System.out.println("exit = " + process.waitFor());
}
static String readAll(BufferedReader reader) throws IOException {
String result = "";
String line;
while((line = reader.readLine()) != null) {
result += line + "\n";
}
return result;
}
{code}
Yet all four output very different things:
{code}
cmd = /bin/sh -c echo `date +%Y%m%d%H`
out = '
'
err = ''
exit = 0
cmd = echo `date +%Y%m%d%H`
out = '`date +%Y%m%d%H`
'
err = ''
exit = 0
cmd = date +%Y%m%d%H
out = '2012102219
'
err = ''
exit = 0
cmd = /bin/sh -c date +%Y%m%d%H
out = 'Mon Oct 22 19:11:59 CDT 2012
'
err = ''
exit = 0
{code}
I could go into why but I hope we don't have to go there. Building shell
commands in java is a fools game. We can simply take whatever command is passed
to us, write it to a file, and let the shell interpret it. This gives the
functionality asked for without wasting ours and our users time figuring how to
correctly quote commands which will be executed by process builder.
> ExecSource cannot execute (little complicated..) *nix commands
> --------------------------------------------------------------
>
> Key: FLUME-1661
> URL: https://issues.apache.org/jira/browse/FLUME-1661
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Yoonseok Woo
> Attachments: FLUME-1661-1.patch
>
>
> * command line parsing
> ** conf/flume.conf
> {code}
> agent.sources.source1.type = exec
> agent.sources.source1.command = tail -f
> /some/path/logs/exception/error.log.`date +%Y%m%d%H`
> {code}
> ** result
> {code}
> tail: /some/path/logs/exception/error.log.`date: No such file or directory
> tail: +%Y%m%d%H`: No such file or directory
> {code}
> ** needs to be improved
> {code}
> (ExecSouce.java:242) String[] commandArgs = command.split("\\s+")
> {code}
> * using special character (e.g. *, `, ', ...)
> ** conf/flume.conf
> {code}
> agent.sources.source1.type = exec
> agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.*
> {code}
> ** result
> {code}
> tail: /some/path/logs/exception/error.log.*: No such file or directory
> {code}
> ** needs to be improved
> {code}
> (ExecSouce.java:243) process = new ProcessBuilder(commandArgs).start();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
