[
https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831074#action_12831074
]
Aaron Kimball commented on HADOOP-6541:
---------------------------------------
Allen,
You can indeed shell escape with {{! command}}. No notion of a local working
dir, unfortunately. (Next task: lcd and lls commands!)
Currently you can't launch a Hadoop job within this shell. I think that's
another patch / separate issue -- dealing with the classloader for a jar has
enough subtleties associated with it that I punted for the first version.
Eventually this shell should be able to do all the {{hadoop jar}}, {{hadoop
job}}, {{mradmin/dfsadmin}} etc commands too, but there's a lot of
functionality to implement here. All separate tasks in my mind.
In order to establish this code base, my first cut of this system is just
focused on implementing filesystem access.
You're right that 'rmr' is a hack. A better argument parsing system is probably
necessary moving forward to support things like 'rm -[rf]', 'test', etc. Right
now all arguments are treated as filenames in commands like rm; supporting
various flags would require using CommandLineParser and other fun. Not
impossible, but not first-cut material. Also, jline's tab-completor makes it a
bit tricky to handle command-specific {{-arguments}}. I'll need to poke around
more to figure out if/how that can be done.
As for cd taking a URL -- technically it already does. But the current design
initializes a FileContext and uses its setWorkingDirectory() method to move
around, and then uses FsShell to do some operations like deletes (which require
globbing). FsShell doesn't allow reinitialization of the FileSystem, so this
code checks for changes in the default FS and reinitializes the structures it
uses. We should probably refactor FsShell soon to allow it to work on an
arbitrary FileContext instead of the one it instantiates at construction time
(or instantiate a new FsShell for each operation? That seems unnecessary). But
again, refactoring FsShell I think is beyond the scope of this issue.
All of these are great ideas but I think that they deserve separate tasks.
There are enough discussion points as to how to implement each of these that I
think a single thread would be confusing. Also getting this committed first
would allow different people to attack these problems in parallel if they'd
like. I'll file these follow-up tasks after this is resolved. Of course, feel
free to file your own in the meantime if you'd like :)
Thanks for the suggestions!
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch,
> HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a
> single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
