Thanks Adam and Oleg. My case is similar to Adam's. NiFi is running on a node, that doesn’t have Hadoop client configured. It have to invoke a script on other edge node that has Hadoop client setup.
I think you guys may have configured password less login for SSH (keys?) In my case the edge node is managed by different team and they don’t allow me to add my SSH key. I am thinking we need ExecuteRemoteCommand processor (based on https://github.com/int128/groovy-ssh) that will take care of key or password base SSH login. ExecuteRemoteCommand should have configurable attributes and return command output as flowfile host : Hostname or IP address. port : Port. Defaults to 22. user : User name. password: A password for password authentication. identity : A private key file for public-key authentication. execute - Execute a command. executeBackground - Execute a command in background. executeSudo - Execute a command with sudo support. shell - Execute a shell. If there is enough interest, I can code this extension :) -Sumo > On Nov 24, 2015, at 10:28 AM, Adam Taft <a...@adamtaft.com> wrote: > > Right. As an example, I'm currently using ExecuteCommand to transfer data > over an SSH pipe. I'm actually transferring data from a NiFi dataflow* to > an HDFS cluster, using something like: > > Command Path: bash > Command Arguments: -c; ssh $host 'hadoop fs -appendToFile - > /path/to/hdfs/file' > > For some reason (that I don't remember), I liked "bash -c" better than > calling ssh straight in the "Command Path" property. I think maybe it had > a better environment configuration that I needed. You might be able to > just call ssh directly. > > * note, the nifi in question isn't directly on the hdfs network segment, so > this was an easy/quick way to transfer data into hdfs from outside. > > ** ssh -X is not required, contrary to Oleg's comment. The -X option is > for forwarding X Windows sessions, probably not what you need. > > > On Tue, Nov 24, 2015 at 12:06 PM, Oleg Zhurakousky < > ozhurakou...@hortonworks.com> wrote: > >> Sumo >> >> You may also want to consider ExecuteCommand processor. Your command could >> be ‘ssh’ with ‘-X’ option which would allow you to invoke remote process >> over SSH. >> Not necessarily sure it would address your use case fully, but give it a >> shot and see what happens. May be you would discover some more details that >> you can feed back that would eventually lead into a first class support for >> such case. >> >> Cheers >> Oleg >> >>> On Nov 24, 2015, at 11:51 AM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Hello Sumo, >>> >>> At present there are no such processors to do this. I know it has >>> been done in the past but that was not in an open source environment >>> so don't have anything to show for it. >>> >>> It could be a great contrib. >>> >>> Thanks >>> Joe >>> >>> On Tue, Nov 24, 2015 at 11:32 AM, Sumanth Chinthagunta >>> <xmlk...@gmail.com> wrote: >>>> Are there any NiFi processors to execute remote commands via SSH? >>>> >>>> I need to SSH to a remoter server and run a shell script on schedule >> basses. >>>> thinking of using NiFi’s scheduling and argument passing capability. >>>> >>>> I find this lib can be used, if no such processor exist. >>>> https://github.com/int128/groovy-ssh < >> https://github.com/int128/groovy-ssh> >>>> >>>> -Sumo >>> >> >>
