All: I just pushed an extremely early stage project called fbus[1] to github. This project is meant to facilitate moving complete files between various configurable end points including HDFS. Spring and Spring Integration are used as the underbelly and allow users to wire up channels and move files around. Current tested end point wiring:
* Local -> HDFS * Local -> Local The plan is to extend this to be bidirectional and incorporate various other management features. Spring Integration was chosen so that additional end points may be used (FTP, SSH transports, etc.) and other nice messaging features can be used (wire tapping, transformation, content based routing, etc.). Warning: This is very early stage and not production quality. I wanted to throw this out there for those who are interested. Potential fun uses: * Fire events over JMS or AMQP as files arrive to update management UIs. * Monitor file movement. * Implement a standard for file queuing and configuration. * Batch and aggregate files on a schedule. This project is for fun. If people find it useful / interesting, I'll continue it. Feedback, bug reports, feature requests, and help are all welcome. [1] http://github.com/esammer/fbus -- Eric Sammer phone: +1-917-287-2675 twitter: esammer data: www.cloudera.com
