Hi folks, We have a requirement in one of our time critical application wherein we are looking for transferring upto 40-50 GBs worth images within few seconds between remote machine and HDFS.
Assuming network connectivity between the two is on a 10Gbe link and NIC and socket buffers tuned optimally to give best performance , does Nifi have a capability to support desired performance using a combination of "getFile" and "putHDFS" on a high ended cluster of >8 nodes. We are also exploring a combination of HDFS+GrdiFTP for fast transfer of images from remote machine to HDFS cluster. Any thoughts or pointers shall be helpful. Thanks!!
