Re: Nifi Capability for Fast transfer of Data

2016-11-25 Thread Lee Laim
Shweta, While this may deviate from your initial requirements, NiFi offers the ability to compress, resize, and extract metadata from your images. You can use NiFi to build a image-processing pipeline for incoming images to prioritize and route ~10% of images data that needs to arrive in 4 se

Re: Nifi Capability for Fast transfer of Data

2016-11-25 Thread Andy LoPresto
Unless my back of the envelope math is way off, to transfer 50GB (400Gb) per second, you would need 40 parallel 10GbE connections, assuming absolutely no overhead. Your precision for "a few seconds" would need to be 40+ seconds using a single 10 GbE link and optimal transmission speed. From th

Nifi Capability for Fast transfer of Data

2016-11-24 Thread shweta Aggarwal
Hi folks, We have a requirement in one of our time critical application wherein we are looking for transferring upto 40-50 GBs worth images within few seconds between remote machine and HDFS. Assuming network connectivity between the two is on a 10Gbe link and NIC and socket buffers tuned optimal