In my current project we are planning to streams of data to Namenode (20 Node Cluster). Data Volume would be around 1 PB per day. But there are application which can publish data at 1GBPS.
Few queries: 1. Can a single Namenode handle such high speed writes? Or it becomes unresponsive when GC cycle kicks in. 2. Can we have multiple federated Name nodes sharing the same slaves and then we can distribute the writes accordingly. 3. Can multiple region servers of HBase help us ?? Please suggest how we can design the streaming part to handle such scale of data. Regards, Jagaran Das
