Hi St.Ack: Wondering how many nodes in a cluster do you recommend to hold 5B data? Eventually we need to handle X times 5B data. I want to get an idea of how many resources we need.
Thanks, Xueling On Thu, Dec 17, 2009 at 3:45 PM, stack <[email protected]> wrote: > Hey Xueling, 5B into a single node ain't going to work. Get yourself a bit > of a cluster somewhere. Single node is for messing around. Not for doing > 'real' stuff. > > St.Ack > > > On Thu, Dec 17, 2009 at 3:29 PM, stack <[email protected]> wrote: > > > On Thu, Dec 17, 2009 at 2:38 PM, Xueling Shu <[email protected] > >wrote: > > > >> > >> Things started fine until 5 mins after the data population started. > >> > >> Here is the exception: > >> Exception in thread "main" > >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to > >> contact > >> region server 10.0.176.64:39045 for region Genome,,1261087437258, row > >> > >> > '\x00\x00\x00\x00\x0E\xB00\xAC\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00s\xAD', > >> but failed after 10 attempts. > >> Exceptions: > >> java.io.IOException: java.io.IOException: Server not running, aborting > >> > > > > See why it quit by looking in the regionserver log. > > > > Make sure you have latest hbase and read the 'Getting Started' section. > > > > St.Ack > > > > > > > > > >> at > >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:2347) > >> at > >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.put(HRegionServer.java:1826) > >> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > >> at > >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > >> at java.lang.reflect.Method.invoke(Method.java:597) > >> at > >> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648) > >> at > >> > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > >> > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> java.net.ConnectException: Connection refused > >> > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002) > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers$2.doCall(HConnectionManager.java:1193) > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers$Batch.process(HConnectionManager.java:1115) > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:1201) > >> at > >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:605) > >> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:470) > >> at HadoopTrigger.populateData(HadoopTrigger.java:126) > >> at HadoopTrigger.main(HadoopTrigger.java:52) > >> > >> Can anybody let me know how to fix it? > >> Thanks, > >> Xueling > >> > > > > >
