Re: comparing hbase backed by HDFS verses S3

2008-05-01 Thread Leon Mergen
On Thu, May 1, 2008 at 2:30 AM, Chris K Wensel <[EMAIL PROTECTED]> wrote: > Further, once support for appends is added to Hadoop/HDFS, I am unsure if > it will be inherited by S3FS. I think this is a critical issue for HBase. > Does that mean that support for an S3-backed storage backend will be

Re: comparing hbase backed by HDFS verses S3

2008-05-01 Thread Chris K Wensel
No. It just means I have no idea how appends will be implemented and how it affects the other FileSystems. On May 1, 2008, at 2:59 AM, Leon Mergen wrote: On Thu, May 1, 2008 at 2:30 AM, Chris K Wensel <[EMAIL PROTECTED]> wrote: Further, once support for appends is added to Hadoop/HDFS, I

Re: comparing hbase backed by HDFS verses S3

2008-05-01 Thread Clint Morgan
Thanks for the input as it confirmed my suspicions. We were debating running off of S3 just to minimize moving parts. But it does not look feasible. We are wanting the cluster to "live forever" in that once the app is live, hbase will always be needed to serve data. A primary concern is data los

HBase Master only listening to Local Connections on 60000?

2008-05-01 Thread Daniel Leffel
Not sure what's going on with this and thought I'd ask. I can't find any rationale for this on the wiki or in the logs. I've begun to set up our production environment which is a multi-node setup. (To-date our development environment was a single node in psudeo-distributed mode). The HBase master

Re: HBase Master only listening to Local Connections on 60000?

2008-05-01 Thread Daniel Leffel
Actually, I did find something curious in the master log with debug turned on: 2008-05-01 11:05:56,407 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=6, port=6 2008-05-01 11:05:56,482 INFO org.apache.hadoop.hbase.HMaster: HMaster initialized on 127.0.

Re: HBase Master only listening to Local Connections on 60000?

2008-05-01 Thread stack
Daniel: The line out of RpcMetrics reporting port for hostname looks like a little bug we should fix up in hadoop core (I took a quick look; looks like the right args are being passed -- problem must be further up the stack or perhaps your hadoop is older?). Regards the next line reporting l

Re: comparing hbase backed by HDFS verses S3

2008-05-01 Thread stack
Thanks for trying this "interesting" experiment Clint. I'm a little surprised the thing worked at all (smile). What do you need for HBASE-50? Is it sufficient forcing the cluster to go read-only flushing all in memory while the copy runs? CopyFiles/distcp should be able to go between filesy

Re: HBase Master only listening to Local Connections on 60000?

2008-05-01 Thread Daniel Leffel
Switching to all IPs (both in hbase-site.xml and regionservers file) causes the local region server and master to start up fine. The remote ones start throwing the following exception (debug enabled): 2008-05-01 14:45:32,284 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics w

Re: HBase Master only listening to Local Connections on 60000?

2008-05-01 Thread Daniel Leffel
Also, in looking at the master, it clearly thinks it's ip address is 127.0.0.1, although it is accepting remote connections on port 6 now. On Thu, May 1, 2008 at 12:47 PM, Daniel Leffel <[EMAIL PROTECTED]> wrote: > Switching to all IPs (both in hbase-site.xml and regionservers file) causes >

Re: comparing hbase backed by HDFS verses S3

2008-05-01 Thread Clint Morgan
> What do you need for HBASE-50? Is it sufficient forcing the cluster to go > read-only flushing all in memory while the copy runs? Hopefully we can minimize the time we are read-only. We'd like the system to behave as close to normally as possible while snapshotting. Is the only danger of allow

benefited

2008-05-01 Thread Budge Mizutani
Ciao, Are you a frrequent visitor of reetail ssoftware stores? We know what you're overrpaying for: - box manufacturiing - CD - salespersson salary - Rentt of shop spaace - Yeear-to-year increasinng taxes in your coountry Well, what for ?! You're able to downnload everytthing legally NOW