On Mon, Jul 2, 2012 at 11:16 PM, Taylor, Ronald C
wrote:
...
>
Thanks for the interesting setup description Ronald. Definitely
interested in how things progress.
Go easy,
St.Ack
Hi All,
This issue has been solved by passing Hbase configuration on the mapreduce
codes directly.
*conf* = HBaseConfiguration.*create*();
*conf*.clear();
*conf.set("hbase.zookeeper.quorum", "");*
* *
*conf.set("hbase.zookeeper.property.clientPort","");*
* *
*conf.set("hbase.master", ":6"
While generating some load against a library that makes extensive use of
HTablePool in 0.92, I noticed that the largest heap consumer was
java.lang.ref.Finalizer. Digging in, I discovered that HTablePool's
internal PooledHTable extends HTable, which instantiates a
ThreadPoolExecutor and supporting
Hi Stack,
Re Lustre use: I'm not a hardware infrastructure type of guy, but I can tell
you that we have a very fast interconnect for access into the global filesystem:
"The Olympus Infiniband topology is a combination of 2:1 oversubscribed 36 port
leaf switches and direct links into a 648 port
Hi bryan,
WAL is used only during minor compaction. During major compaction WAL is
not used.
Cheers,
Manoj.P
On Mon, Jul 2, 2012 at 11:00 PM, Bryan Keller wrote:
> During an upgrade of my cluster to 0.90 to 0.92 over the weekend, the WAL
> (files in the /hbase/.logs directory) was corrupted an
Hi anand,
Try increasing your rowBatchSize() using hbaseString() function.This
will increase the no.of rows fetched during a single RPC call. This might
improve the speed of retrieval.
Cheers,
Manoj.P
On Mon, Jul 2, 2012 at 10:53 AM, AnandaVelMurugan Chandra Mohan <
ananthu2...@gmail.com> w
2012/7/2, Amandeep Khurana :
>> Here are the 2 options now. Both with a new table.
>>
>> 1) I store the key "personID" and a:a1 to a:an for the addresses.
>> 2) I store the key "personID" + "address
>>
>> In both I will have the same amount of data. In #1 total size will be
>> smaller since the key
I think your devrackA-03 zookeeper is not quite "ok" - it doesn't seem
to be part of the quorum.
http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-JMX-Monitoring-suggestion-td6681354.html
>>> [hadoop@devrackA-00 ~]$ zookeeper-check
>>> devrackA-03
>>> imok
>>> This ZooKeeper instance is not cu
By "power down" below, I mean powering down the host with the RS that
holds the .META. table. (So - essentially, the host IP is unreachable
and the RS/DN is gone.)
Just wanted to clarify my below steps ...
--S
On Mon, Jul 2, 2012 at 5:36 PM, Suraj Varma wrote:
> Hello:
> We've been doing some fa
When I do "locate hbase-site.xml", "locate hdfs-site.xml", and "locate
core-site.xml" there are 2 locations for each on the HRegionServers.
All files are either in $HADOOP_HOME/conf or $HBASE_HOME/conf and there
are files of the same name in "example" directories.
I moved my HRegionServers back to
I was about to propose that next ... you could just try back porting
that uber hbck (or the relevant portions) and run the patched ruby
script ... I've done this for like online meta table merges ... but
not with hbck.
So - yes, that might be your solve. Good luck!
--S
On Mon, Jul 2, 2012 at 4:44
Hello:
We've been doing some failure scenario tests by powering down a .META.
holding region server host and while the HBase cluster itself recovers
and reassigns the META region and other regions (after we tweaked down
the default timeouts), our client apps using HBaseClient take a long
time to re
Thanks a bunch for the insight. This message was actually coming from
master, but it still needs to grab the HLog files from hdfs, so I can still
see it being what you mentioned. I'm going to look into tuning these
parameters down in preparation for future failures.
On Mon, Jul 2, 2012 at 7:56 P
This looks like it is trying to reach a datanode ... doesn't it?
> 12/06/30 00:07:22 INFO ipc.Client: Retrying connect to server:
> /10.125.18.129:50020. Already tried 14 time(s).
Is this from a master log or from a region server log? (I'm guess the
above is from a region server log while trying
Inline
On Monday, July 2, 2012 at 4:48 PM, Jean-Marc Spaggiari wrote:
> Addresses will mainly be accessed independently, and sometime only,
> with the other data.
>
> I'm not sure either to prefer the "versions" option. So if I go with a
> 2nd table, does it mean it's better to have more rows
Bryan,
There is a script attached to
https://issues.apache.org/jira/browse/HBASE-4232 , I have used it a couple
of times to add region on filesystem but not in .META.
A sample run goes like this (note you need to provide path to regioninfo):
${HBASE_HOME}/bin/hbase org.jruby.Main addregion.rb
/hb
Addresses will mainly be accessed independently, and sometime only,
with the other data.
I'm not sure either to prefer the "versions" option. So if I go with a
2nd table, does it mean it's better to have more rows than more
columns?
Here are the 2 options now. Both with a new table.
1) I store t
I'm on cdh3u2 to be precise. I don't have the fixMeta option, or any other
special fix options aside from the basic -fix.
I'll take a look at what 0.90.6 version does when it fixes meta. Maybe I
can just follow what it does there.
Thanks again!
On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma wrot
Ok - thanks for checking connectivity.
I presume you already have doublechecked the hbase-site.xml in your
region server that points to the zookeeper and hdfs-site.xml pointed
to the namenode.
I once got a similar error when HBase was picking up a stray
core-site.xml / hdfs-site.xml from the hdfs
Are you using apache hbase 0.90.4 ... or the one from CDH3?
Check what other hbck options you have (do you have -fixMeta?) on the
version you are on.
What the uber-hbck (part of hbase-0.90.6 and later releases) does is
"automate" this meta table surgery to fix the issue.
I will let folks who have
Actually I just noticed part that I must have skimmed over. It mentions
that I have 3 regions for this table not in .META. but deployed on servers.
Interestingly there are 4 other regions for other tables that are on HDFS
but not in META or deployed on any server.
Can I manually add these someh
Thanks for the response.
hbck consistently shows 7 inconsistencies. I tried -fix (a few times) and
it didn't seem to fix it. I tried -details and it does seem to print out
more, but nothing super usable from what I can tell.
On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma wrote:
> Try doing an hb
If I understand you right, you are asking about how region splitting works ...
See http://hbase.apache.org/book/regions.arch.html section 9.7.4
In a nutshell, the parent region on your RS1 will split into two
daughter regions on the same RS1. If you have load balancer turned on,
the master can the
First, thank you.
I moved my HRegionservers not my HQuorumPeers.
I have checked the network and everyone can talk to everyone. I can
even talk to my HQuorumPeers via "nc" from the nodes that should be
running my HMaster on it and my HRegionservers.
[hadoop@devrackA-00 ~]$ zookeeper-check
devrac
Session expired usually results from a long GC that exceeds the
zookeeper.session.timeout.
> 2012-07-01 18:20:00,961 FATAL
> org.apache.hadoop.hbase.master.HMaster:master:6-0x238444cf77e
> master:6-0x238444cf77e received
expired from ZooKeeper, aborting
org.apache.zookeeper.Keepe
It would be good to update the thread on how you fixed it ... for
users who tread the same path tomorrow. :)
Was it dos2unix on your conf / bin directories that fixed it?
--S
On Mon, Jul 2, 2012 at 6:49 AM, syed kather wrote:
> Thanks Marcin Cylke now its is working
>
> Thanks and Re
Try doing an hbase hbck to see if it reports inconsistency. And do an
hbase hbck -fix to see if it fixes it for you. See
http://hbase.apache.org/book.html#hbck.in.depth
Note that since 0.90.4 is old, some of the documented options won't be
available ... but hbase hbck -fix will be available.
--S
The error you are getting is:
> 2012-07-02 12:39:02,205 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server devrackA-05/172.18.0.6:2181
> 2012-07-02 12:39:02,211 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x0 for server null, unexpected error, closing socket connection
On Mon, Jul 2, 2012 at 7:37 PM, Bryan Keller wrote:
> During an upgrade of my cluster to 0.90 to 0.92 over the weekend, the WAL
> (files in the /hbase/.logs directory) was corrupted and it prevented HBase
> from starting up. The exact exception was "java.io.IOException: Could not
> obtain the l
On Mon, Jul 2, 2012 at 8:19 PM, Taylor, Ronald C wrote:
> Pacific Northwest National Laboratory (www.pnl.gov) - Hadoop and HBase
> (Cloudera distribution) are being used within PNNL's Computational Biology &
> Bioinformatics Group for a systems biology data warehouse project that
> integrates h
First, Yep I am a newbie to Hadoop/Hbase. I have read both of the
O'Reilly books (Hadoop and Hbase), so my knowledge level at this point
is pure book learning and understanding the log messages is very vexing.
Second, based on the recommendations of this mail-list I decided to move
my HRegionserve
Responses inline
On Monday, July 2, 2012 at 12:53 PM, Jean-Marc Spaggiari wrote:
> Hi Amandeep,
>
> Thanks for your prompt reply.
>
> I forgot to add that all the addresses are valid at the same time.
> There is no orders int the addresses. They are all active addresses at
> the same time. If
Hi Amandeep,
Thanks for your prompt reply.
I forgot to add that all the addresses are valid at the same time.
There is no orders int the addresses. They are all active addresses at
the same time. If one is not valid any more, it's removed. If there is
a new one, it's added to the list, not replac
Apache will be hosting a series of events in the two days prior to
OSCON, the 16th and 17th of July. These events are a unique
opportunity to engage new participants and build awareness for
HBase.
The goal is to provide an environment where newcomers and veteran
Apache contributors alike can lear
Wasn't there an EC2 outage or am I imagining things?
On Mon, Jul 2, 2012 at 2:30 PM, Norbert Burger wrote:
> From what I understand, the leap second bug could've hit anytime in the 24
> hours before 23:59:59. We had it start happening early afternoon Sat on a
> few of our boxes.
>
> Norbert
>
>
Jean-Marc,
These are great questions! Find my answers (and some questions for you) inline.
-ak
On Monday, July 2, 2012 at 12:04 PM, Jean-Marc Spaggiari wrote:
> Hi,
>
> I have a question regarding the best way to design a table.
>
> Let's imagine I want to store all the people in the world
Hello Stack, Lars,
Here is a scientific application, if you want to add it:
Pacific Northwest National Laboratory (www.pnl.gov) - Hadoop and HBase
(Cloudera distribution) are being used within PNNL's Computational Biology &
Bioinformatics Group for a systems biology data warehouse project that
Hi,
I have a question regarding the best way to design a table.
Let's imagine I want to store all the people in the world on a database.
Everyone has a name, last name, phone number, lot of flags (sex, age, etc.).
Now, people can have one address, but they can also have 2, or 3, or
even more...
I agree that increasing the timeout is not the best option, I will work
both on better balancing the load and maybe doing it in increments like you
suggested. However for now I want a quick fix to the problem.
Just to see if I understand this right: a zookeeper node redirects my
client to a region
>From what I understand, the leap second bug could've hit anytime in the 24
hours before 23:59:59. We had it start happening early afternoon Sat on a
few of our boxes.
Norbert
On Mon, Jul 2, 2012 at 12:58 PM, Kevin O'dell wrote:
> How recently would you say this is happening? Did this start la
As someone who has been developing/running/using the software for a longer
period of time than the person who is asking the question, you can best serve
the poser by making them aware of the trade offs and why it's a good/bad idea
to do things a certain way. At the end of the day, it's their cho
Hi Sever,
It seems one of the nodes in your cluster is overwhelmed with the load
you are giving him.
So IMO, you have two options here:
First, you can try to reduce the load. I mean, split the bulk in
multiple smaller bulks and load them one by one to give the time to
your cluster to dispatch it
On Mon, Jul 2, 2012 at 4:40 AM, Michael Segel wrote:
> I am not finding fault with what Andy was saying. The problem is that we tend
> not to use stronger language when discussing these topics. And my point
> wasn't just on this topic but others posts where we say 'not a good idea' yet
> someo
IMO, the application that you are referring should be set up to impersonate
other users (called proxy-user authentication).
Have a look at
http://hadoop.apache.org/common/docs/r1.0.3/Secure_Impersonation.html. This can
be mapped to the HBase land..
I think the class org.apache.hadoop.hbase.se
During an upgrade of my cluster to 0.90 to 0.92 over the weekend, the WAL
(files in the /hbase/.logs directory) was corrupted and it prevented HBase from
starting up. The exact exception was "java.io.IOException: Could not obtain the
last block locations" on the WAL files.
I was able to recover
We were also caught by this - we're running CentOS 6. Likewise, once we reset
the date/time, HBase was happy again. I wonder why Java processes seemed to be
affected more than other processes?
On Jul 2, 2012, at 8:34 AM, Dean Banks wrote:
> We were caught by this issue. It impacted all of our
my understanding is that the WAL log is used for replication as well.
If all your data has been persisted to disk (i.e. all data in memstores have
been flushed to disks) and replication is disabled, I believe you can delete
the WAL without data loss.
just my 2 cents
On 2012-07-02, at 1:37 PM,
During an upgrade of my cluster to 0.90 to 0.92 over the weekend, the WAL
(files in the /hbase/.logs directory) was corrupted and it prevented HBase from
starting up. The exact exception was "java.io.IOException: Could not obtain the
last block locations" on the WAL files.
I was able to recover
Can someone please help me with this?
Thanks,
Sever
On Tue, Jun 26, 2012 at 8:14 PM, Sever Fundatureanu <
fundatureanu.se...@gmail.com> wrote:
> My keys are built of 4 8-byte Ids. I am currently doing the load with MR
> but I get a timeout when doing the loadIncrementalFiles call:
>
> 12/06/24
How recently would you say this is happening? Did this start last Sat
around midnight?
On Mon, Jul 2, 2012 at 11:50 AM, Nicolas Thiébaud
wrote:
> Hi,
>
> We have been successfully running a cdh3 HBase cluster on c1.xlarge
> instances for over a month, but we recently started hitting what looks l
Hi,
We have been successfully running a cdh3 HBase cluster on c1.xlarge
instances for over a month, but we recently started hitting what looks like
connectivity issues in the clusters. Zookeeper sessions are expired by the
zk server and the region servers throw a YouAreDeadException before
crashin
We were caught by this issue. It impacted all of our JVM processes,
including the full hbase/zookeeper/HDFS stack.
Before we had determined root cause, we somewhat hastily bounced some of
our services (ZK, HM), and had to forcefully kill a few RSes in an
attempt to get things back online.
A
Well...
I wasn't sure if St.Ack was displeased by my comments on Andrew's response, or
my references to KISS where the second S is stupid, reference to 'dead hookers'
or reference to drugs.
I was just covering my bases. :-)
With respect to Andrew's response, I saw something that I wasn't sure
Hey all,
I'm running hbase 0.90.4, and somehow the regions for one of my tables have
become unlinked from the table itself. The Master UI shows no regions for
the table, and attempting to call Table.getRegionsInfo() returns nothing.
However, I can see the regions in hdfs in their proper location
Adding capcha didn't help?
Sent from my iPad
On 2 ביול 2012, at 15:13, Stack wrote:
> On Mon, Jul 2, 2012 at 1:06 PM, Ben Cuthbert wrote:
>> Thanks Lars
>>
>> Just a note how do I edit and add? I have just registered.
>>
>>
>
> Oh yeah... you have to be granted perms to edit wiki because it wa
Thanks Marcin Cylke now its is working
Thanks and Regards,
S SYED ABDUL KATHER
On Mon, Jul 2, 2012 at 6:05 PM, Marcin Cylke wrote:
> On 02/07/12 10:34, syed kather wrote:
>
> > java.lang.RuntimeException: org.xml.sax.SAXParseException: Content is not
> > allowed in prolog.
On 02/07/12 10:34, syed kather wrote:
> java.lang.RuntimeException: org.xml.sax.SAXParseException: Content is not
> allowed in prolog.
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1237)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.j
On Mon, Jul 2, 2012 at 10:34 AM, syed kather wrote:
> Team ,
> while i am trying to import the data from exported backup . I am getting
> this "Content is not allowed in prolog". Please help me
>
What JVM are you using?
St.Ack
On Mon, Jul 2, 2012 at 1:06 PM, Ben Cuthbert wrote:
> Thanks Lars
>
> Just a note how do I edit and add? I have just registered.
>
>
Oh yeah... you have to be granted perms to edit wiki because it was
being spammed at a fierce rate... I should have remembered (might
explain why our powered-by pag
What kind of explanation is this???
Regards,
Mohammad Tariq
On Mon, Jul 2, 2012 at 5:10 PM, Michael Segel wrote:
> Sorry St. Ack,
>
> Which is why I said that I was losing it...
>
> The entire quote was...
> "On Sun, Jul 1, 2012 at 2:05 PM, Jay Wilson
> wrote:
>> Can a regionserver
Sorry St. Ack,
Which is why I said that I was losing it...
The entire quote was...
"On Sun, Jul 1, 2012 at 2:05 PM, Jay Wilson
wrote:
> Can a regionserver and quorumpeer reside on the same node?
It can, but you want to consider how disk is allocated in the cluster.
A typical and recommended c
Thanks Lars
Just a note how do I edit and add? I have just registered.
On 2 Jul 2012, at 11:50, Ulrich Staudinger wrote:
> Cheers everyone,
> I tried already adding mine, but the page says "Immutable Page"
>
> Why is it immutable?
>
> Thanks
>
>
> On Mon, Jul 2, 2012 at 12:32 PM, Lars Georg
Cheers everyone,
I tried already adding mine, but the page says "Immutable Page"
Why is it immutable?
Thanks
On Mon, Jul 2, 2012 at 12:32 PM, Lars George wrote:
> Hi Ben,
>
> Please do so, you can create yourself an account and edit the Wiki page.
> Let us know if you get stuck.
>
> Thanks fo
Hi Ben,
Please do so, you can create yourself an account and edit the Wiki page. Let us
know if you get stuck.
Thanks for sharing!
Lars
On Jul 2, 2012, at 12:20 PM, Ben Cuthbert wrote:
> Hi Lars
>
> We would love to add our company
>
> http://www.celer-tech.com
>
> Regards
>
> Ben
> On 2
Hi Lars
We would love to add our company
http://www.celer-tech.com
Regards
Ben
On 2 Jul 2012, at 11:11, Lars George wrote:
> Hi,
>
> Please see http://wiki.apache.org/hadoop/Hbase/PoweredBy
>
> Everyone on this list, kindly consider verifying that your entry on the
> Powered By page is curr
Hi,
Please see http://wiki.apache.org/hadoop/Hbase/PoweredBy
Everyone on this list, kindly consider verifying that your entry on the Powered
By page is current.
For those who are users of HBase but have not added yourself to the above page:
if you are happy to share this with us and the rest
Hi lztaomin,
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired
indicates that you have experienced the "Juliet Pause" issue, which means you
ran into a JVM garbage collection that lasted longer than the configured
ZooKeeper timeout threshold.
Team ,
while i am trying to import the data from exported backup . I am getting
this "Content is not allowed in prolog". Please help me
* Error:*
java.lang.RuntimeException: org.xml.sax.SAXParseException: Content is not
allowed in prolog.
at
org.apache.hadoop.conf.Configuration.loadResou
On Sat, Jun 30, 2012 at 5:55 PM, Asaf Mesika wrote:
> I've tried editing but I don't have permissions. What should be done to
> obtain them?
>
Check out the hbase source code: http://hbase.apache.org/source-repository.html
The docs are in src/docbkx.
Don't worry about getting the formatting rig
On Mon, Jul 2, 2012 at 7:11 AM, Michael Segel wrote:
> I'm sorry I'm losing it.
>
Its plain. Do us a favor and try keeping your psychotic breakdown to
yourself going forward.
St.Ack
Hi Mike,
> Running RS on a machine where DN isn't running?
I am not following here. Andy said that both are on the same node. Where in
this thread did someone imply something else? Just curious.
Cheers,
Lars
On Jul 2, 2012, at 7:11 AM, Michael Segel wrote:
> I'm sorry I'm losing it.
>
>
71 matches
Mail list logo
