The DataNode caused an java.io.IOException: Premature EOF from inputStream
error.
This error seems to have killed the region server.
One second after this error
I found Error log on the master server.
RegionServerTracker: RegionServer ephemeral node deleted
Thanks
__
Hi,
Please look into the following issue
Scenario:
1) Created an AWS EMR HBASE cluster with 1 Master node and 2 Core(Slave)
nodes
2) Created HBase table with 20 regions auto split across the 2 Core nodes
3) I downscaled my cluster to 1 Core node
4) The table doesn't seem to work
Questions:
Is it
Can you pastebin relevant logs from region server and master around this time ?
Please also check hdfs health.
> On Feb 9, 2017, at 3:44 AM, Kang Minwoo wrote:
>
> The DataNode caused an java.io.IOException: Premature EOF from inputStream
> error.
>
> This error seems to have killed the regi
Can you be specific about how the table didn't work ?
Were some of its regions in transition or offline ?
Which hbase release are you using ?
Please pastebin relevant master log / region server log.
Thanks
> On Feb 9, 2017, at 3:50 AM, Devi Sunil Kumar Shegu
> wrote:
>
> Hi,
>
> Please loo
So here is what I observed.
Dropping this large table had an immediate effect on average locality for
the entire cluster. The locality of regions for OTHER tables on the same
regionserver also fell drastically in the cluster. This was unexpected (I
only thought locality of regions for the dropped t
bq. The locality of regions for OTHER tables on the same regionserver also
fell drastically
Can you be a bit more specific on how you came to the above conclusion ?
Dropping one table shouldn't affect locality of other tables - unless
number of regions on each server becomes unbalanced which trigg
It could be that the table you dropped had a very good locality while
the other tables had less. So, your overall locality went down (when the
"good" locality regions were no longer included). This wouldn't have
affected your system's performance because the locality for the table
didn't change
Thanks Ted/Josh.
Ted-
I store historical metrics on the locality of regions in each regionserver
in the cluster. I noticed that the old table had many regions with low
locality before the drop while the other newer tables had very few cases of
low locality. After the drop, the new table's regions
One additional question:
bq. "This wouldn't have affected your system's performance because the
locality for the table didn't change -- just the system-wide locality."
What does "locality for the table" mean and how is that different from
system-wide locality?
Do you mean other tables (in the sys
bq. After the drop, the new table's regions showed a large drop in locality.
Close to 1700 regions were dropped, please check master log for the
duration table was dropped to see how many regions (of the other tables)
were moved.
Region movement might result in drop in locality.
In StochasticLoad
Here are logs.
[{HOST1} Datanode Log]
2017-02-08 11:08:10,145 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
Exception for {BLOCK1}
2017-02-08 11:08:10,145 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
PacketResponder: {BLOCK1}, type=HAS_DOWNSTREAM_IN_PIPELINE: Thread is
interru
The 'Premature EOF from inputStream' log was at INFO level - it may not be
critical.
Please pastebin more of region server log when you reply.
Was there long pause prior to 2017-02-08 11:08:11,878 ?
Thanks
On Thu, Feb 9, 2017 at 5:59 PM, Kang Minwoo wrote:
> Here are logs.
>
>
> [{HOST1} Datan
Unfortunately, there are no other logs.
Is your word if I ever stopped the region server at that time?
The answer is no.
Thanks.
보낸 사람: Ted Yu
보낸 날짜: 2017년 2월 10일 금요일 오전 11:24:21
받는 사람: user@hbase.apache.org
제목: Re: Region server dies at regular intervals fo
Hey Ted,
Thanks for the reply.
The problem with AWS EMR which doesn't allow downscaling of the slave(core)
nodes.
On Thu, Feb 9, 2017 at 7:17 PM, Ted Yu wrote:
> Can you be specific about how the table didn't work ?
> Were some of its regions in transition or offline ?
>
> Which hbase release
Hey Devi,
I'm able to successfully scale my HBase clusters up and down (core and
task) on EMR. Can you please provide the logs in pastebin so that we can
help?
On Feb 10, 2017 12:10 AM, "Devi Sunil Kumar Shegu"
wrote:
Hey Ted,
Thanks for the reply.
The problem with AWS EMR which doesn't allow
Hey Anthony,
AWS EMR documentation says
1) Core Nodes can be added but cannot be removed
2) Task Nodes can be added and removed
How are you able to downscale the Core Nodes?
Thanks
On Fri, Feb 10, 2017 at 10:44 AM, Anthony Nguyen <
anthony.an.ngu...@gmail.com> wrote:
> Hey Devi,
>
> I'm able t
You're definitely able to scale down core nodes on EMR. I just tried
through the console on emr-5.3.1 to confirm in case that changed :)
The documentation does seem to be inconsistent or old - I did see one
page that I think you found, but other more recent documentation
states that you can resize
17 matches
Mail list logo
