Sven, did you ever put up a Dockerfile? I've done something like this
waiting for HDFS to come out of safe mode.
echo "Starting Hadoop"
${HADOOP_HOME}/sbin/start-dfs.sh
echo "Waiting for hadoop to come out of safeMode"
local x=0
while [ $x -lt 1 ] do # try 5 times, for slow computers
${HADOOP_HOME}/bin/hdfs dfsadmin -safemode && x=5
x=$(( $x + 1 ))
done
On Tue, Mar 22, 2016 at 2:41 PM, Josh Elser <josh.el...@gmail.com
<mailto:josh.el...@gmail.com>> wrote:
Every time I use Docker, I seem to run into odd situations like
this. Very frustrating.
I would love to know what's happening and get this working
reliably. Having an "Apache-owned" docker image for Accumulo would
be nice (and would help us internally as well as externally, IMO).
Andrew Hulbert wrote:
+1 for Josh's suggestion.
Not sure if its the same problem, but I too had to add some
sleeps for
single node accumulo startup scripts directly before and after
init'ing
accumulo. I dug in once and it seemed that the datanode needed
more time
to let other processes know the files in HDFS existed at
/accumulo.
Perhaps it was awaiting the exit from safe mode? Three seconds
did it
for me:
https://github.com/jahhulbert-ccri/cloud-local/blob/master/bin/cloud-local.sh#L109
On 03/14/2016 01:49 PM, Josh Elser wrote:
Well, I'm not sure wtf is going on, but I added some
sleeps to your
init.sh script (30seconds before starting ZooKeeper and before
init'ing Accumulo) and now things worked just fine.
I'm not seeing any Accumulo problems here, but rather the
same Docker
absurdities I run into every time I try to use it :). I
also just
upgrade to Docker 1.10.3 if that helps.
Josh Elser wrote:
So, I got it running, and ran into the same problem
you described.
The odd thing is that both the Accumulo Master and
TabletServer seemed
to be... stuck. They both claimed to be running, but
they were both hung
(unable to communicate). This ultimately caused the
TabletServer to lose
its ZooKeeper lock (and why the Shell didn't think it
was running).
I'm running into all sorts of docker issues ATM
(trying to upgrade now,
hoping it will help), but I'd recommend trying to
increase the amount of
heap for both the master and tserver from 128M to 256M in
/accumulo-1.7.1/conf/accumulo-env.sh (the -Xmx value).
Josh Elser wrote:
Ok! Thanks, I'll look for that init.sh script.
We're not too... docker-savvy here yet :), but we
can definitely help
out if we can find an automation script. Let me
get back to you.
Sven Hodapp wrote:
Hi Josh,
thanks for your answer.
Currently I haven't uploaded the Dockerfile...
If you want I'll upload
it!
(But currently it only is an debian with ssh,
rsync, jdk7, the
unzipped distributions of hadoop, zookeeper
and accumulo, and the set
of environment variables)
Currently, and for testing, I start all things
manually.
The steps I do is documented in /init.sh
within the docker container.
I've enshured that dfs and zookeeper are up
before starting accumulo.
Regards,
Sven
----- Ursprüngliche Mail -----
Von: "Josh
Elser"<<mailto:josh.el...@gmail.com>josh.el...@gmail.com>
An: "user"<user@accumulo.apache.org
<mailto:user@accumulo.apache.org>>
Gesendet: Montag, 14. März 2016 15:41:16
Betreff: Re: Accumulo 1.7.1 on Docker
Hi Sven,
I can't seem to find the source for your
Docker image (nor am smart
enough to figure out how to extract it).
Any chance you can point
us to
that?
That said, I did run your image, and it
looks like you didn't
actually
start Hadoop, ZooKeeper, or Accumulo.
Our documentation treats Hadoop and
ZooKeeper primarily as
prerequisites, so check their docs for
good instructions.
For Accumulo -
Configuration:
http://accumulo.apache.org/1.7/accumulo_user_manual.html#_installation
Initialization:
http://accumulo.apache.org/1.7/accumulo_user_manual.html#_initialization
Starting:
http://accumulo.apache.org/1.7/accumulo_user_manual.html#_starting_accumulo
Sven Hodapp wrote:
Dear reader,
I'd like to create a docker image with
accumulo 1.7.1.
I've install it from scratch with
hadoop-2.7.2, zookeeper-3.4.8 and
accumulo-1.7.1.
I'm going though the installation and
every time I'll end up like
that:
root@deebd8e29683:/# accumulo shell -u
root
Password: ******
2016-03-13 13:18:31,627
[trace.DistributedTrace] INFO :
SpanReceiver
org.apache.accumulo.tracer.ZooTraceClient
was loaded successfully.
2016-03-13 13:20:31,955
[impl.ServerClient] WARN : There are no
tablet servers:
check that zookeeper and accumulo are
running.
Anybody got an idea whats wrong? I
have no idea anymore...
I'll currently share this docker image
on docker hub.
If you want to try it you can simply
start:
docker run -it themerius/accumulo
/bin/bash
All binaries and configs are in there.
In ./init.sh are my setup
steps.
Regards,
Sven
