I'll go ahead and make a ticket there with my findings. Thanks for your time.
Regards, Koen On Wed, Jul 24, 2019 at 10:08 AM Norbert Kalmar <nkal...@cloudera.com.invalid> wrote: > Hello Koen, > > Thanks for looking into this. > Unfortunately, we are not maintaining the docker image, and I'm also pretty > much out of ideas at this point, except that it looks like permission > setting issue with the docker script to me. > I suggest you file a ticker here: > https://github.com/31z4/zookeeper-docker/issues > Or in the general docker forums. There are multiple issues posted already, > like this one for example: > > https://forums.docker.com/t/cannot-get-zookeeper-to-work-running-in-docker-using-swarm-mode/27109 > > Sorry I couldn't be any further help, I don't work much with docker :( > > Regards, > Norbert > > On Tue, Jul 23, 2019 at 3:28 PM Koen De Groote < > koen.degro...@limecraft.com> > wrote: > > > Hello again, Norbert, > > > > I haven't been able to get it working yet, but did notice something else > > concerning the zookeeper user getting that error for the directory not > > being found. > > > > After looking into it a bit more, these are my findings: > > > > The zookeeper dockerfile can be found here: > > https://github.com/31z4/zookeeper-docker/blob/master/3.4.14/Dockerfile > > > > And the relevant part shows up at the very top: > > > > ENV ZOO_CONF_DIR=/conf \ > > ZOO_DATA_DIR=/data \ > > ZOO_DATA_LOG_DIR=/datalog \ > > ZOO_LOG_DIR=/logs \ > > ZOO_TICK_TIME=2000 \ > > ZOO_INIT_LIMIT=5 \ > > ZOO_SYNC_LIMIT=2 \ > > ZOO_AUTOPURGE_PURGEINTERVAL=0 \ > > ZOO_AUTOPURGE_SNAPRETAINCOUNT=3 \ > > ZOO_MAX_CLIENT_CNXNS=60 > > > > This sets these settings as environment variables inside the container. > > > > First thing of note: These environment variables are only available to > the > > root user. The process does run as the zookeeper user, to which said > > environment variables are not available. > > > > As can be seen from the output of this command, it is indeed the > zookeeper > > user running the process: > > > > bash-4.4# ps -a | grep zookeeper > > 1 zookeepe 0:04 /usr/lib/jvm/java-1.8-openjdk/jre/bin/java > > -Dzookeeper.log.dir=/logs -Dzookeeper.root.logger=INFO,CONSOLE -cp > > > > > /zookeeper-3.4.13/bin/../build/classes:/zookeeper-3.4.13/bin/../build/lib/*.jar:/zookeeper-3.4.13/bin/../lib/slf4j-log4j12-1.7.25.jar:/zookeeper-3.4.13/bin/../lib/slf4j-api-1.7.25.jar:/zookeeper-3.4.13/bin/../lib/netty-3.10.6.Final.jar:/zookeeper-3.4.13/bin/../lib/log4j-1.2.17.jar:/zookeeper-3.4.13/bin/../lib/jline-0.9.94.jar:/zookeeper-3.4.13/bin/../lib/audience-annotations-0.5.0.jar:/zookeeper-3.4.13/bin/../zookeeper-3.4.13.jar:/zookeeper-3.4.13/bin/../src/java/lib/*.jar:/conf: > > -Dcom.sun.management.jmxremote > > -Dcom.sun.management.jmxremote.local.only=false > > org.apache.zookeeper.server.quorum.QuorumPeerMain /conf/zoo.cfg > > > > Second thing of note: The 3rd part of the dockerfile installs gosu, but > the > > user isn't actually changed to the zookeeper user at this point. This > > happens later in docker-entrypoint.sh. Only the install of gosu is > verified > > to work at this point. > > > > Third thing of note: At the end of the dockerfile, this happens: > > > > ENV PATH=$PATH:/$DISTRO_NAME/bin \ > > ZOOCFGDIR=$ZOO_CONF_DIR > > > > But again, this environment variable is only available to the root user > and > > not the zookeeper user. > > > > Then, zkServer.sh is executed to start the process. The thing of note > here > > is the docker-entrypoint file: > > > > > https://github.com/31z4/zookeeper-docker/blob/master/3.4.14/docker-entrypoint.sh > > > > Which does in fact change the user to zookeeper, but doesn't take along > > with it any environment variables. > > > > The part where it all goes wrong is when zkCleanup.sh calls zkEnv.sh to > get > > environment variables. Since the zookeeper user is the one running that > > process, it won't actually see what it needs to see. > > > > This part: > > > > > > if [ "x$ZOOCFGDIR" = "x" ] > > then > > if [ -e "${ZOOKEEPER_PREFIX}/conf" ]; then > > ZOOCFGDIR="$ZOOBINDIR/../conf" > > else > > ZOOCFGDIR="$ZOOBINDIR/../etc/zookeeper" > > fi > > fi > > > > > > Does not follow the logic of directory layout we see at the start of the > > dockerfile(the environment variables) at all. > > > > The warning about the folder not being found is fixed if I perform this > > first: > > > > export ZOOCFGDIR="/conf" > > > > But the cleanup still doesn't work. The script just finishes with no > output > > and the files are still there. The user is correct, zookeeper is owner of > > the files and owner has write permissions. > > There's no extended file attributes on the files either. > > > > So I'm at my wit's end here. > > > > For information: the command that the script generates and runs is this: > > > > java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp > > > > > '/zookeeper-3.4.13/bin/../build/classes:/zookeeper-3.4.13/bin/../build/lib/*.jar:/zookeeper-3.4.13/bin/../lib/slf4j-log4j12-1.7.25.jar:/zookeeper-3.4.13/bin/../lib/slf4j-api-1.7.25.jar:/zookeeper-3.4.13/bin/../lib/netty-3.10.6.Final.jar:/zookeeper-3.4.13/bin/../lib/log4j-1.2.17.jar:/zookeeper-3.4.13/bin/../lib/jline-0.9.94.jar:/zookeeper-3.4.13/bin/../lib/audience-annotations-0.5.0.jar:/zookeeper-3.4.13/bin/../zookeeper-3.4.13.jar:/zookeeper-3.4.13/bin/../src/java/lib/*.jar:/conf:' > > org.apache.zookeeper.server.PurgeTxnLog /data /data -n 3 > > > > Changing logger to TRACE offers no output either. > > > > > > > > > > On Mon, Jul 22, 2019 at 10:13 AM Koen De Groote < > > koen.degro...@limecraft.com> > > wrote: > > > > > Performing "bash -ex ./zkCleanup.sh /data/version-2 -n 3" as root > results > > > in the creation of another version-2 folder(empty) in the existing > > > version-2 folder. > > > > > > As both root and zookeeper user I am able to create files in the > > > /data/version-2 directory inside the container. > > > > > > The zookeeper user is indeed not the owner of anything in the zk/bin > > > folder(/zookeeper-3.4.13/bin). Executing zkCli.sh works, but creating a > > > file in there doesn't. > > > > > > Permission level for the folder seems to be 0755 on all files and the > > > folder itself. > > > > > > Just ran into what I think is the problem: the relative path to the > > > zoo.cfg file isn't correct. > > > > > > I tried running just plain "./zkCleanup.sh" as the zookeeper user from > > > within the folder and it printed that it could not find the zoo.cfg > file, > > > but the path it printed was basically > "current_dir/../expected_cfg_dir", > > > which is one ".." too little. > > > > > > Will check if this is due to a setting of mine. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 19, 2019 at 2:23 PM Norbert Kalmar > > > <nkal...@cloudera.com.invalid> wrote: > > > > > >> I would first check the permission on zkCleanup.sh and the bin folder. > > >> Sounds like zookeeper user has no access to the /zk/bin directory. > > >> That might also explain why it is not getting deleted by the zk > > instance. > > >> > > >> And I'm not sure in this one, but did you try giving the full path to > > the > > >> txn log files like bash -ex ./zkCleanup.sh /data/version-2 -n 3 as > root? > > >> I think this script might be expecting the full path, including the > > >> version-2 directory. > > >> > > >> Regards, > > >> Norbert > > >> > > >> On Fri, Jul 19, 2019 at 2:00 PM Koen De Groote < > > >> koen.degro...@limecraft.com> > > >> wrote: > > >> > > >> > Hello Norbert, > > >> > > > >> > I've set up a new environment which then reached at least 4 *.log > > files > > >> > All snapshots and log files are kept in /data/version-2/(default for > > the > > >> > image) > > >> > > > >> > I went into the zookeeper container and executed: > > >> > > > >> > bash -ex ./zkCleanup.sh /data -n 3 > > >> > > > >> > As root, this changes nothing. There are still 4 *.log files > > >> > > > >> > Changing to the zookeeper user, I get the following output: > > >> > > > >> > Path '/zookeeper-3.4.13/bin' does not exist. > > >> > Usage: > > >> > PurgeTxnLog dataLogDir [snapDir] -n count > > >> > dataLogDir -- path to the txn log directory > > >> > snapDir -- path to the snapshot directory > > >> > count -- the number of old snaps/logs you want to keep, value should > > be > > >> > greater than or equal to 3 > > >> > > > >> > And the 4 *.log files still exist. > > >> > Also printing the usage, indicating, to me at least, that something > > >> about > > >> > the input is wrong, even though it is identical to the one used as > > root, > > >> > which did not result in this output. > > >> > > > >> > No actual error messages seem to be printed or logged anywhere. > > >> > > > >> > Not sure what to do next. > > >> > > > >> > > > >> > > > >> > On Fri, Jul 19, 2019 at 11:01 AM Norbert Kalmar > > >> > <nkal...@cloudera.com.invalid> wrote: > > >> > > > >> > > Hi Koen, > > >> > > > > >> > > It should do just as you said. You can also set > > >> > autopurge.snapRetainCount, > > >> > > bu default it is set to 3, so if you didn't set anything it is > not a > > >> > reason > > >> > > to keep old logs. > > >> > > > > >> > > As a plan B you could use zkCleanup.sh [snapshotDir] -n 3 to > delete > > >> all > > >> > > except the last 3 log files. You can add this to a cron job. > > >> > > > > >> > > As for why the old log files not getting deleted, could be > something > > >> > > related to the docker image, maybe a permission problem? Do you > see > > >> any > > >> > > errors in the server log? > > >> > > > > >> > > Regards, > > >> > > Norbert > > >> > > > > >> > > On Thu, Jul 18, 2019 at 9:25 PM Koen De Groote < > > >> > > koen.degro...@limecraft.com> > > >> > > wrote: > > >> > > > > >> > > > Greetings, > > >> > > > > > >> > > > Working with Zookeeper version 3.4.13 in the official docker > > image. > > >> > > > > > >> > > > I was under the impression that the setting > > >> "autopurge.purgeInterval=1" > > >> > > > meant that log files would be cleaned up every hour. > > >> > > > > > >> > > > Instead, I now find that months of these files are just sitting > in > > >> > their > > >> > > > directory, untouched. > > >> > > > > > >> > > > So perhaps I'm wrong about that, but I'm not sure. > > >> > > > > > >> > > > What I wish to achieve is that these log files stop accumulating > > and > > >> > keep > > >> > > > only the most recent. Is there a way to achieve this? Or are > they > > >> > merely > > >> > > > historical and can they be deleted freely? > > >> > > > > > >> > > > Kind regards, > > >> > > > Koen De Groote > > >> > > > > > >> > > > > >> > > > >> > > > > > >
