Hi Sanjay, Instead of using --env SKIP_SCHEMA_INIT=TRUE,
Please use --env IS_RESUME="true" while running, docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore --add-host=host.docker.internal:host-gateway \ --env IS_RESUME="true" \ --env DB_DRIVER=mysql \ --env SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true -Djavax.jdo.option.ConnectionUserName=hive -Djavax.jdo.option.ConnectionPassword=****" \ --mount source=warehouse,target=/opt/hive/data/warehouse \ --name metastore-standalone apache/hive:${HIVE_VERSION} /bin/bash This should skip the schema initOrUpgrade process: https://github.com/apache/hive/blob/5022b85b5f50615f85da07bce42aebd414deb9b0/packaging/src/docker/entrypoint.sh#L24 from the 2nd time you run the container. Regards, Akshat On Fri, Dec 22, 2023 at 11:53 AM Sanjay Gupta <sanja...@gmail.com> wrote: > Hi All, > > If my metastore schema already exists with correct version so what I > need to do so it doesn't do init or upgrade when starting metastore > container > > I have tried following command line > > > On MAC environment variables > export HIVE_VERSION=3.1.3 > and even > SKIP_SCHEMA_INIT=TRUE > > docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore > --add-host=host.docker.internal:host-gateway \ > --env SKIP_SCHEMA_INIT=TRUE \ > --env DB_DRIVER=mysql \ > --env > SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver > > > -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true > -Djavax.jdo.option.ConnectionUserName=hive > -Djavax.jdo.option.ConnectionPassword=****" \ > --mount source=warehouse,target=/opt/hive/data/warehouse \ > --name metastore-standalone apache/hive:${HIVE_VERSION} /bin/bash > > > > ----------------------- > > Docker Logs , it still tries to initSchema > > docker logs 1c > + : mysql > + SKIP_SCHEMA_INIT=false > + export HIVE_CONF_DIR=/opt/hive/conf > + HIVE_CONF_DIR=/opt/hive/conf > + '[' -d '' ']' > + export 'HADOOP_CLIENT_OPTS= -Xmx1G > -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver > > -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true > -Djavax.jdo.option.ConnectionUserName=hive > -Djavax.jdo.option.ConnectionPassword=hive' > + HADOOP_CLIENT_OPTS=' -Xmx1G > -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver > > -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true > -Djavax.jdo.option.ConnectionUserName=hive > -Djavax.jdo.option.ConnectionPassword=hive' > + [[ false == \f\a\l\s\e ]] > + initialize_hive > + COMMAND=-initOrUpgradeSchema > ++ cut -d . -f1 > ++ echo 3.1.3 > + '[' 3 -lt 4 ']' > + COMMAND=-initSchema > + /opt/hive/bin/schematool -dbType mysql -initSchema > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > > [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > > [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory] > Metastore connection URL: > jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true > Metastore Connection Driver : com.mysql.cj.jdbc.Driver > Metastore connection User: hive > Starting metastore schema initialization to 3.1.0 > Initialization script hive-schema-3.1.0.mysql.sql > > > Error: Table 'ctlgs' already exists (state=42S01,code=1050) > org.apache.hadoop.hive.metastore.HiveMetaException: Schema > initialization FAILED! Metastore state would be inconsistent !! > Underlying cause: java.io.IOException : Schema script failed, errorcode 2 > Use --verbose for detailed stacktrace. > *** schemaTool failed *** > [WARN] Failed to create directory: /home/hive/.beeline > No such file or directory > + '[' 1 -eq 0 ']' > + echo 'Schema initialization failed!' > Schema initialization failed! > + exit 1 > ----------------- > > > > Docker entrypoint.sh have following code > > SKIP_SCHEMA_INIT="${IS_RESUME:-false}" > > function initialize_hive { > COMMAND="-initOrUpgradeSchema" > if [ "$(echo "$HIVE_VER" | cut -d '.' -f1)" -lt "4" ]; then > COMMAND="-${SCHEMA_COMMAND:-initSchema}" > fi > $HIVE_HOME/bin/schematool -dbType $DB_DRIVER $COMMAND > if [ $? -eq 0 ]; then > echo "Initialized schema successfully.." > else > echo "Schema initialization failed!" > exit 1 > fi > } > > export HIVE_CONF_DIR=$HIVE_HOME/conf > if [ -d "${HIVE_CUSTOM_CONF_DIR:-}" ]; then > find "${HIVE_CUSTOM_CONF_DIR}" -type f -exec \ > ln -sfn {} "${HIVE_CONF_DIR}"/ \; > export HADOOP_CONF_DIR=$HIVE_CONF_DIR > export TEZ_CONF_DIR=$HIVE_CONF_DIR > fi > > export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Xmx1G $SERVICE_OPTS" > if [[ "${SKIP_SCHEMA_INIT}" == "false" ]]; then > # handles schema initialization > initialize_hive > fi > > \ >