Hi Sanjay,
Instead of using --env SKIP_SCHEMA_INIT=TRUE,
Please use --env IS_RESUME="true" while running,
docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore
--add-host=host.docker.internal:host-gateway \
--env IS_RESUME="true" \
--env DB_DRIVER=mysql \
--env
SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=****" \
--mount source=warehouse,target=/opt/hive/data/warehouse \
--name metastore-standalone apache/hive:${HIVE_VERSION} /bin/bash
This should skip the schema initOrUpgrade process:
https://github.com/apache/hive/blob/5022b85b5f50615f85da07bce42aebd414deb9b0/packaging/src/docker/entrypoint.sh#L24
from the 2nd time you run the container.
Regards,
Akshat
On Fri, Dec 22, 2023 at 11:53 AM Sanjay Gupta <sanja...@gmail.com> wrote:
> Hi All,
>
> If my metastore schema already exists with correct version so what I
> need to do so it doesn't do init or upgrade when starting metastore
> container
>
> I have tried following command line
>
>
> On MAC environment variables
> export HIVE_VERSION=3.1.3
> and even
> SKIP_SCHEMA_INIT=TRUE
>
> docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore
> --add-host=host.docker.internal:host-gateway \
> --env SKIP_SCHEMA_INIT=TRUE \
> --env DB_DRIVER=mysql \
> --env
> SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
>
>
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
> -Djavax.jdo.option.ConnectionUserName=hive
> -Djavax.jdo.option.ConnectionPassword=****" \
> --mount source=warehouse,target=/opt/hive/data/warehouse \
> --name metastore-standalone apache/hive:${HIVE_VERSION} /bin/bash
>
>
>
> -----------------------
>
> Docker Logs , it still tries to initSchema
>
> docker logs 1c
> + : mysql
> + SKIP_SCHEMA_INIT=false
> + export HIVE_CONF_DIR=/opt/hive/conf
> + HIVE_CONF_DIR=/opt/hive/conf
> + '[' -d '' ']'
> + export 'HADOOP_CLIENT_OPTS= -Xmx1G
> -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
>
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
> -Djavax.jdo.option.ConnectionUserName=hive
> -Djavax.jdo.option.ConnectionPassword=hive'
> + HADOOP_CLIENT_OPTS=' -Xmx1G
> -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
>
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
> -Djavax.jdo.option.ConnectionUserName=hive
> -Djavax.jdo.option.ConnectionPassword=hive'
> + [[ false == \f\a\l\s\e ]]
> + initialize_hive
> + COMMAND=-initOrUpgradeSchema
> ++ cut -d . -f1
> ++ echo 3.1.3
> + '[' 3 -lt 4 ']'
> + COMMAND=-initSchema
> + /opt/hive/bin/schematool -dbType mysql -initSchema
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
>
> [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
>
> [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> Metastore connection URL:
> jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
> Metastore Connection Driver : com.mysql.cj.jdbc.Driver
> Metastore connection User: hive
> Starting metastore schema initialization to 3.1.0
> Initialization script hive-schema-3.1.0.mysql.sql
>
>
> Error: Table 'ctlgs' already exists (state=42S01,code=1050)
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema
> initialization FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> Use --verbose for detailed stacktrace.
> *** schemaTool failed ***
> [WARN] Failed to create directory: /home/hive/.beeline
> No such file or directory
> + '[' 1 -eq 0 ']'
> + echo 'Schema initialization failed!'
> Schema initialization failed!
> + exit 1
> -----------------
>
>
>
> Docker entrypoint.sh have following code
>
> SKIP_SCHEMA_INIT="${IS_RESUME:-false}"
>
> function initialize_hive {
> COMMAND="-initOrUpgradeSchema"
> if [ "$(echo "$HIVE_VER" | cut -d '.' -f1)" -lt "4" ]; then
> COMMAND="-${SCHEMA_COMMAND:-initSchema}"
> fi
> $HIVE_HOME/bin/schematool -dbType $DB_DRIVER $COMMAND
> if [ $? -eq 0 ]; then
> echo "Initialized schema successfully.."
> else
> echo "Schema initialization failed!"
> exit 1
> fi
> }
>
> export HIVE_CONF_DIR=$HIVE_HOME/conf
> if [ -d "${HIVE_CUSTOM_CONF_DIR:-}" ]; then
> find "${HIVE_CUSTOM_CONF_DIR}" -type f -exec \
> ln -sfn {} "${HIVE_CONF_DIR}"/ \;
> export HADOOP_CONF_DIR=$HIVE_CONF_DIR
> export TEZ_CONF_DIR=$HIVE_CONF_DIR
> fi
>
> export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Xmx1G $SERVICE_OPTS"
> if [[ "${SKIP_SCHEMA_INIT}" == "false" ]]; then
> # handles schema initialization
> initialize_hive
> fi
>
> \
>
