Re: Help with Docker Apache/Hive metastore using mysql remote database

2023-12-17 Thread Ayush Saxena
I think the similar problem is being chased as part of
https://github.com/apache/hive/pull/4948

On Mon, 18 Dec 2023 at 09:48, Sanjay Gupta  wrote:
>
>
>
>
> Issue with Docker container using mysql RDBMS ( Failed to load driver)
>
> https://hub.docker.com/r/apache/hive
>
> According to readme
>
> Launch Standalone Metastore With External RDBMS (Postgres/Oracle/MySql/MsSql)
>
> I want to use MySQL
>
> I tried com.mysql.jdbc.Driver or com.mysql.cj.jdbc.Driver
>
> docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore 
> --add-host=host.docker.internal:host-gateway \
>  --env DB_DRIVER=mysql \
>  --env 
> SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.jdbc.Driver 
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=password" \
>  --mount source=warehouse,target=/opt/hive/data/warehouse \
>  --name metastore-standalone apache/hive:${HIVE_VERSION}
>
>
> docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore 
> --add-host=host.docker.internal:host-gateway \
>  --env DB_DRIVER=mysql \
>  --env 
> SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
>   
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=password" \
>  --mount source=warehouse,target=/opt/hive/data/warehouse \
>  --name metastore-standalone apache/hive:${HIVE_VERSION}
>
> Docker logs shows this for both drivers ( same error )
>
> docker logs f3
> + : mysql
> + SKIP_SCHEMA_INIT=false
> + export HIVE_CONF_DIR=/opt/hive/conf
> + HIVE_CONF_DIR=/opt/hive/conf
> + '[' -d '' ']'
> + export 'HADOOP_CLIENT_OPTS= -Xmx1G 
> -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver  
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=hive'
> + HADOOP_CLIENT_OPTS=' -Xmx1G 
> -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver  
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=hive'
> + [[ false == \f\a\l\s\e ]]
> + initialize_hive
> + /opt/hive/bin/schematool -dbType mysql -initSchema
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Metastore connection URL: 
> jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
> Metastore Connection Driver : com.mysql.cj.jdbc.Driver
> Metastore connection User: hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
> Underlying cause: java.lang.ClassNotFoundException : com.mysql.cj.jdbc.Driver
> Use --verbose for detailed stacktrace.
> *** schemaTool failed ***
> + '[' 1 -eq 0 ']'
> + echo 'Schema initialization failed!'
> Schema initialization failed!
> + exit 1
>
> Any idea, why I am getting failed to load driver for MySQL DB.
>
> Isn't docker container comes with MySQL Driver ?
>
> Docker container exits so I can't check whether driver is already installed.
>
> Let me know, what I can do to make it work.
>
> --
>
>
> Thanks
> Sanjay Gupta
>
>
>
> --
>
> Thanks
> Sanjay Gupta
>
>
>
> --
>
> Thanks
> Sanjay Gupta
>


Fwd: Help with Docker Apache/Hive metastore using mysql remote database

2023-12-17 Thread Sanjay Gupta
Issue with Docker container using mysql RDBMS ( Failed to load driver)

https://hub.docker.com/r/apache/hive

According to readme

Launch Standalone Metastore With External RDBMS
(Postgres/Oracle/MySql/MsSql)

I want to use MySQL

I tried com.mysql.jdbc.Driver or com.mysql.cj.jdbc.Driver

docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore
--add-host=host.docker.internal:host-gateway \
 --env DB_DRIVER=mysql \
 --env 
SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.jdbc.Driver
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=password" \
 --mount source=warehouse,target=/opt/hive/data/warehouse \
 --name metastore-standalone apache/hive:${HIVE_VERSION}


docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore
--add-host=host.docker.internal:host-gateway \
 --env DB_DRIVER=mysql \
 --env 
SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
 
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=password" \
 --mount source=warehouse,target=/opt/hive/data/warehouse \
 --name metastore-standalone apache/hive:${HIVE_VERSION}

Docker logs shows this for both drivers ( same error )

docker logs f3
+ : mysql
+ SKIP_SCHEMA_INIT=false
+ export HIVE_CONF_DIR=/opt/hive/conf
+ HIVE_CONF_DIR=/opt/hive/conf
+ '[' -d '' ']'
+ export 'HADOOP_CLIENT_OPTS= -Xmx1G
-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=hive'
+ HADOOP_CLIENT_OPTS=' -Xmx1G
-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=hive'
+ [[ false == \f\a\l\s\e ]]
+ initialize_hive
+ /opt/hive/bin/schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:   
jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :com.mysql.cj.jdbc.Driver
Metastore connection User:   hive
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
Underlying cause: java.lang.ClassNotFoundException : com.mysql.cj.jdbc.Driver
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
+ '[' 1 -eq 0 ']'
+ echo 'Schema initialization failed!'
Schema initialization failed!
+ exit 1

Any idea, why I am getting failed to load driver for MySQL DB.

Isn't docker container comes with MySQL Driver ?

Docker container exits so I can't check whether driver is already installed.

Let me know, what I can do to make it work.

--


Thanks
Sanjay Gupta



-- 

Thanks
Sanjay Gupta



-- 

Thanks
Sanjay Gupta