Re: creating database issue

2021-12-07 Thread bitfox 

And, I can't start spark-sql shell, the error as below.
Does this mean I need to install Hive on local machine?

Caused by: java.sql.SQLException: Failed to start database 
'metastore_db' with class loader 
jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27, see the next 
exception for details.
	at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown 
Source)
	at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown 
Source)

at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown 
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source)
at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown 
Source)

at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
at com.jolbox.bonecp.BoneCP.(BoneCP.java:416)
... 89 more
Caused by: ERROR XJ040: Failed to start database 'metastore_db' with 
class loader jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27, 
see the next exception for details.
	at org.apache.derby.iapi.error.StandardException.newException(Unknown 
Source)
	at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown 
Source)

... 105 more


Thanks.

On 2021/12/8 9:28, bitfox wrote:

Hello

This is just a standalone deployment for testing purpose.
The version:
Spark 3.2.0 (git revision 5d45a415f3) built for Hadoop 3.3.1
Build flags: -B -Pmesos -Pyarn -Pkubernetes -Psparkr -Pscala-2.12 
-Phadoop-3.2 -Phive -Phive-thriftserver


I just started one master and one worker for the test.

Thanks


On 2021/12/8 9:15, Qian Sun wrote:
   It seems to be a hms question. Would u like to provide the 
information about spark version, hive version and spark application 
configuration?


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org



-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: creating database issue

2021-12-07 Thread bitfox 

Hello

This is just a standalone deployment for testing purpose.
The version:
Spark 3.2.0 (git revision 5d45a415f3) built for Hadoop 3.3.1
Build flags: -B -Pmesos -Pyarn -Pkubernetes -Psparkr -Pscala-2.12 
-Phadoop-3.2 -Phive -Phive-thriftserver


I just started one master and one worker for the test.

Thanks


On 2021/12/8 9:15, Qian Sun wrote:

   It seems to be a hms question. Would u like to provide the information about 
spark version, hive version and spark application configuration?


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: creating database issue

2021-12-07 Thread Qian Sun 
Hi,
 
  It seems to be a hms question. Would u like to provide the information about 
spark version, hive version and spark application configuration?

Best

> 2021年12月8日 上午9:04，bitfox  写道：
> 
> sorry I am newbie to spark.
> 
> When I created a database in pyspark shell following the book content of 
> learning spark 2.0, it gets:
> 
> >>> spark.sql("CREATE DATABASE learn_spark_db")
> 21/12/08 09:01:34 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout 
> does not exist
> 21/12/08 09:01:34 WARN HiveConf: HiveConf of name hive.stats.retries.wait 
> does not exist
> 21/12/08 09:01:39 WARN ObjectStore: Version information not found in 
> metastore. hive.metastore.schema.verification is not enabled so recording the 
> schema version 2.3.0
> 21/12/08 09:01:39 WARN ObjectStore: setMetaStoreSchemaVersion called but 
> recording version is disabled: version = 2.3.0, comment = Set by MetaStore 
> pyh@185.213.174.249
> 21/12/08 09:01:40 WARN ObjectStore: Failed to get database default, returning 
> NoSuchObjectException
> 21/12/08 09:01:40 WARN ObjectStore: Failed to get database global_temp, 
> returning NoSuchObjectException
> 21/12/08 09:01:40 WARN ObjectStore: Failed to get database learn_spark_db, 
> returning NoSuchObjectException
> 
> Can you point to me where is wrong?
> 
> Thanks.
> 
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> 


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

creating database issue

2021-12-07 Thread bitfox 

sorry I am newbie to spark.

When I created a database in pyspark shell following the book content of 
learning spark 2.0, it gets:


>>> spark.sql("CREATE DATABASE learn_spark_db")
21/12/08 09:01:34 WARN HiveConf: HiveConf of name 
hive.stats.jdbc.timeout does not exist
21/12/08 09:01:34 WARN HiveConf: HiveConf of name 
hive.stats.retries.wait does not exist
21/12/08 09:01:39 WARN ObjectStore: Version information not found in 
metastore. hive.metastore.schema.verification is not enabled so 
recording the schema version 2.3.0
21/12/08 09:01:39 WARN ObjectStore: setMetaStoreSchemaVersion called but 
recording version is disabled: version = 2.3.0, comment = Set by 
MetaStore pyh@185.213.174.249
21/12/08 09:01:40 WARN ObjectStore: Failed to get database default, 
returning NoSuchObjectException
21/12/08 09:01:40 WARN ObjectStore: Failed to get database global_temp, 
returning NoSuchObjectException
21/12/08 09:01:40 WARN ObjectStore: Failed to get database 
learn_spark_db, returning NoSuchObjectException


Can you point to me where is wrong?

Thanks.

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org