I'm still looking into this, to figure out a way to add HIVE_LOCKS table in the Spark side. Anyway I will create an issue first to track this.
Best regards, Saisai Ryan Blue <rb...@netflix.com> 于2019年8月9日周五 上午4:58写道： > Any ideas on how to fix this? Can we create the HIVE_LOCKS table if it is > missing automatically? > > On Wed, Aug 7, 2019 at 7:13 PM Saisai Shao <sai.sai.s...@gmail.com> wrote: > >> Thanks guys for your reply. >> >> I didn't do anything special, I don't even have a configured Hive. I just >> simply put the iceberg (assembly) jar into Spark and start a local Spark >> process. I think the built-in Hive version of Spark is 1.2.1-spark (has a >> slight pom change), and all the configurations related to SparkSQL/Hive are >> default. I guess the reason is like Anton mentioned, I will take a try by >> creating all tables (HIVE_LOCKS) using script. But I think we should fix >> it, this potentially stops user to do a quick start by using local spark. >> >> think the reason why it works in tests is because we create all tables >>> (including HIVE_LOCKS) using a script >>> >> >> Best regards, >> Saisai >> >> Anton Okolnychyi <aokolnyc...@apple.com> 于2019年8月7日周三 下午11:56写道： >> >>> I think the reason why it works in tests is because we create all tables >>> (including HIVE_LOCKS) using a script. I am not sure lock tables are always >>> created in embedded mode. >>> >>> > On 7 Aug 2019, at 16:49, Ryan Blue <rb...@netflix.com> wrote: >>> > >>> > This is the right list. Iceberg is fairly low in the stack, so most >>> questions are probably dev questions. >>> > >>> > I'm surprised that this doesn't work with an embedded metastore >>> because we use an embedded metastore in tests: >>> https://github.com/apache/incubator-iceberg/blob/master/hive/src/test/java/org/apache/iceberg/hive/TestHiveMetastore.java >>> > >>> > But we are also using Hive 1.2.1 and a metastore schema for 3.1.0. I >>> wonder if a newer version of Hive would avoid this problem? What version >>> are you linking with? >>> > >>> > On Tue, Aug 6, 2019 at 8:59 PM Saisai Shao <sai.sai.s...@gmail.com> >>> wrote: >>> > Hi team, >>> > >>> > I just met some issues when trying Iceberg with quick start guide. Not >>> sure if it is proper to send this to @dev mail list (seems there's no user >>> mail list). >>> > >>> > One issue is that seems current Iceberg cannot run with embedded >>> metastore. It will throw an exception. Is this an on-purpose behavior >>> (force to use remote HMS), or just a bug? >>> > >>> > Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Unable >>> to update transaction database java.sql.SQLSyntaxErrorException: Table/View >>> 'HIVE_LOCKS' does not exist. >>> > at >>> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown >>> Source) >>> > at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown >>> Source) >>> > at >>> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown >>> Source) >>> > >>> > Followed by this issue, seems like current Iceberg only binds to HMS >>> as catalog, this is fine for production usage. But I'm wondering if we >>> could have a simple catalog like in-memory catalog as Spark, so that it is >>> easy for user to test and play. Is there any concern or plan? >>> > >>> > Best regards, >>> > Saisai >>> > >>> > >>> > >>> > >>> > -- >>> > Ryan Blue >>> > Software Engineer >>> > Netflix >>> >>> > > -- > Ryan Blue > Software Engineer > Netflix >
