[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1527544632 @codope Thanks for your testing. It's so confusing. I just use `hive` cli without any other conf. My hive version is 3.1.2, I don't have hive2, what's your hive version? Is this a matter of version differences？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1519710563 steps to reproduce 1. create table in spark sql ``` create table test_partition ( id bigint, name string, ts bigint, part string ) using hudi tblproperties ( type = 'mor', primaryKey = 'id', preCombineField = 'ts', 'hoodie.embed.timeline.server' = 'false', 'hoodie.datasource.hive_sync.enable'='true', 'hoodie.datasource.hive_sync.mode'='HMS', 'hoodie.datasource.hive_sync.table'='test_partition', 'hoodie.datasource.hive_sync.metastore.uris'='thrift://host-10-19-37-172:9083', 'hoodie.datasource.hive_sync.partition_fields'='part' ) partitioned by (part) location '/tmp/hoodie/test_partition'; ``` 2. insert a record ``` insert into test_partition partition (part) select 1 as id, 'a1' as name, 1000 as ts, '2021-12-09' as part; ``` 3. query in hive ``` select * from test_partition_ro where part = '2021-12-09'; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1519706117  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1519705139  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355:
URL: https://github.com/apache/hudi/pull/7355#issuecomment-1519702809
```
++
| createtab_stmt |
++
| CREATE EXTERNAL TABLE `test_partition`(|
| `_hoodie_commit_time` string,|
| `_hoodie_commit_seqno` string, |
| `_hoodie_record_key` string, |
| `_hoodie_partition_path` string, |
| `_hoodie_file_name` string, |
| `id` bigint, |
| `name` string, |
| `ts` bigint) |
| PARTITIONED BY ( |
| `part` string) |
| ROW FORMAT SERDE |
| 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' |
| WITH SERDEPROPERTIES ( |
| 'path'='hdfs://xxx/tmp/hoodie/test_partition') |
| STORED AS INPUTFORMAT |
| 'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat' |
| OUTPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' |
| LOCATION |
| 'hdfs://xx/tmp/hoodie/test_partition' |
| TBLPROPERTIES (|
| 'bucketing_version'='2', |
| 'hoodie.datasource.hive_sync.enable'='true', |
| 'hoodie.datasource.hive_sync.metastore.uris'='thrift://:9083', |
| 'hoodie.datasource.hive_sync.mode'='HMS',|
| 'hoodie.datasource.hive_sync.partition_fields'='part', |
| 'hoodie.datasource.hive_sync.table'='test_partition', |
| 'hoodie.embed.timeline.server'='false', |
| 'preCombineField'='ts', |
| 'primaryKey'='id', |
| 'spark.sql.create.version'='3.1.1', |
| 'spark.sql.sources.provider'='hudi', |
| 'spark.sql.sources.schema.numPartCols'='1', |
| 'spark.sql.sources.schema.numParts'='1', |
|
'spark.sql.sources.schema.part.0'='{"type":"struct","fields":[{"name":"_hoodie_commit_time","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_commit_seqno","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_record_key","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_partition_path","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_file_name","type":"string","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"name","type":"string","nullable":true,"metadata":{}},{"name":"ts","type":"long","nullable":true,"metadata":{}},{"name":"part","type":"string","nullable":true,"metadata":{}}]}',
|
| 'spark.sql.sources.schema.partCol.0'='part', |
| 'transient_lastDdlTime'='1682327384',|
| 'type'='mor')|
++
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1519368205 > Is the problematic table partitioning in hive style: `par1=val1` ? I was astonished that the partition path queries for Hive return nulls. Hive style and none hive style all returns null. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1457335604 > Which version of Hive has this issue ? I test with hive 3.1.2, I believe hive 2.x has this issue too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1369454072 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1368159302 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1367685026 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1367282191 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1352843272 The ci failure is caused by running longer than the maximum time of 150 minutes. Hive test has passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
