Paresh Devalia created ATLAS-5034:
-------------------------------------
Summary: Entities not created in Atlas when hive table created on
top a OFS bucket/volume
Key: ATLAS-5034
URL: https://issues.apache.org/jira/browse/ATLAS-5034
Project: Atlas
Issue Type: Bug
Reporter: Paresh Devalia
Assignee: Aditya Gupta
Facing issues when trying to create hive tables on top of ozone ofs
buckets/volumes
Steps to reproduce
{code:java}
# ozone sh volume create /volume1
# ozone sh bucket create /volume1/bucket1
# ozone sh bucket create /volume1/bucket1.bucket1.bucket1
# beeline
0: jdbc:hive2://altas-hive> CREATE EXTERNAL TABLE table_volume(id int) row
format delimited fields terminated by ' ' stored as textfile location
'ofs://ozone1724268703/volume1';
INFO : Compiling
command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44):
CREATE EXTERNAL TABLE table_volume(id int) row format delimited fields
terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1'
INFO : Semantic Analysis Completed (retrial = false)
INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling
command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44); Time
taken: 0.053 seconds
INFO : Executing
command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44):
CREATE EXTERNAL TABLE table_volume(id int) row format delimited fields
terminated by ' ' stored as textfile location 'ofs://ozone1724268703/volume1'
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=hive_20240822174025_98caa1a2-f814-4d83-9f9a-31b157a49a44); Time
taken: 0.417 seconds
INFO : OK
No rows affected (0.544 seconds)
0: jdbc:hive2://atlas-hive> CREATE EXTERNAL TABLE table_bucket_dotted(id int)
row format delimited fields terminated by ' ' stored as textfile location
'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1';
INFO : Compiling
command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa):
CREATE EXTERNAL TABLE table_bucket_dotted(id int) row format delimited fields
terminated by ' ' stored as textfile location
'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1'
INFO : Semantic Analysis Completed (retrial = false)
INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling
command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa); Time
taken: 0.182 seconds
INFO : Executing
command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa):
CREATE EXTERNAL TABLE table_bucket_dotted(id int) row format delimited fields
terminated by ' ' stored as textfile location
'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1'
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=hive_20240822172811_7868d63e-ff31-462b-bcce-7dd6fe9d49fa); Time
taken: 0.465 seconds
INFO : OK
No rows affected (1.155 seconds)
0: jdbc:hive2://atlas-hive> CREATE EXTERNAL TABLE table_key_bucket_dotted(id
int) row format delimited fields terminated by ' ' stored as textfile location
'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1/key2';
INFO : Compiling
command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421):
CREATE EXTERNAL TABLE table_key_bucket_dotted(id int) row format delimited
fields terminated by ' ' stored as textfile location
'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1/key2'
INFO : Semantic Analysis Completed (retrial = false)
INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling
command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421); Time
taken: 0.026 seconds
INFO : Executing
command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421):
CREATE EXTERNAL TABLE table_key_bucket_dotted(id int) row format delimited
fields terminated by ' ' stored as textfile location
'ofs://ozone1724268703/volume1/bucket1.bucket1.bucket1/key2'
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=hive_20240822173256_3da9c7ff-76b8-4116-8b5b-929428b77421); Time
taken: 0.226 seconds
INFO : OK
No rows affected (0.342 seconds)
0: jdbc:hive2://atlas-hive> show tables;
INFO : Compiling
command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515): show
tables
INFO : Semantic Analysis Completed (retrial = false)
INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name,
type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling
command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515); Time
taken: 0.045 seconds
INFO : Executing
command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515): show
tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=hive_20240822174219_dc6e9ada-4c30-4e82-84cb-d2c0ec071515); Time
taken: 0.026 seconds
INFO : OK
+--------------------------+
| tab_name |
+--------------------------+
| sample_07 |
| sample_08 |
| table_bucket_dotted |
| table_key |
| table_key_bucket_dotted |
| table_volume |
| web_logs |
+--------------------------+
In Atlas, key based tables are created properly (eg table_key,
table_key_bucket_dotted). However, bucket/volume based tables do not show up in
Atlas
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
