[
https://issues.apache.org/jira/browse/HUDI-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17391128#comment-17391128
]
Biswajit mohapatra edited comment on HUDI-2199 at 8/1/21, 10:43 AM:
--------------------------------------------------------------------
so we will need properties for dynamodb also , like in emr we have a table name
emrfs , can we have a table like that called as hudifs which gets created if
the table doesn't exist ?
Just one question does it creates a table for each table name ingested or it
uses a generic table for table ingested into hudi ?
also can you say me the class name for hbase from where i can get an idea ?
EDIT
i was just checking some hbase implementation classes in hudi
i found out it stores the data in this format
partition_path -> is it string ?
, fileID ---> string or number ?
commitTime -> new Date().getTime();
As in dynamodb there is a concept of PK and SK can you let me know what would
be the pk and sk for this would be and can you give an example of data that
gets stored here ?
was (Author: biswajit11):
so we will need properties for dynamodb also , like in emr we have a table name
emrfs , can we have a table like that called as hudifs which gets created if
the table doesn't exist ?
Just one question does it creates a table for each table name ingested or it
uses a generic table for table ingested into hudi ?
also can you say me the class name for hbase from where i can get an idea ?
EDIT
i was just checking some hbase implementation classes in hudi
i found out it stores the data in this format
partition_path -> is it string ?
, fileID ---> string or number ?
commitTime -> what is the format of the time ?
As in dynamodb there is a concept of PK and SK can you let me know what would
be the pk and sk for this would be and can you give an example of data that
gets stored here ?
> DynamoDB based external index implementation
> --------------------------------------------
>
> Key: HUDI-2199
> URL: https://issues.apache.org/jira/browse/HUDI-2199
> Project: Apache Hudi
> Issue Type: New Feature
> Components: Index
> Reporter: Vinoth Chandar
> Assignee: Biswajit mohapatra
> Priority: Major
>
> We have a HBaseIndex, that provides uses with ability to store fileID <=>
> recordKey mappings in an external kv store, for fast lookups during upsert
> operations. We can potentially create a similar one for DynamoDB.
> We just use a single column family in HBase, so we should be able to largely
> re-use the code/key-value schema across them even.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
