[
https://issues.apache.org/jira/browse/SENTRY-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671815#comment-16671815
]
kalyan kumar kalvagadda commented on SENTRY-2249:
-------------------------------------------------
Batching in datanucleus works as below. It batches the commands and sends them
to datastore at once.
{noformat}
BATCH [INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<1>,<1>,<'/u/h/w/db9/tb6/30'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<2>,<1>,<'/u/h/w/db9/tb6/9'>); INSERT
INT O AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<3>,<1>,<'/u/h/w/db9/tb6/8'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<4>,<1>,<'/u/h/w/db9/tb6/7'>); INSERT
INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<5>,<1>,<'/u/h/w/db9/tb6/6'>); INSERT INTO AUTHZ_PATH (PATH_ID,AU
THZ_OBJ_ID,PATH_NAME) VALUES (<6>,<1>,<'/u/h/w/db9/tb6/5'>); INSERT INTO
AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<7>,<1>,<'/u/h/w/db9/tb6/4'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<8>,<1>,<'/u/h/w/db9/tb6/3'>); INSERT
INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VA LUES
(<9>,<1>,<'/u/h/w/db9/tb6/26'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<10>,<1>,<'/u/h/w/db9/tb6/2'>); INSERT
INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<11>,<1>,<'/u/h/w/db9/tb6/27'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<12>,<1>,<'/u/ h/w/db9/tb6/1'>);
INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<13>,<1>,<'/u/h/w/db9/tb6/24'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<14>,<1>,<'/u/h/w/db9/tb6/25'>);
INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<15>,<1>,<'/u/h/w/db9/tb6/22'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<16>,<1>,<'/u/h/w/db9/tb6/23'>);
INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<17>,<1>,<'/u/h/w/db9/tb6/20'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<18>,<1>,<'/u/h/w/db9/tb6/21'>);
INSERT INTO AUTHZ_ PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<19>,<1>,<'/u/h/w/db9/tb6/28'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<20>,<1>,<'/u/h/w/db9/tb6/29'>);
INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<21>,<1>,<'/u/h/w/db9/tb6/15'>); INSERT INTO AUTHZ_PATH (PATH_ID,AUTH
Z_OBJ_ID,PATH_NAME) VALUES (<22>,<1>,<'/u/h/w/db9/tb6/16'>); INSERT INTO
AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<23>,<1>,<'/u/h/w/db9/tb6/13'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<24>,<1>,<'/u/h/w/db9/tb6/14'>);
INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME ) VALUES
(<25>,<1>,<'/u/h/w/db9/tb6/11'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<26>,<1>,<'/u/h/w/db9/tb6/12'>);
INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<27>,<1>,<'/u/h/w/db9/tb6/10'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<28>,<1>
,<'/u/h/w/db9/tb6/19'>); INSERT INTO AUTHZ_PATH
(PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES (<29>,<1>,<'/u/h/w/db9/tb6/17'>);
INSERT INTO AUTHZ_PATH (PATH_ID,AUTHZ_OBJ_ID,PATH_NAME) VALUES
(<30>,<1>,<'/u/h/w/db9/tb6/18'>)]
{noformat}
> Enable batch insert of HMS paths in Full Snapshot.
> ---------------------------------------------------
>
> Key: SENTRY-2249
> URL: https://issues.apache.org/jira/browse/SENTRY-2249
> Project: Sentry
> Issue Type: Improvement
> Components: Sentry
> Affects Versions: 2.1.0
> Reporter: kalyan kumar kalvagadda
> Assignee: kalyan kumar kalvagadda
> Priority: Major
> Attachments: SENTRY-2249.001.patch, SENTRY-2249.002.patch,
> SENTRY-2249.003.patch, SENTRY-2249.004.patch, SENTRY-2249.005.patch
>
>
> Currently each entry in full snapshot of HMS is persisted one entry at a
> time. Instead it could be optimized by persisting the path entries in
> batches. DB operations are expensive, reducing the number of database
> operations and around trip time will help. This would decrease the time to
> persist the snapshot in to database significantly.
> Size of the batch could be configurable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
