Re: Re: KUDU INSERT SLOWLY
(putting dev@ in bcc again, please be mindful of which mailing list you write to) The best way would be to follow the "Batch insert" method described here: http://kudu.apache.org/docs/kudu_impala_integration.html#kudu_impala_insert_bulk J-D On Wed, Aug 23, 2017 at 6:30 PM, sky wrote: > Thank you for your method. > Is there any other way to quickly load text files into kudu table ? > > > > > > > > > At 2017-08-22 16:12:47, "Balázs Jeszenszky" wrote: > >[dev@ to bcc] > > > >Hello, > > > >Setting up a query has some overhead (in query planning). I expect it > >to be in the 100 ms range, multiplying that by 2k adds up to the time > >you report. > >Inserting all the values through one query would help. > > > >HTH > > > >On Tue, Aug 22, 2017 at 10:06 AM, sky wrote: > >> Hi all, > >> I do kudu insert sql through the impala shell, it is very > slowly.Approximately 4 minutes two thousand insert sqls. Is there any way > to speed up kudu insert speed? > >> Command: > >> ./impala-shell.sh -f a.sql > >> a.sql: > >> insert into table t1 values("a1","b"); > >> insert into table t1 valuse("a2","b"); > >> .. > > > > > > > >-- > >Balázs Jeszenszky > >Customer Operations Engineer > >Email: bala...@cloudera.com > >Data:http://www.cloudera.com >
Re: KUDU INSERT SLOWLY
There are some improvements we hope to make in Impala to reduce the planning overhead for these simple DML statements. You can track that work with: https://issues.apache.org/jira/browse/IMPALA-5277 -Matt On Tue, Aug 22, 2017 at 1:12 AM, Balázs Jeszenszky wrote: > [dev@ to bcc] > > Hello, > > Setting up a query has some overhead (in query planning). I expect it > to be in the 100 ms range, multiplying that by 2k adds up to the time > you report. > Inserting all the values through one query would help. > > HTH > > On Tue, Aug 22, 2017 at 10:06 AM, sky wrote: >> Hi all, >> I do kudu insert sql through the impala shell, it is very >> slowly.Approximately 4 minutes two thousand insert sqls. Is there any way to >> speed up kudu insert speed? >> Command: >> ./impala-shell.sh -f a.sql >> a.sql: >> insert into table t1 values("a1","b"); >> insert into table t1 valuse("a2","b"); >> .. > > > > -- > Balázs Jeszenszky > Customer Operations Engineer > Email: bala...@cloudera.com > Data:http://www.cloudera.com
Re: KUDU INSERT SLOWLY
[dev@ to bcc] Hello, Setting up a query has some overhead (in query planning). I expect it to be in the 100 ms range, multiplying that by 2k adds up to the time you report. Inserting all the values through one query would help. HTH On Tue, Aug 22, 2017 at 10:06 AM, sky wrote: > Hi all, > I do kudu insert sql through the impala shell, it is very > slowly.Approximately 4 minutes two thousand insert sqls. Is there any way to > speed up kudu insert speed? > Command: > ./impala-shell.sh -f a.sql > a.sql: > insert into table t1 values("a1","b"); > insert into table t1 valuse("a2","b"); > .. -- Balázs Jeszenszky Customer Operations Engineer Email: bala...@cloudera.com Data:http://www.cloudera.com