[ https://issues.apache.org/jira/browse/KUDU-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554207#comment-17554207 ]
ASF subversion and git services commented on KUDU-2671: ------------------------------------------------------- Commit 9091f31cfc5c102b6a734fa20b015735251392d4 in kudu's branch refs/heads/master from Alexey Serbin [ https://gitbox.apache.org/repos/asf?p=kudu.git;h=9091f31cf ] KUDU-2671 refactor PartitionSchema::CreatePartitions() This patch refactors the code of the PartitionSchema::CreatePartitions() method and the code around its invocations, addressing a few TODOs. As a consequence, this patch fixes a bug in the CreateTable path: prior to this patch, a client could submit a CreateTableRequestPB request such that the order of ranges in CreateTableRequestPB::split_rows_range_bounds and CreateTableRequestPB::partition_schema::custom_hash_schema_ranges were different, and the result table would be created with wrong hash schemas. For example: requested: {{range_boundary_a0, range_boundary_a1}, hash_schema_a} {{range_boundary_b0, range_boundary_b1}, hash_schema_b} created: {{range_boundary_a0, range_boundary_a1}, hash_schema_b} {{range_boundary_b0, range_boundary_b1}, hash_schema_a} I'm planning to add corresponding test scenario to catch the regressions in a separate changelist for ease of reviewing and tracking the changes. However, I updated the existing test scenarios affected by this change. Change-Id: I5bac1f8ee349577e2f912530a28776415ed0a5b0 Reviewed-on: http://gerrit.cloudera.org:8080/18582 Tested-by: Alexey Serbin <ale...@apache.org> Reviewed-by: Abhishek Chennaka <achenn...@cloudera.com> Reviewed-by: Attila Bukor <abu...@apache.org> > Change hash number for range partitioning > ----------------------------------------- > > Key: KUDU-2671 > URL: https://issues.apache.org/jira/browse/KUDU-2671 > Project: Kudu > Issue Type: Improvement > Components: client, java, master, server > Affects Versions: 1.8.0 > Reporter: yangz > Assignee: Mahesh Reddy > Priority: Major > Labels: feature, roadmap-candidate, scalability > Attachments: 屏幕快照 2019-01-24 下午12.03.41.png > > > For our usage, the kudu schema design isn't flexible enough. > We create our table for day range such as dt='20181112' as hive table. > But our data size change a lot every day, for one day it will be 50G, but for > some other day it will be 500G. For this case, it be hard to set the hash > schema. If too big, for most case, it will be too wasteful. But too small, > there is a performance problem in the case of a large amount of data. > > So we suggest a solution we can change the hash number by the history data of > a table. > for example > # we create schema with one estimated value. > # we collect the data size by day range > # we create new day range partition by our collected day size. > We use this feature for half a year, and it work well. We hope this feature > will be useful for the community. Maybe the solution isn't so complete. > Please help us make it better. -- This message was sent by Atlassian Jira (v8.20.7#820007)