ShiminHuang created BAHIR-255:
---------------------------------
Summary: flink-kudu-sql add hash partitions nums config&onwer
config
Key: BAHIR-255
URL: https://issues.apache.org/jira/browse/BAHIR-255
Project: Bahir
Issue Type: Improvement
Components: Flink Streaming Connectors
Affects Versions: Flink-Next
Environment: maxos
Reporter: ShiminHuang
Fix For: Flink-Next
h1. Hash Partition Feature
* The default number of buckets in the previous hash partition was 2*
replicas, and there were some problems in the production environment, so I felt
I should provide the displayed hash partition configuration.
* Add the hash partition number configuration 'kudu.hash-partition-nums',which
defaults to 3 if not specified.
h1. Singleton Test
{code:java}
public class NewTablePropertiesTest extends KuduCatalogTest{
private KuduCatalog catalog;
private StreamTableEnvironment tableEnv;
@BeforeEach
public void init() {
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
catalog = new KuduCatalog(harness.getMasterAddressesAsString());
tableEnv =
KuduTableTestUtils.createTableEnvWithBlinkPlannerStreamingMode(env);
tableEnv.registerCatalog("kudu", catalog);
tableEnv.useCatalog("kudu");
}
@Test
public void testHashPartitionNums() throws TableNotExistException {
catalog.dropTable(ObjectPath.fromString(EnvironmentSettings.DEFAULT_BUILTIN_DATABASE
+ ".TestTable1"), true);
catalog.dropTable(ObjectPath.fromString(EnvironmentSettings.DEFAULT_BUILTIN_DATABASE
+ ".TestTable2"), true);
tableEnv.executeSql("CREATE TABLE TestTable1 (`first` STRING, `second`
String) WITH ('kudu.hash-columns' = 'first', 'kudu.primary-key-columns' =
'first')");
tableEnv.executeSql("CREATE TABLE TestTable2 (`first` STRING, `second`
String) WITH ('kudu.hash-columns' = 'first','kudu.hash-partition-nums'='6',
'kudu.primary-key-columns' = 'first')");
}
@Test
public void testOwner() throws TableNotExistException {
catalog.dropTable(ObjectPath.fromString(EnvironmentSettings.DEFAULT_BUILTIN_DATABASE
+ ".TestTable1"), true);
tableEnv.executeSql("CREATE TABLE TestTable1 (`first` STRING, `second`
String) WITH ('kudu.hash-columns' = 'first', 'kudu.primary-key-columns' =
'first','kudu.table-owner'='admin')");
}
}
{code}
h1. Other Ideas
* Regarding the range partition, I think a JSON List can be passed in to
describe the corresponding range partition, and I think the hash partition and
range partition must exist in one
h1.
h1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
