Emitswang opened a new issue #9628:
URL: https://github.com/apache/shardingsphere/issues/9628
## Feature Request
Request to optimize the startup speed for a slew of 'config-' files
### Is your feature request related to a problem?
In my case, there are 2000 schema config files, it's the same configuration
and just have diffrent schemaName and jdbcUrl.
For each schema config file load, it need about 500ms, so the proxy will
spent about 20min to start.
It's necessary to optimize the loading speed.
### Describe the feature you would like.
A. Provide new configuration to reduce schema files number , like
"schemaNames". We can set a list of schemaName use the same config contents.
B. Concurrent execute 'SchemaBuilder.build'.
C. Other ways which can optimize the loading speed.
**On the other hand, will the 2000 schema config cause some performance
problems in run time?**
### Shells
these shells to build 2000 config files
**File1:template.yaml**
```yaml
schemaName: V_SCHEMA_NAME
dataSources:
master:
url:
jdbc:mysql://V_MASTER_IP_PORT/V_SCHEMA_NAME?serverTimezone=UTC&useSSL=false
username: V_MASTER_NAME
password: V_MASTER_PASSW
connectionTimeoutMilliseconds: 30000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 30000
maxPoolSize: V_MASTER_POOL_SIZE
minPoolSize: V_MASTER_POOL_SIZE
slave:
url:
jdbc:mysql://V_SLAVE_IP_PORT/V_SCHEMA_NAME?serverTimezone=UTC&useSSL=false
username: V_SLAVE_NAME
password: V_SLAVE_PASSW
connectionTimeoutMilliseconds: 30000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 30000
maxPoolSize: V_SLAVE_POOL_SIZE
minPoolSize: V_SLAVE_POOL_SIZE
rules:
- !REPLICA_QUERY
dataSources:
db:
name: db
primaryDataSourceName: master
replicaDataSourceNames:
- slave
loadBalancerName: random
loadBalancers:
round:
type: ROUND_ROBIN
random:
type: RANDOM
```
**File2: create.sh**
```shell
for i in {0..19}
do
cat proxy.create | while read schemaName master masterName
masterPassw masterPool slave slaveName slavePassw slavePool
do
echo ${schemaName}
rm -f config-t${i}-${schemaName}.yaml
# copy template file
cp template.yaml config-t${i}-${schemaName}.yaml
# sed modify fields
sed -i "s/schemaName: V_SCHEMA_NAME/schemaName:
test${i}_${schemaName}/g;s/V_SCHEMA_NAME/${schemaName}/g;s/V_MASTER_IP_PORT/${master}/g;s/V_MASTER_NAME/${masterName}/g;s/V_MASTER_PASSW/${masterPassw}/g;s/V_MASTER_POOL_SIZE/${masterPool}/g;s/V_SLAVE_IP_PORT/${slave}/g;s/V_SLAVE_NAME/${slaveName}/g;s/V_SLAVE_PASSW/${slavePassw}/g;s/V_SLAVE_POOL_SIZE/${slavePool}/g;"
config-t${i}-${schemaName}.yaml
done
done
```
**File3: proxy.create**
```text
sharding_test_00_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_01_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_02_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_03_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_04_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_05_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_06_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_07_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_08_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_09_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_10_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_11_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_12_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_13_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_14_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_15_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_16_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_17_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_18_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_19_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_20_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_21_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_22_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_23_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_24_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_25_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_26_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_27_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_28_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_29_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_30_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_31_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_32_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_33_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_34_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_35_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_36_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_37_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_38_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_39_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_40_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_41_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_42_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_43_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_44_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_45_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_46_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_47_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_48_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_49_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_50_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_51_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_52_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_53_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_54_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_55_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_56_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_57_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_58_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_59_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_60_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_61_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_62_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_63_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_64_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_65_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_66_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_67_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_68_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_69_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_70_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_71_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_72_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_73_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_74_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_75_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_76_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_77_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_78_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_79_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_80_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_81_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_82_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_83_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_84_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_85_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_86_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_87_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_88_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_89_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_90_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_91_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_92_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_93_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_94_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_95_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_96_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_97_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_98_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
sharding_test_99_db 10.9.21.220:10000 root root 4 10.9.21.221:10000 root
root 4
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
