pabloem commented on code in PR #26355:
URL: https://github.com/apache/beam/pull/26355#discussion_r1189111294
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/BigtableChangeStreamAccessor.java:
##########
@@ -100,9 +101,30 @@ private static BigtableChangeStreamAccessor
createAccessor(@NonNull BigtableConf
dataSettingsBuilder.setInstanceId(instanceId);
tableAdminSettingsBuilder.setInstanceId(instanceId);
- if (appProfileId != null) {
- dataSettingsBuilder.setAppProfileId(appProfileId);
- }
+ String appProfileId =
checkArgumentNotNull(bigtableConfig.getAppProfileId()).get();
+ dataSettingsBuilder.setAppProfileId(appProfileId);
+
+ dataSettingsBuilder
+ .stubSettings()
+ .setTransportChannelProvider(
+ EnhancedBigtableStubSettings.defaultGrpcTransportProviderBuilder()
+ .setAttemptDirectPath(false) // Disable DirectPath
+ .setChannelPoolSettings( // Autoscale Channel Size
+ ChannelPoolSettings.builder()
+ // Make sure that there are at least 2 channels
regardless of RPCs
+ .setMinChannelCount(2)
+ // Limit number of channels to 100 regardless of QPS
+ .setMaxChannelCount(100)
+ // Start off with 5
+ .setInitialChannelCount(5)
+ // Make sure the channels are primed before use
+ .setPreemptiveRefreshEnabled(true)
+ // evict channels when there are less than 10
outstanding RPCs
+ .setMinRpcsPerChannel(10)
+ // add more channels when the channel has 50
outstanding RPCs
+ .setMaxRpcsPerChannel(50)
+ .build())
Review Comment:
just curious - how did you arrive at these numbers? (and the below 10
attempts?) - maybe worth mentioning in comments whether you tested with these
and so on
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
