[
https://issues.apache.org/jira/browse/GOBBLIN-2185?focusedWorklogId=949929&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-949929
]
ASF GitHub Bot logged work on GOBBLIN-2185:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 24/Dec/24 06:33
Start Date: 24/Dec/24 06:33
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #4087:
URL: https://github.com/apache/gobblin/pull/4087#discussion_r1896450791
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/dynamic/ScalingDirectiveParser.java:
##########
@@ -263,29 +283,50 @@ public static String asString(ScalingDirective directive)
{
directive.getOptDerivedFrom().ifPresent(derivedFrom -> {
sb.append(',').append(derivedFrom.getBasisProfileName());
sb.append(derivedFrom.getOverlay() instanceof ProfileOverlay.Adding ?
"+(" : "-(");
- ProfileOverlay overlay = derivedFrom.getOverlay();
- if (overlay instanceof ProfileOverlay.Adding) {
- ProfileOverlay.Adding adding = (ProfileOverlay.Adding) overlay;
- for (ProfileOverlay.KVPair kv : adding.getAdditionPairs()) {
-
sb.append(kv.getKey()).append('=').append(urlEncode(kv.getValue())).append(",
");
- }
- if (adding.getAdditionPairs().size() > 0) {
- sb.setLength(sb.length() - 2); // remove trailing ", "
- }
- } else {
- ProfileOverlay.Removing removing = (ProfileOverlay.Removing) overlay;
- for (String key : removing.getRemovalKeys()) {
- sb.append(key).append(", ");
- }
- if (removing.getRemovalKeys().size() > 0) {
- sb.setLength(sb.length() - 2); // remove trailing ", "
- }
- }
+ sb.append(stringifyProfileOverlay(derivedFrom.getOverlay()));
sb.append(')');
});
return sb.toString();
}
+ /** @return the `scalingDirective` invariably stringified as two parts, a
{@link StringWithPlaceholderPlusOverlay} - regardless of stringified length */
+ public static StringWithPlaceholderPlusOverlay
asStringWithPlaceholderPlusOverlay(ScalingDirective directive) {
+ StringBuilder sb = new StringBuilder();
+
sb.append(directive.getTimestampEpochMillis()).append('.').append(directive.getProfileName()).append('=').append(directive.getSetPoint());
+ Optional<String> optProfileOverlayStr =
directive.getOptDerivedFrom().map(derivedFrom ->
+ stringifyProfileOverlay(derivedFrom.getOverlay())
+ );
+ directive.getOptDerivedFrom().ifPresent(derivedFrom -> {
+ sb.append(',').append(derivedFrom.getBasisProfileName());
+ sb.append(derivedFrom.getOverlay() instanceof ProfileOverlay.Adding ?
"+(" : "-(");
+ sb.append(OVERLAY_DEFINITION_PLACEHOLDER);
+ sb.append(')');
+ });
+ return new StringWithPlaceholderPlusOverlay(sb.toString(),
optProfileOverlayStr.orElse(""));
+ }
+
+ private static String stringifyProfileOverlay(ProfileOverlay overlay) {
+ StringBuilder sb = new StringBuilder();
+ if (overlay instanceof ProfileOverlay.Adding) {
+ ProfileOverlay.Adding adding = (ProfileOverlay.Adding) overlay;
+ for (ProfileOverlay.KVPair kv : adding.getAdditionPairs()) {
+
sb.append(kv.getKey()).append('=').append(urlEncode(kv.getValue())).append(",
");
+ }
+ if (adding.getAdditionPairs().size() > 0) {
+ sb.setLength(sb.length() - 2); // remove trailing ", "
+ }
+ } else {
+ ProfileOverlay.Removing removing = (ProfileOverlay.Removing) overlay;
+ for (String key : removing.getRemovalKeys()) {
+ sb.append(key).append(", ");
+ }
+ if (removing.getRemovalKeys().size() > 0) {
+ sb.setLength(sb.length() - 2); // remove trailing ", "
+ }
+ }
+ return sb.toString();
+ }
Review Comment:
it could, but truly this is the inverse of `ScalingDirectiveParser::parse`,
so it feels appropriate to live together in this class. after all, I'm not
ready define `asString(ScalingDirective)` as a `ScalingDirective` method.
usually, the particulars of a format should be separate from the object of
formatting, to allow multiple formats for the same type.
but you did help flag that I forgot handling of `ProfileOverlay.Combo`, so I
now throw `IllegalArgumentException`
Issue Time Tracking
-------------------
Worklog Id: (was: 949929)
Time Spent: 50m (was: 40m)
> Implement heuristic-based GoT Dynamic Auto-Scaling
> --------------------------------------------------
>
> Key: GOBBLIN-2185
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2185
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: gobblin-core
> Reporter: Kip Kohn
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Using a configured (constant) Data Transfer Rate (in bytes per time), presume
> a linear relationship holds between "Work" (WU) throughput and scaling the
> number of worker-containers. Provide a heuristic-based recommendation for
> how many worker-containers to allocate in order to complete processing of a
> job within a given time budget, with volume of Work conveyed via
> `WorkUnitsSizeSummary`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
