This is an automated email from the ASF dual-hosted git repository.
asekretenko pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit a91747eaf60351507f16d8d167a0fa0207a9811a
Author: Andrei Sekretenko <asekrete...@apache.org>
AuthorDate: Thu Jul 16 15:48:03 2020 +0200
Added protobuf messages for regex-based offer constraints.
This patch adds protobuf messages for setting offer constraints
that check if agent's (pseudo)attribute matches a specified RE2 regular
expression.
Both added contsraint predicates will evaluate to `true` when the
attribute is not TEXT. This way, schedulers will have to apply on their
own whatever filtration they do for non-TEXT attributes which happen
to be selected by the constraint's `Selector`.
Given that in the real world schedulers seem to rarely put constraints
on attributes that are normally Scalar/Ranges, this should not prevent
them from obtaining performance benefits by setting offer constraints.
Review: https://reviews.apache.org/r/72784
---
include/mesos/scheduler/scheduler.proto | 53 ++++++++++++++++++++++++++++--
include/mesos/v1/scheduler/scheduler.proto | 53 ++++++++++++++++++++++++++++--
2 files changed, 102 insertions(+), 4 deletions(-)
diff --git a/include/mesos/scheduler/scheduler.proto
b/include/mesos/scheduler/scheduler.proto
index 06d62f5..f70738c 100644
--- a/include/mesos/scheduler/scheduler.proto
+++ b/include/mesos/scheduler/scheduler.proto
@@ -325,8 +325,54 @@ message AttributeConstraint {
required string value = 1;
}
- // TODO(asekretenko): add predicates for regular expression matching
- // (MESOS-10173).
+ // Predicates for regular expression constraints.
+ //
+ // The regular expressions are interpreted according to
+ // RE2 regular expression syntax and semantics:
+ // https://github.com/google/re2/blob/master/doc/syntax.txt
+ //
+ // The call using the constraints is invalid if the specified string is not
+ // a valid RE2 regex, or the regex is deemed too expensive to
+ // construct/store/execute, namely:
+ //
+ // - the RE2 object cannot be constructed from the regex without breaching
+ // the limit on the estimated memory footprint of an individual RE2
+ // (controlled by the --offer_constraints_re2_max_mem master flag), or
+ //
+ // - the regex program size reported by `RE2::ProgramSize()`
+ // exceeds the value of --offer_constraints_re2_max_program_size flag.
+ //
+ // Frameworks using the C++ implementation of RE2 (directly or via the JNI
+ // wrapper) for regexp validation on their side, are advised to set
+ // the `max_mem` RE2 option and limit `ProgramSize()` to values no larger
+ // than the defaults for the corresonding Mesos master flags.
+ //
+ // TODO(asekretenko): Provide an API for the frameworks and/or operators
+ // to validate regexp constraints against the Mesos master setup.
+ //
+ // Similarly to the string equality predicates, both regexp predicates
match
+ // (evaluate to `true`) when the (pseudo)attribute is not a TEXT/string.
+ // This way, schedulers will have to apply purely on their own
+ // whatever filtration they do for non-TEXT attributes which happen
+ // to be selected by the constraint's `Selector`
+
+ // Yields `true` if the (pseudo)attribute exists and is a TEXT/string
+ // matching the specified regex.
+ //
+ // Always yields `true` for existing non-TEXT attributes (and non-string
+ // pseudoattributes) for the reasons explained above.
+ message TextMatches {
+ required string regex = 1;
+ }
+
+ // Yields `true` if the (pseudo)attribute either does not exists or is
+ // not a TEXT/string matching the specified regex.
+ //
+ // Always yields `true` for existing non-TEXT attributes (and non-string
+ // pseudoattributes) for the reasons explained above.
+ message TextNotMatches {
+ required string regex = 1;
+ }
oneof predicate {
Exists exists = 1;
@@ -334,6 +380,9 @@ message AttributeConstraint {
TextEquals text_equals = 3;
TextNotEquals text_not_equals = 4;
+
+ TextMatches text_matches = 5;
+ TextNotMatches text_not_matches = 6;
}
}
diff --git a/include/mesos/v1/scheduler/scheduler.proto
b/include/mesos/v1/scheduler/scheduler.proto
index 08fb10a..364d2c8 100644
--- a/include/mesos/v1/scheduler/scheduler.proto
+++ b/include/mesos/v1/scheduler/scheduler.proto
@@ -323,8 +323,54 @@ message AttributeConstraint {
required string value = 1;
}
- // TODO(asekretenko): add predicates for regular expression matching
- // (MESOS-10173).
+ // Predicates for regular expression constraints.
+ //
+ // The regular expressions are interpreted according to
+ // RE2 regular expression syntax and semantics:
+ // https://github.com/google/re2/blob/master/doc/syntax.txt
+ //
+ // The call using the constraints is invalid if the specified string is not
+ // a valid RE2 regex, or the regex is deemed too expensive to
+ // construct/store/execute, namely:
+ //
+ // - the RE2 object cannot be constructed from the regex without breaching
+ // the limit on the estimated memory footprint of an individual RE2
+ // (controlled by the --offer_constraints_re2_max_mem master flag), or
+ //
+ // - the regex program size reported by `RE2::ProgramSize()`
+ // exceeds the value of --offer_constraints_re2_max_program_size flag.
+ //
+ // Frameworks using the C++ implementation of RE2 (directly or via the JNI
+ // wrapper) for regexp validation on their side, are advised to set
+ // the `max_mem` RE2 option and limit `ProgramSize()` to values no larger
+ // than the defaults for the corresonding Mesos master flags.
+ //
+ // TODO(asekretenko): Provide an API for the frameworks and/or operators
+ // to validate regexp constraints against the Mesos master setup.
+ //
+ // Similarly to the string equality predicates, both regexp predicates
match
+ // (evaluate to `true`) when the (pseudo)attribute is not a TEXT/string.
+ // This way, schedulers will have to apply purely on their own
+ // whatever filtration they do for non-TEXT attributes which happen
+ // to be selected by the constraint's `Selector`
+
+ // Yields `true` if the (pseudo)attribute exists and is a TEXT/string
+ // matching the specified regex.
+ //
+ // Always yields `true` for existing non-TEXT attributes (and non-string
+ // pseudoattributes) for the reasons explained above.
+ message TextMatches {
+ required string regex = 1;
+ }
+
+ // Yields `true` if the (pseudo)attribute either does not exists or is
+ // not a TEXT/string matching the specified regex.
+ //
+ // Always yields `true` for existing non-TEXT attributes (and non-string
+ // pseudoattributes) for the reasons explained above.
+ message TextNotMatches {
+ required string regex = 1;
+ }
oneof predicate {
Exists exists = 1;
@@ -332,6 +378,9 @@ message AttributeConstraint {
TextEquals text_equals = 3;
TextNotEquals text_not_equals = 4;
+
+ TextMatches text_matches = 5;
+ TextNotMatches text_not_matches = 6;
}
}
