jenkins-bot has submitted this change and it was merged.
Change subject: Add a native script for detecting noop updates
......................................................................
Add a native script for detecting noop updates
The builtin Elasticsearch detect_noop feature doesn't allow any slop when
detecting noops. This super_detect_noop script does. You configure it like so:
curl -XPOST localhost:9200/test/test/1/_update -d'{
"script": "super_detect_noop",
"lang": "native",
"params": {
"source": {
"foo": {
"bar": 5
},
},
"detectors": {
"foo.bar": "within 20%"
}
}
}'
Change-Id: I027ad6fa841cd33382dfbf3c1ea5eedb2cf9d169
(cherry picked from commit a319cf14144360e6f5169df925a8818497381cab)
---
M README.md
M docs/safer.md
A docs/super_detect_noop.md
M src/main/java/org/wikimedia/search/extra/ExtraPlugin.java
A
src/main/java/org/wikimedia/search/extra/superdetectnoop/CloseEnoughDetector.java
A
src/main/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScript.java
A
src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinAbsoluteDetector.java
A
src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinPercentageDetector.java
A
src/test/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScriptTest.java
9 files changed, 695 insertions(+), 5 deletions(-)
Approvals:
Manybubbles: Looks good to me, approved
jenkins-bot: Verified
diff --git a/README.md b/README.md
index ded6150..3bacd1d 100644
--- a/README.md
+++ b/README.md
@@ -1,8 +1,9 @@
Extra Queries and Filters
=========================
-The plan is for this to include any extra queries and filters we end up
-creating to make search nice for Wikimedia. At this point it only contains:
+The plan is for this to include any extra queries, filters, and native scripts
+we end up creating to make search nice for Wikimedia. At this point it only
+contains:
Filters:
* [source_regex](docs/source_regex.md) - An nGram accelerated regular
@@ -18,6 +19,10 @@
potentially expensive constructs. Expensive constructs either cause errors to
be sent back to the user or are degraded into cheaper, less precise constructs.
+Native Scripts:
+* [super_detect_noop](docs/super_detect_noop.md) - Like ```detect_noop``` but
+supports configurable sloppiness. New in 1.5.0, 1.4.1, and 1.3.1.
+
| Extra Queries and Filters Plugin | ElasticSearch |
|----------------------------------|-----------------|
| master | 1.3.4 -> 1.3.X |
diff --git a/docs/safer.md b/docs/safer.md
index 14c8ba1..4f0d864 100644
--- a/docs/safer.md
+++ b/docs/safer.md
@@ -126,7 +126,7 @@
}
public static class MySafeifierActionsModule extends AbstractModule {
- public SafeifierActionsModule(Settings settings) {
+ public MySafeifierActionsModule(Settings settings) {
}
@Override
diff --git a/docs/super_detect_noop.md b/docs/super_detect_noop.md
new file mode 100644
index 0000000..af4b7b0
--- /dev/null
+++ b/docs/super_detect_noop.md
@@ -0,0 +1,97 @@
+super_detect_noop
+=================
+
+The ```super_detect_noop``` native script is just like Elasticsearch's
+```detect_noop``` but it allows configurable sloppiness.
+
+Options
+-------
+
+```super_detect_noop``` supports only the following options:
+* ```source``` The source to merge into the existing source. Required.
+* ```detectors``` Configures sloppiness detectors. Optional, defaults to
+behaving exactly as Elasticsearch's ```detect_noop```. Possible field values:
+ * ```equals``` If the new value isn't equal to the old value then the new
+ value is written to the source. This is the default if no value is
+ specified in the ```detectors``` object.
+ * ```within nnn%``` If the new value isn't within nnn percent of the old
+ value then its written to the source. nnn is parsed as a double and all
+ math is performed with doubles.
+ * ```within nnn``` If the new value isn't within nnn of the old value then
+ its written to the source. nnn is parsed as a double and all math is
+ performed with doubles.
+
+Examples
+-------
+```bash
+curl -XDELETE localhost:9200/test?pretty
+curl -XPUT localhost:9200/test?pretty
+curl -XGET
'http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=50s&pretty'
+curl -XPUT localhost:9200/test/test/1?pretty -d'{
+ "foo": 6
+}'
+curl -XPOST localhost:9200/test/test/1/_update?pretty -d'{
+ "script": "super_detect_noop",
+ "lang": "native",
+ "params": {
+ "source": {
+ "foo": 5
+ },
+ "detectors": {
+ "foo": "within 20%"
+ }
+ }
+}'
+```
+
+```bash
+curl -XDELETE localhost:9200/test?pretty
+curl -XPUT localhost:9200/test?pretty
+curl -XGET
'http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=50s&pretty'
+curl -XPUT localhost:9200/test/test/1 -d'{
+ "foo": {
+ "bar": 6
+ }
+}'
+curl -XPOST localhost:9200/test/test/1/_update -d'{
+ "script": "super_detect_noop",
+ "lang": "native",
+ "params": {
+ "source": {
+ "foo": {
+ "bar": 5
+ },
+ },
+ "detectors": {
+ "foo.bar": "within 20%"
+ }
+ }
+}'
+```
+
+Integrating
+-----------
+This native script was designed to allow other plugins to hook into it. Doing
+so looks like this:
+```java
+public class MyPlugin extends AbstractPlugin {
+ @Override
+ public Collection<Class<? extends Module>> modules() {
+ return ImmutableList.<Class<? extends
Module>>of(MyCloseEnoughDetectorsModule.class);
+ }
+
+ public static class MyCloseEnoughDetectorsModule extends AbstractModule {
+ public MyCloseEnoughDetectorsModule(Settings settings) {
+ }
+
+ @Override
+ @SuppressWarnings("rawtypes")
+ protected void configure() {
+ Multibinder<CloseEnoughDetector.Recognizer> detectors = Multibinder
+ .newSetBinder(binder(),
CloseEnoughDetector.Recognizer.class);
+ detectors.addBinding().toInstance(new
WithinPercentageDetector.Factory());
+ }
+ }
+}
+```
+
diff --git a/src/main/java/org/wikimedia/search/extra/ExtraPlugin.java
b/src/main/java/org/wikimedia/search/extra/ExtraPlugin.java
index 232abfa..418ceca 100644
--- a/src/main/java/org/wikimedia/search/extra/ExtraPlugin.java
+++ b/src/main/java/org/wikimedia/search/extra/ExtraPlugin.java
@@ -10,12 +10,17 @@
import org.elasticsearch.index.query.QueryParser;
import org.elasticsearch.indices.query.IndicesQueriesModule;
import org.elasticsearch.plugins.AbstractPlugin;
+import org.elasticsearch.script.ScriptModule;
import org.wikimedia.search.extra.idhashmod.IdHashModFilterParser;
import org.wikimedia.search.extra.regex.SourceRegexFilterParser;
import org.wikimedia.search.extra.safer.ActionModuleParser;
import org.wikimedia.search.extra.safer.SaferQueryParser;
import
org.wikimedia.search.extra.safer.phrase.PhraseTooLargeActionModuleParser;
import org.wikimedia.search.extra.safer.simple.SimpleActionModuleParser;
+import org.wikimedia.search.extra.superdetectnoop.CloseEnoughDetector;
+import org.wikimedia.search.extra.superdetectnoop.SuperDetectNoopScript;
+import org.wikimedia.search.extra.superdetectnoop.WithinAbsoluteDetector;
+import org.wikimedia.search.extra.superdetectnoop.WithinPercentageDetector;
/**
* Setup the Elasticsearch plugin.
@@ -38,12 +43,19 @@
public void onModule(IndicesQueriesModule module) {
module.addFilter(new SourceRegexFilterParser());
module.addFilter(new IdHashModFilterParser());
- module.addQuery((Class<QueryParser>) (Class<?>)SaferQueryParser.class);
+ module.addQuery((Class<QueryParser>) (Class<?>)
SaferQueryParser.class);
+ }
+
+ /**
+ * Register our scripts.
+ */
+ public void onModule(ScriptModule module) {
+ module.registerScript("super_detect_noop",
SuperDetectNoopScript.Factory.class);
}
@Override
public Collection<Class<? extends Module>> modules() {
- return ImmutableList.<Class<? extends
Module>>of(SafeifierActionsModule.class);
+ return ImmutableList.<Class<? extends Module>>
of(SafeifierActionsModule.class, CloseEnoughDetectorsModule.class);
}
public static class SafeifierActionsModule extends AbstractModule {
@@ -58,4 +70,18 @@
moduleParsers.addBinding().to(SimpleActionModuleParser.class).asEagerSingleton();
}
}
+
+ public static class CloseEnoughDetectorsModule extends AbstractModule {
+ public CloseEnoughDetectorsModule(Settings settings) {
+ }
+
+ @Override
+ protected void configure() {
+ Multibinder<CloseEnoughDetector.Recognizer> detectors = Multibinder
+ .newSetBinder(binder(),
CloseEnoughDetector.Recognizer.class);
+ detectors.addBinding().toInstance(new
CloseEnoughDetector.Equal.Factory());
+ detectors.addBinding().toInstance(new
WithinPercentageDetector.Factory());
+ detectors.addBinding().toInstance(new
WithinAbsoluteDetector.Factory());
+ }
+ }
}
diff --git
a/src/main/java/org/wikimedia/search/extra/superdetectnoop/CloseEnoughDetector.java
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/CloseEnoughDetector.java
new file mode 100644
index 0000000..a27f873
--- /dev/null
+++
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/CloseEnoughDetector.java
@@ -0,0 +1,115 @@
+package org.wikimedia.search.extra.superdetectnoop;
+
+/**
+ * Detects if two values are different enough to be changed.
+ *
+ * @param <T> type of the thin being checked
+ */
+public interface CloseEnoughDetector<T> {
+ /**
+ * Two objects are never close enough.
+ */
+ static final CloseEnoughDetector<Object> EQUALS = new
NullSafe<>(Equal.INSTANCE);
+
+ boolean isCloseEnough(T oldValue, T newValue);
+
+ /**
+ * Builds CloseEnoughDetectors from the string description sent in the
+ * script parameters. Returning null from build just means that this
recognizer
+ * doesn't recognize that parameter.
+ */
+ static interface Recognizer {
+ CloseEnoughDetector<Object> build(String description);
+ }
+
+ /**
+ * Wraps another detector and only delegates to it if both values aren't
+ * null. If both values are null returns true, if only one is null then
+ * returns false.
+ *
+ * @param <T> type on which the wrapped detector operates
+ */
+ class NullSafe<T> implements CloseEnoughDetector<T> {
+ private final CloseEnoughDetector<T> delegate;
+
+ public NullSafe(CloseEnoughDetector<T> delegate) {
+ this.delegate = delegate;
+ }
+
+ @Override
+ public boolean isCloseEnough(T oldValue, T newValue) {
+ if (oldValue == null) {
+ return newValue == null;
+ }
+ if (newValue == null) {
+ return false;
+ }
+ return delegate.isCloseEnough(oldValue, newValue);
+ }
+ }
+
+ /**
+ * Objects are only close enough if they are {@link Object#equals(Object)}
+ * to each other. Doesn't do any null checking - wrap in NullSafe or just
+ * use CloseEnoughDetector.EQUALS if you need it.
+ */
+ class Equal implements CloseEnoughDetector<Object> {
+ public static CloseEnoughDetector<Object> INSTANCE = new Equal();
+
+ public static class Factory implements CloseEnoughDetector.Recognizer {
+ @Override
+ public CloseEnoughDetector<Object> build(String description) {
+ if (description.equals("equals")) {
+ return INSTANCE;
+ }
+ return null;
+ }
+ }
+
+ private Equal() {
+ }
+
+ @Override
+ public boolean isCloseEnough(Object oldValue, Object newValue) {
+ return oldValue.equals(newValue);
+ }
+ }
+
+ /**
+ * Wraps another detector and only delegates to it if both values are of a
+ * certain type. If they aren't then it delegates to Equal. Doesn't perform
+ * null checking - wrap me in NullSafe or just use the nullAndTypeSafe
+ * static method to build me.
+ *
+ * @param <T> type on which the wrapped detector operates
+ */
+ class TypeSafe<T> implements CloseEnoughDetector<Object> {
+ /**
+ * Wraps a CloseEnoughDetector in a null-safe, type-safe way.
+ */
+ static <T> CloseEnoughDetector<Object> nullAndTypeSafe(Class<T> type,
CloseEnoughDetector<T> delegate) {
+ return new CloseEnoughDetector.NullSafe<>(new
CloseEnoughDetector.TypeSafe<>(type, delegate));
+ }
+
+ private final Class<T> type;
+ private final CloseEnoughDetector<T> delegate;
+
+ public TypeSafe(Class<T> type, CloseEnoughDetector<T> delegate) {
+ this.type = type;
+ this.delegate = delegate;
+ }
+
+ @Override
+ public boolean isCloseEnough(Object oldValue, Object newValue) {
+ T oldValueCast;
+ T newValueCast;
+ try {
+ oldValueCast = type.cast(oldValue);
+ newValueCast = type.cast(newValue);
+ } catch (ClassCastException e) {
+ return Equal.INSTANCE.isCloseEnough(oldValue, newValue);
+ }
+ return delegate.isCloseEnough(oldValueCast, newValueCast);
+ }
+ }
+}
diff --git
a/src/main/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScript.java
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScript.java
new file mode 100644
index 0000000..2735982
--- /dev/null
+++
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScript.java
@@ -0,0 +1,137 @@
+package org.wikimedia.search.extra.superdetectnoop;
+
+import static org.elasticsearch.common.base.Preconditions.checkNotNull;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.elasticsearch.common.collect.ImmutableList;
+import org.elasticsearch.common.collect.ImmutableMap;
+import org.elasticsearch.common.inject.Inject;
+import org.elasticsearch.script.AbstractExecutableScript;
+import org.elasticsearch.script.ExecutableScript;
+import org.elasticsearch.script.NativeScriptFactory;
+
+/**
+ * Like the detect_noop option on updates but with pluggable "close enough"
+ * detectors! So much power!
+ */
+public class SuperDetectNoopScript extends AbstractExecutableScript {
+ public static class Factory implements NativeScriptFactory {
+ private final List<CloseEnoughDetector.Recognizer>
closeEnoughFactories;
+
+ @Inject
+ public Factory(Set<CloseEnoughDetector.Recognizer> factories) {
+ // Note that detectors are tried in a random order....
+ this.closeEnoughFactories = ImmutableList.copyOf(factories);
+ }
+
+ @Override
+ public ExecutableScript newScript(Map<String, Object> params) {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> source = (Map<String, Object>)
params.get("source");
+ return new SuperDetectNoopScript(source, detectors(params));
+ }
+
+ private Map<String, CloseEnoughDetector<Object>> detectors(Map<String,
Object> params) {
+ @SuppressWarnings("unchecked")
+ Map<String, String> detectorConfigs = (Map<String, String>)
params.get("detectors");
+ if (detectorConfigs == null) {
+ return Collections.emptyMap();
+ }
+ ImmutableMap.Builder<String, CloseEnoughDetector<Object>>
detectors = ImmutableMap.builder();
+ for (Map.Entry<String, String> detectorConfig :
detectorConfigs.entrySet()) {
+ detectors.put(detectorConfig.getKey(),
detector(detectorConfig.getValue()));
+ }
+ return detectors.build();
+ }
+
+ private CloseEnoughDetector<Object> detector(String config) {
+ for (CloseEnoughDetector.Recognizer factory :
closeEnoughFactories) {
+ CloseEnoughDetector<Object> detector = factory.build(config);
+ if (detector != null) {
+ return detector;
+ }
+ }
+ throw new IllegalArgumentException("Don't recognize this type of
detector: " + config);
+ }
+ }
+
+ private final Map<String, Object> source;
+ private final Map<String, CloseEnoughDetector<Object>> pathToDetector;
+ private Map<String, Object> ctx;
+
+ public SuperDetectNoopScript(Map<String, Object> source, Map<String,
CloseEnoughDetector<Object>> pathToDetector) {
+ this.source = checkNotNull(source, "source must be specified");
+ this.pathToDetector = checkNotNull(pathToDetector, "detectors must be
specified");
+ }
+
+ @Override
+ public Object run() {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> oldSource = (Map<String, Object>)
ctx.get("_source");
+ boolean changed = update(oldSource, source, "");
+ if (!changed) {
+ ctx.put("op", "none");
+ }
+ // The return value is ignored
+ return null;
+ }
+
+ @Override
+ @SuppressWarnings("unchecked")
+ public void setNextVar(String name, Object value) {
+ if (name.equals("ctx")) {
+ ctx = (Map<String, Object>) value;
+ }
+ }
+
+ /**
+ * Update old with the source and detector configuration of this script.
+ */
+ boolean update(Map<String, Object> old, Map<String, Object> updateSource,
String path) {
+ boolean modified = false;
+ for (Map.Entry<String, Object> sourceEntry : updateSource.entrySet()) {
+ String nextPath = path + sourceEntry.getKey();
+ Object oldValue = old.get(sourceEntry.getKey());
+ if (oldValue instanceof Map && sourceEntry.getValue() instanceof
Map) {
+ // recursive merge maps
+ @SuppressWarnings("unchecked")
+ Map<String, Object> nextOld = (Map<String, Object>)
old.get(sourceEntry.getKey());
+ @SuppressWarnings("unchecked")
+ Map<String, Object> nextUpdateSource = (Map<String, Object>)
sourceEntry.getValue();
+ modified |= update(nextOld, nextUpdateSource, nextPath);
+ continue;
+ }
+ if (detector(nextPath).isCloseEnough(oldValue,
sourceEntry.getValue())) {
+ continue;
+ }
+ if (sourceEntry.getValue() == null) {
+ old.remove(sourceEntry.getKey());
+ } else {
+ old.put(sourceEntry.getKey(), sourceEntry.getValue());
+ }
+ modified = true;
+ continue;
+ }
+ /*
+ * Right now if a field isn't in the source passed to the script the
+ * close enough detectors never get a chance to look at it - the field
+ * is never changed.
+ */
+ return modified;
+ }
+
+ /**
+ * Get the close enough detector for a path, defaulting to EQUALS.
+ */
+ private CloseEnoughDetector<Object> detector(String path) {
+ CloseEnoughDetector<Object> d = pathToDetector.get(path);
+ if (d == null) {
+ return CloseEnoughDetector.EQUALS;
+ }
+ return d;
+ }
+}
diff --git
a/src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinAbsoluteDetector.java
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinAbsoluteDetector.java
new file mode 100644
index 0000000..887ee0b
--- /dev/null
+++
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinAbsoluteDetector.java
@@ -0,0 +1,36 @@
+package org.wikimedia.search.extra.superdetectnoop;
+
+import static
org.wikimedia.search.extra.superdetectnoop.CloseEnoughDetector.TypeSafe.nullAndTypeSafe;
+
+/**
+ * Checks if a number is different by some absolute amount.
+ */
+public class WithinAbsoluteDetector implements CloseEnoughDetector<Number> {
+ public static class Factory implements CloseEnoughDetector.Recognizer {
+ private static final String PREFIX = "within ";
+
+ @Override
+ public CloseEnoughDetector<Object> build(String description) {
+ if (!description.startsWith(PREFIX)) {
+ return null;
+ }
+ try {
+ double absoluteDifference =
Double.parseDouble(description.substring(PREFIX.length(),
description.length()));
+ return nullAndTypeSafe(Number.class, new
WithinAbsoluteDetector(absoluteDifference));
+ } catch (NumberFormatException e) {
+ return null;
+ }
+ }
+ }
+
+ private final double absoluteDifference;
+
+ public WithinAbsoluteDetector(double absoluteDifference) {
+ this.absoluteDifference = absoluteDifference;
+ }
+
+ @Override
+ public boolean isCloseEnough(Number oldValue, Number newValue) {
+ return Math.abs(newValue.doubleValue() - oldValue.doubleValue()) <
absoluteDifference;
+ }
+}
diff --git
a/src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinPercentageDetector.java
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinPercentageDetector.java
new file mode 100644
index 0000000..8729d46
--- /dev/null
+++
b/src/main/java/org/wikimedia/search/extra/superdetectnoop/WithinPercentageDetector.java
@@ -0,0 +1,42 @@
+package org.wikimedia.search.extra.superdetectnoop;
+
+import static java.lang.Math.abs;
+import static
org.wikimedia.search.extra.superdetectnoop.CloseEnoughDetector.TypeSafe.nullAndTypeSafe;
+
+/**
+ * Checks if a number is different by some percentage.
+ */
+public class WithinPercentageDetector implements CloseEnoughDetector<Number> {
+ public static class Factory implements CloseEnoughDetector.Recognizer {
+ private static final String PREFIX = "within ";
+ private static final String SUFFIX = "%";
+
+ @Override
+ public CloseEnoughDetector<Object> build(String description) {
+ if (!description.startsWith(PREFIX)) {
+ return null;
+ }
+ if (!description.endsWith(SUFFIX)) {
+ return null;
+ }
+ try {
+ double percentage =
Double.parseDouble(description.substring(PREFIX.length(), description.length()
- SUFFIX.length()));
+ return nullAndTypeSafe(Number.class, new
WithinPercentageDetector(percentage / 100));
+ } catch (NumberFormatException e) {
+ // Not a valid number even with the % sign....
+ return null;
+ }
+ }
+ }
+
+ private final double absoluteDifference;
+
+ public WithinPercentageDetector(double absoluteDifference) {
+ this.absoluteDifference = absoluteDifference;
+ }
+
+ @Override
+ public boolean isCloseEnough(Number oldValue, Number newValue) {
+ return abs((newValue.doubleValue() - oldValue.doubleValue()) /
oldValue.doubleValue()) < absoluteDifference;
+ }
+}
diff --git
a/src/test/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScriptTest.java
b/src/test/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScriptTest.java
new file mode 100644
index 0000000..688b7ad
--- /dev/null
+++
b/src/test/java/org/wikimedia/search/extra/superdetectnoop/SuperDetectNoopScriptTest.java
@@ -0,0 +1,232 @@
+package org.wikimedia.search.extra.superdetectnoop;
+
+import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
+import static
org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertThrows;
+import static org.hamcrest.Matchers.anything;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.greaterThan;
+import static org.hamcrest.Matchers.hasEntry;
+import static org.hamcrest.Matchers.not;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.elasticsearch.action.index.IndexResponse;
+import org.elasticsearch.action.update.UpdateRequestBuilder;
+import org.elasticsearch.common.xcontent.XContentBuilder;
+import org.elasticsearch.common.xcontent.XContentHelper;
+import org.elasticsearch.rest.RestStatus;
+import org.elasticsearch.script.ScriptService.ScriptType;
+import org.junit.Test;
+import org.wikimedia.search.extra.AbstractPluginIntegrationTest;
+
+public class SuperDetectNoopScriptTest extends AbstractPluginIntegrationTest {
+ @Test
+ public void newField() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("bar", 2);
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) 3));
+ assertThat(r, hasEntry("bar", (Object) 2));
+ }
+
+ @Test
+ public void notModified() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", 3);
+ Map<String, Object> r = update(b, false);
+ assertThat(r, hasEntry("int", (Object) 3));
+ }
+
+ @Test
+ public void setToNull() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", null);
+ Map<String, Object> r = update(b, true);
+ assertThat(r, not(hasEntry(equalTo("int"), anything())));
+ }
+
+ @Test
+ public void newValue() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", 2);
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) 2));
+ }
+
+ @Test
+ public void withinPercentage() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", 5, "within 200%");
+ Map<String, Object> r = update(b, false);
+ assertThat(r, hasEntry("int", (Object) 3));
+ }
+
+ @Test
+ public void withinPercentageNegative() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", -1, "within 200%");
+ Map<String, Object> r = update(b, false);
+ assertThat(r, hasEntry("int", (Object) 3));
+ }
+
+ @Test
+ public void outsidePercentage() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", 9, "within 200%");
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) 9));
+ }
+
+ @Test
+ public void outsidePercentageNegative() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", -3, "within 200%");
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) (-3)));
+ }
+
+ @Test
+ public void percentageOnString() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("string", "cat", "within 200%");
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) 3));
+ assertThat(r, hasEntry("string", (Object) "cat"));
+ }
+
+ @Test
+ public void withinAbsolute() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", 4, "within 2");
+ Map<String, Object> r = update(b, false);
+ assertThat(r, hasEntry("int", (Object) 3));
+ }
+
+ @Test
+ public void withinAbsoluteNegative() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", -1, "within 7");
+ Map<String, Object> r = update(b, false);
+ assertThat(r, hasEntry("int", (Object) 3));
+ }
+
+ @Test
+ public void outsideAbsolute() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", 5, "within 2");
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) 5));
+ }
+
+ @Test
+ public void outsideAbsoluteNegative() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", -4, "within 7");
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) (-4)));
+ }
+
+ @Test
+ public void absoluteOnString() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("string", "cat", "within 2");
+ Map<String, Object> r = update(b, true);
+ assertThat(r, hasEntry("int", (Object) 3));
+ assertThat(r, hasEntry("string", (Object) "cat"));
+ }
+
+ @Test
+ public void garbageDetector() throws IOException {
+ indexSeedData();
+ XContentBuilder b = x("int", "cat", "not a valid detector");
+ assertThrows(toUpdateRequest(b), RestStatus.BAD_REQUEST);
+ }
+
+ /**
+ * Tests path matching.
+ */
+ @Test
+ public void path() throws IOException {
+ indexSeedData();
+ XContentBuilder b = jsonBuilder().startObject();
+ b.startObject("source");
+ {
+ b.startObject("foo");
+ {
+ b.field("bar", 10);
+ }
+ b.endObject();
+ }
+ b.endObject();
+ b.startObject("detectors");
+ {
+ b.field("foo.bar", "within 10");
+ }
+ b.endObject();
+ b.endObject();
+ Map<String, Object> r = update(b, false);
+ assertThat(r, hasEntry("int", (Object) 3));
+ }
+
+ private XContentBuilder x(String field, Object value) throws IOException {
+ return x(field, value, null);
+ }
+
+ /**
+ * Builds the xcontent for the request parameters for a single field.
+ */
+ private XContentBuilder x(String field, Object value, String detector)
throws IOException {
+ XContentBuilder b = jsonBuilder().startObject();
+ b.startObject("source");
+ {
+ b.field(field, value);
+ }
+ b.endObject();
+ if (detector != null) {
+ b.startObject("detectors");
+ {
+ b.field(field, detector);
+ }
+ b.endObject();
+ }
+ b.endObject();
+ return b;
+ }
+
+ private void indexSeedData() throws IOException {
+ XContentBuilder b = jsonBuilder().startObject();
+ {
+ b.field("int", 3);
+ b.field("string", "cake");
+ b.startObject("foo");
+ {
+ b.field("bar", 10);
+ }
+ b.endObject();
+ }
+ b.endObject();
+ IndexResponse ir = client().prepareIndex("test", "test",
"1").setSource(b).setRefresh(true).get();
+ assertTrue("Test data is newly created", ir.isCreated());
+ }
+
+ private Map<String, Object> update(XContentBuilder b, boolean
shouldUpdate) {
+ long beforeNoops =
client().admin().indices().prepareStats("test").get().getTotal().indexing.getTotal().getIndexCount();
+ toUpdateRequest(b).get();
+ long afterNoops =
client().admin().indices().prepareStats("test").get().getTotal().indexing.getTotal().getIndexCount();
+ if (shouldUpdate) {
+ assertThat("there have not been updates but we expected some",
afterNoops, greaterThan(beforeNoops));
+ } else {
+ assertEquals("there have been updates and we don't expect them",
beforeNoops, afterNoops);
+ }
+ return client().prepareGet("test", "test", "1").get().getSource();
+ }
+
+ private UpdateRequestBuilder toUpdateRequest(XContentBuilder b) {
+ b.close();
+ Map<String, Object> m = XContentHelper.convertToMap(b.bytes(),
true).v2();
+ return client().prepareUpdate("test", "test",
"1").setScript("super_detect_noop", ScriptType.INLINE).setScriptLang("native")
+ .setScriptParams(m).setRefresh(true);
+
+ }
+}
--
To view, visit https://gerrit.wikimedia.org/r/207278
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I027ad6fa841cd33382dfbf3c1ea5eedb2cf9d169
Gerrit-PatchSet: 2
Gerrit-Project: search/extra
Gerrit-Branch: 1.3
Gerrit-Owner: Manybubbles <never...@wikimedia.org>
Gerrit-Reviewer: Chad <ch...@wikimedia.org>
Gerrit-Reviewer: Jdouglas <jdoug...@wikimedia.org>
Gerrit-Reviewer: Manybubbles <never...@wikimedia.org>
Gerrit-Reviewer: Smalyshev <smalys...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
