Can you please check the branch in my fork ?
fork: https://github.com/asereda-gs/calcite
branch: javadoc-fixes
I've tried to remove most of the warnings and javadoc, javadoc-test and
site all work for me.
~/I/c/elasticsearch5 ❯❯❯ mvn --version
javadoc-fixes ✭
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe;
2018-06-17T14:33:14-04:00)
Maven home: /usr/local/Cellar/maven/3.5.4/libexec
Java version: 1.8.0_151, vendor: Oracle Corporation, runtime:
/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.2", arch: "x86_64", family: "mac"
On Sun, Jun 24, 2018 at 12:40 PM Volodymyr Vysotskyi <volody...@apache.org>
wrote:
> I have also checked "mvn javadoc:javadoc javadoc:test-javadoc" on my
> machine (Ubuntu, JDK 8, Maven 3.3.9) and it fails on elasticsearch2 module
> (didn't check this problem for the next modules).
>
> "mvn clean -DskipTests site" fails for MongoDB module.
>
> Kind regards,
> Volodymyr Vysotskyi
>
>
> сб, 23 черв. 2018 о 03:12 Julian Hyde <jh...@apache.org> пише:
>
> > Same failure for me on JDK 8/macOS. The most likely other variable is
> > maven: I have 3.5.2 on macOS, 3.5.3 on ubuntu.
> >
> > I’m mystified why no one else is seeing javadoc failures on macOS.
> >
> > Julian
> >
> >
> > > On Jun 22, 2018, at 4:35 PM, Julian Hyde <jh...@apache.org> wrote:
> > >
> > > On JDK 8/ubuntu, “mvn clean -DskipTests site” gives the following
> > failure:
> > >
> > > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-site-plugin:3.7:site (default-site) on
> > project calcite-mongodb: Error generating
> > maven-javadoc-plugin:3.0.1:test-javadoc report:
> > > [ERROR] Exit code: 1 -
> >
> /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java:22:
> > error: cannot find symbol
> > > [ERROR] import org.apache.calcite.test.CalciteAssert;
> > > [ERROR] ^
> > > [ERROR] symbol: class CalciteAssert
> > > [ERROR] location: package org.apache.calcite.test
> > > [ERROR]
> >
> /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java:137:
> > error: package CalciteAssert does not exist
> > > [ERROR] private CalciteAssert.AssertThat assertModel(String model) {
> > > [ERROR] ^
> > > [ERROR]
> >
> /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java:145:
> > error: package CalciteAssert does not exist
> > > [ERROR] private CalciteAssert.AssertThat assertModel(URL url) {
> > > [ERROR] ^
> > > [ERROR]
> >
> /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/test/MongoAssertions.java:43:
> > error: reference not found
> > > [ERROR] /** Similar to {@link CalciteAssert#checkResultUnordered},
> but
> > filters strings
> > > [ERROR] ^
> > >
> > >
> > >
> > >> On Jun 22, 2018, at 3:46 PM, Andrei Sereda <and...@sereda.cc> wrote:
> > >>
> > >> Hmm. Both master and c12cb4b0de work for me. Perhaps the issue is JDK
> 8
> > vs
> > >> 9 ?
> > >>
> > >> # JDK8 (oracle) / macOS
> > >> $ mvn clean -DskipTests site
> > >>
> > >> On Fri, Jun 22, 2018 at 5:56 PM Julian Hyde <jh...@apache.org> wrote:
> > >>
> > >>> I just tried it again. The following fails for me (jdk9, ubuntu):
> > >>>
> > >>> $ git checkout c12cb4b0de
> > >>> $ mvn clean -DskipTests site
> > >>>
> > >>> Constructing Javadoc information...
> > >>> 1 error
> > >>> [INFO]
> > >>>
> > ------------------------------------------------------------------------
> > >>> [INFO] Reactor Summary:
> > >>> [INFO]
> > >>> [INFO] Calcite 1.17.0-SNAPSHOT ............................ FAILURE
> > [02:29
> > >>> min]
> > >>> [INFO] Calcite Linq4j ..................................... SKIPPED
> > >>> [INFO] Calcite Core ....................................... SKIPPED
> > >>> [INFO] Calcite Cassandra .................................. SKIPPED
> > >>> [INFO] Calcite Druid ...................................... SKIPPED
> > >>> [INFO] Calcite Elasticsearch .............................. SKIPPED
> > >>> [INFO] Calcite Elasticsearch5 ............................. SKIPPED
> > >>> [INFO] Calcite Examples ................................... SKIPPED
> > >>> [INFO] Calcite Example CSV ................................ SKIPPED
> > >>> [INFO] Calcite Example Function ........................... SKIPPED
> > >>> [INFO] Calcite File ....................................... SKIPPED
> > >>> [INFO] Calcite Geode ...................................... SKIPPED
> > >>> [INFO] Calcite MongoDB .................................... SKIPPED
> > >>> [INFO] Calcite Pig ........................................ SKIPPED
> > >>> [INFO] Calcite Piglet ..................................... SKIPPED
> > >>> [INFO] Calcite Plus ....................................... SKIPPED
> > >>> [INFO] Calcite Server ..................................... SKIPPED
> > >>> [INFO] Calcite Spark ...................................... SKIPPED
> > >>> [INFO] Calcite Splunk ..................................... SKIPPED
> > >>> [INFO] Calcite Ubenchmark 1.17.0-SNAPSHOT ................. SKIPPED
> > >>> [INFO]
> > >>>
> > ------------------------------------------------------------------------
> > >>> [INFO] BUILD FAILURE
> > >>> [INFO]
> > >>>
> > ------------------------------------------------------------------------
> > >>> [INFO] Total time: 02:29 min
> > >>> [INFO] Finished at: 2018-06-22T14:53:13-07:00
> > >>> [INFO]
> > >>>
> > ------------------------------------------------------------------------
> > >>> [ERROR] Failed to execute goal
> > >>> org.apache.maven.plugins:maven-site-plugin:3.7:site (default-site) on
> > >>> project calcite: Error generating
> > maven-javadoc-plugin:3.0.1:test-aggregate
> > >>> report:
> > >>> [ERROR] Exit code: 1 -
> > >>>
> >
> /home/jhyde/open1/calcite.4/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java:29:
> > >>> error: package org.elasticsearch.node.internal does not exist
> > >>> [ERROR] import
> > org.elasticsearch.node.internal.InternalSettingsPreparer;
> > >>> [ERROR] ^
> > >>> [ERROR]
> > >>> [ERROR] Command line was: /usr/lib/jvm/jdk9/bin/javadoc @options
> > @packages
> > >>> @argfile
> > >>> [ERROR]
> > >>> [ERROR] Refer to the generated Javadoc files in
> > >>> '/home/jhyde/open1/calcite.4/target/site/testapidocs' dir.
> > >>> [ERROR] -> [Help 1]
> > >>> [ERROR]
> > >>> [ERROR] To see the full stack trace of the errors, re-run Maven with
> > the
> > >>> -e switch.
> > >>> [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> > >>> [ERROR]
> > >>> [ERROR] For more information about the errors and possible solutions,
> > >>> please read the following articles:
> > >>> [ERROR] [Help 1]
> > >>>
> > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> > >>>
> > >>>
> > >>>> On Jun 22, 2018, at 2:41 PM, Michael Mior <mm...@apache.org> wrote:
> > >>>>
> > >>>> Odd, mvn clean site still works fine for me.
> > >>>> --
> > >>>> Michael Mior
> > >>>> mm...@apache.org
> > >>>>
> > >>>>
> > >>>>
> > >>>> Le ven. 22 juin 2018 à 17:12, Julian Hyde <jh...@apache.org> a
> écrit
> > :
> > >>>>
> > >>>>> Looks like this change broke “mvn site” (perhaps also “mvn
> > >>>>> javadoc:test-javadoc”).
> > >>>>>
> > >>>>> [ERROR] Failed to execute goal
> > >>>>> org.apache.maven.plugins:maven-site-plugin:3.7:site (default-site)
> on
> > >>>>> project calcite: Error generating
> > >>> maven-javadoc-plugin:3.0.1:test-aggregate
> > >>>>> report:
> > >>>>> [ERROR] Exit code: 1 -
> > >>>>>
> > >>>
> >
> /home/jhyde/regress/calcite/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java:29:
> > >>>>> error: package org.elasticsearch.node.internal does not exist
> > >>>>> [ERROR] import
> > org.elasticsearch.node.internal.InternalSettingsPreparer;
> > >>>>> [ERROR] ^
> > >>>>> [ERROR]
> > >>>>>
> > >>>>>> On Jun 21, 2018, at 3:39 AM, mm...@apache.org wrote:
> > >>>>>>
> > >>>>>> [CALCITE-2347] running ElasticSearch in embedded mode for unit
> > tests of
> > >>>>> ES adapter (Andrei Sereda)
> > >>>>>>
> > >>>>>> After discussion on dev-list Integration tests (for ES) have been
> > >>>>> removed. They're now
> > >>>>>> superseded by unit tests (which execute queries against a real
> > elastic
> > >>>>> instance)
> > >>>>>>
> > >>>>>> Added local file (zips-mini.json) which contains a small subset of
> > >>>>> original zips.json
> > >>>>>> (allows to bootstrap tests faster)
> > >>>>>>
> > >>>>>> Created separate ES JUnit rule which can be re-used across
> different
> > >>>>> tests.
> > >>>>>>
> > >>>>>> Both v2 and v5 of ES adapters are supported.
> > >>>>>>
> > >>>>>> Close apache/calcite#716
> > >>>>>>
> > >>>>>>
> > >>>>>> Project: http://git-wip-us.apache.org/repos/asf/calcite/repo
> > >>>>>> Commit:
> > http://git-wip-us.apache.org/repos/asf/calcite/commit/c12cb4b0
> > >>>>>> Tree:
> http://git-wip-us.apache.org/repos/asf/calcite/tree/c12cb4b0
> > >>>>>> Diff:
> http://git-wip-us.apache.org/repos/asf/calcite/diff/c12cb4b0
> > >>>>>>
> > >>>>>> Branch: refs/heads/master
> > >>>>>> Commit: c12cb4b0de1baa3f7cbb9952ee350fdd1701662d
> > >>>>>> Parents: 37944bb
> > >>>>>> Author: Andrei Sereda <and...@nospam.com>
> > >>>>>> Authored: Thu May 31 18:19:10 2018 -0400
> > >>>>>> Committer: Michael Mior <mm...@uwaterloo.ca>
> > >>>>>> Committed: Thu Jun 21 06:38:50 2018 -0400
> > >>>>>>
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> .../AbstractElasticsearchTable.java | 12 +
> > >>>>>> .../elasticsearch/ElasticsearchProject.java | 61 ++-
> > >>>>>> elasticsearch2/pom.xml | 6 +
> > >>>>>> .../Elasticsearch2Enumerator.java | 12 +-
> > >>>>>> .../elasticsearch2/Elasticsearch2Schema.java | 16 +-
> > >>>>>> .../elasticsearch2/Elasticsearch2Table.java | 9 +-
> > >>>>>> .../ElasticSearch2AdapterTest.java | 395
> > >>> ++++++++++++++++++
> > >>>>>> .../elasticsearch2/EmbeddedElasticNode.java | 147 +++++++
> > >>>>>> .../elasticsearch2/EmbeddedElasticRule.java | 97 +++++
> > >>>>>> .../org/apache/calcite/test/ElasticChecker.java | 49 +++
> > >>>>>> .../calcite/test/Elasticsearch2AdapterIT.java | 270
> -------------
> > >>>>>> .../resources/elasticsearch-zips-model.json | 50 ---
> > >>>>>> .../src/test/resources/zips-mini.json | 149 +++++++
> > >>>>>> elasticsearch5/pom.xml | 31 ++
> > >>>>>> .../elasticsearch5/Elasticsearch5Schema.java | 17 +-
> > >>>>>> .../elasticsearch5/Elasticsearch5Table.java | 11 +-
> > >>>>>> .../ElasticSearch5AdapterTest.java | 399
> > >>> +++++++++++++++++++
> > >>>>>> .../elasticsearch5/EmbeddedElasticNode.java | 153 +++++++
> > >>>>>> .../elasticsearch5/EmbeddedElasticRule.java | 98 +++++
> > >>>>>> .../org/apache/calcite/test/ElasticChecker.java | 49 +++
> > >>>>>> .../calcite/test/Elasticsearch5AdapterIT.java | 270
> -------------
> > >>>>>> .../resources/elasticsearch-zips-model.json | 50 ---
> > >>>>>> elasticsearch5/src/test/resources/log4j2.xml | 16 +
> > >>>>>> .../src/test/resources/zips-mini.json | 149 +++++++
> > >>>>>> pom.xml | 20 +-
> > >>>>>> 25 files changed, 1866 insertions(+), 670 deletions(-)
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git
> > >>>>>
> > >>>
> >
> a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java
> > >>>>>
> > >>>
> >
> b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java
> > >>>>>> index 0980469..8cc5933 100644
> > >>>>>> ---
> > >>>>>
> > >>>
> >
> a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java
> > >>>>>> +++
> > >>>>>
> > >>>
> >
> b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java
> > >>>>>> @@ -75,6 +75,18 @@ public abstract class
> AbstractElasticsearchTable
> > >>>>> extends AbstractQueryableTable
> > >>>>>> relOptTable, this, null);
> > >>>>>> }
> > >>>>>>
> > >>>>>> + /**
> > >>>>>> + * In ES 5.x scripted fields start with {@code
> > params._source.foo}
> > >>>>> while in ES2.x
> > >>>>>> + * {@code _source.foo}. Helper method to build correct query
> > based
> > >>> on
> > >>>>> runtime version of elastic.
> > >>>>>> + *
> > >>>>>> + * @see <a href="
> > >>>>> https://github.com/elastic/elasticsearch/issues/20068";>_source
> > >>>>> variable</a>
> > >>>>>> + * @see <a href="
> > >>>>>
> > >>>
> >
> https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-fields.html
> > >>> ">Scripted
> > >>>>> Fields</a>
> > >>>>>> + */
> > >>>>>> + protected String scriptedFieldPrefix() {
> > >>>>>> + // this is default pattern starting 5.x
> > >>>>>> + return "params._source";
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> /** Executes a "find" operation on the underlying type.
> > >>>>>> *
> > >>>>>> * <p>For example,
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git
> > >>>>>
> > >>>
> >
> a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java
> > >>>>>
> > >>>
> >
> b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java
> > >>>>>> index b42abd7..961c8b0 100644
> > >>>>>> ---
> > >>>>>
> > >>>
> >
> a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java
> > >>>>>> +++
> > >>>>>
> > >>>
> >
> b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java
> > >>>>>> @@ -27,11 +27,15 @@ import
> > >>>>> org.apache.calcite.rel.metadata.RelMetadataQuery;
> > >>>>>> import org.apache.calcite.rel.type.RelDataType;
> > >>>>>> import org.apache.calcite.rex.RexNode;
> > >>>>>> import org.apache.calcite.util.Pair;
> > >>>>>> -import org.apache.calcite.util.Util;
> > >>>>>> +
> > >>>>>> +import com.google.common.base.Function;
> > >>>>>> +import com.google.common.collect.Lists;
> > >>>>>>
> > >>>>>> import java.util.ArrayList;
> > >>>>>> import java.util.List;
> > >>>>>>
> > >>>>>> +import javax.annotation.Nullable;
> > >>>>>> +
> > >>>>>> /**
> > >>>>>> * Implementation of {@link org.apache.calcite.rel.core.Project}
> > >>>>>> * relational expression in Elasticsearch.
> > >>>>>> @@ -57,41 +61,60 @@ public class ElasticsearchProject extends
> > Project
> > >>>>> implements ElasticsearchRel {
> > >>>>>> implementor.visitChild(0, getInput());
> > >>>>>>
> > >>>>>> final List<String> inFields =
> > >>>>>> -
> > >>>>>
> ElasticsearchRules.elasticsearchFieldNames(getInput().getRowType());
> > >>>>>> +
> > >>>>>
> ElasticsearchRules.elasticsearchFieldNames(getInput().getRowType());
> > >>>>>> final ElasticsearchRules.RexToElasticsearchTranslator translator
> =
> > >>>>>> - new ElasticsearchRules.RexToElasticsearchTranslator(
> > >>>>>> - (JavaTypeFactory) getCluster().getTypeFactory(),
> > >>> inFields);
> > >>>>>> + new ElasticsearchRules.RexToElasticsearchTranslator(
> > >>>>>> + (JavaTypeFactory)
> > getCluster().getTypeFactory(),
> > >>>>> inFields);
> > >>>>>>
> > >>>>>> - final List<String> findItems = new ArrayList<>();
> > >>>>>> - final List<String> scriptFieldItems = new ArrayList<>();
> > >>>>>> + final List<String> fields = new ArrayList<>();
> > >>>>>> + final List<String> scriptFields = new ArrayList<>();
> > >>>>>> for (Pair<RexNode, String> pair: getNamedProjects()) {
> > >>>>>> final String name = pair.right;
> > >>>>>> final String expr = pair.left.accept(translator);
> > >>>>>>
> > >>>>>> if (expr.equals("\"" + name + "\"")) {
> > >>>>>> - findItems.add(ElasticsearchRules.quote(name));
> > >>>>>> + fields.add(name);
> > >>>>>> } else if (expr.matches("\"literal\":.+")) {
> > >>>>>> - scriptFieldItems.add(ElasticsearchRules.quote(name)
> > >>>>>> - + ":{\"script\": "
> > >>>>>> - + expr.split(":")[1] + "}");
> > >>>>>> + scriptFields.add(ElasticsearchRules.quote(name)
> > >>>>>> + + ":{\"script\": "
> > >>>>>> + + expr.split(":")[1] + "}");
> > >>>>>> } else {
> > >>>>>> - scriptFieldItems.add(ElasticsearchRules.quote(name)
> > >>>>>> - + ":{\"script\":\"params._source."
> > >>>>>> - + expr.replaceAll("\"", "") + "\"}");
> > >>>>>> + scriptFields.add(ElasticsearchRules.quote(name)
> > >>>>>> + + ":{\"script\":"
> > >>>>>> + // _source (ES2) vs params._source (ES5)
> > >>>>>> + + "\"" +
> > >>>>> implementor.elasticsearchTable.scriptedFieldPrefix() + "."
> > >>>>>> + + expr.replaceAll("\"", "") + "\"}");
> > >>>>>> + }
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + StringBuilder query = new StringBuilder();
> > >>>>>> + if (scriptFields.isEmpty()) {
> > >>>>>> + List<String> newList = Lists.transform(fields, new
> > >>>>> Function<String, String>() {
> > >>>>>> + @Nullable
> > >>>>>> + @Override public String apply(@Nullable String input) {
> > >>>>>> + return ElasticsearchRules.quote(input);
> > >>>>>> + }
> > >>>>>> + });
> > >>>>>> +
> > >>>>>> + final String findString = String.join(", ", newList);
> > >>>>>> + query.append("\"_source\" :
> > [").append(findString).append("]");
> > >>>>>> + } else {
> > >>>>>> + // if scripted fields are present, ES ignores _source
> > attribute
> > >>>>>> + for (String field: fields) {
> > >>>>>> + scriptFields.add(ElasticsearchRules.quote(field) +
> > >>>>> ":{\"script\": "
> > >>>>>> + // _source (ES2) vs params._source (ES5)
> > >>>>>> + + "\"" +
> > >>>>> implementor.elasticsearchTable.scriptedFieldPrefix() + "."
> > >>>>>> + + field + "\"}");
> > >>>>>> }
> > >>>>>> + query.append("\"script_fields\": {" + String.join(", ",
> > >>>>> scriptFields) + "}");
> > >>>>>> }
> > >>>>>> - final String findString = Util.toString(findItems, "", ", ",
> > "");
> > >>>>>> - final String scriptFieldString = "\"script_fields\": {"
> > >>>>>> - + Util.toString(scriptFieldItems, "", ", ", "") + "}";
> > >>>>>> - final String fieldString = "\"_source\" : [" + findString +
> "]"
> > >>>>>> - + ", " + scriptFieldString;
> > >>>>>>
> > >>>>>> for (String opfield : implementor.list) {
> > >>>>>> if (opfield.startsWith("\"_source\"")) {
> > >>>>>> implementor.list.remove(opfield);
> > >>>>>> }
> > >>>>>> }
> > >>>>>> - implementor.add(fieldString);
> > >>>>>> + implementor.add(query.toString());
> > >>>>>> }
> > >>>>>> }
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/pom.xml
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git a/elasticsearch2/pom.xml b/elasticsearch2/pom.xml
> > >>>>>> index f24622c..6fbee03 100644
> > >>>>>> --- a/elasticsearch2/pom.xml
> > >>>>>> +++ b/elasticsearch2/pom.xml
> > >>>>>> @@ -73,6 +73,12 @@ limitations under the License.
> > >>>>>> <version>${elasticsearch-java-driver.version}</version>
> > >>>>>> </dependency>
> > >>>>>> <dependency>
> > >>>>>> + <!-- Lang groovy dependency is needed for testing with
> > embedded
> > >>>>> ES (scripted fields like loc[0]) -->
> > >>>>>> + <groupId>org.elasticsearch.module</groupId>
> > >>>>>> + <artifactId>lang-groovy</artifactId>
> > >>>>>> + <scope>test</scope>
> > >>>>>> + </dependency>
> > >>>>>> + <dependency>
> > >>>>>> <groupId>com.carrotsearch</groupId>
> > >>>>>> <artifactId>hppc</artifactId>
> > >>>>>> <version>${hppc.version}</version>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java
> > >>>>>> index 84370ab..c3d2ac0 100644
> > >>>>>> ---
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java
> > >>>>>> +++
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java
> > >>>>>> @@ -26,6 +26,7 @@ import org.elasticsearch.search.SearchHit;
> > >>>>>> import java.util.Date;
> > >>>>>> import java.util.Iterator;
> > >>>>>> import java.util.List;
> > >>>>>> +import java.util.Locale;
> > >>>>>> import java.util.Map;
> > >>>>>>
> > >>>>>> /**
> > >>>>>> @@ -101,15 +102,18 @@ public class Elasticsearch2Enumerator
> > implements
> > >>>>> Enumerator<Object> {
> > >>>>>> private static Function1<SearchHit, Object[]> listGetter(
> > >>>>>> final List<Map.Entry<String, Class>> fields) {
> > >>>>>> return new Function1<SearchHit, Object[]>() {
> > >>>>>> - public Object[] apply(SearchHit searchHitFields) {
> > >>>>>> + public Object[] apply(SearchHit hit) {
> > >>>>>> Object[] objects = new Object[fields.size()];
> > >>>>>> for (int i = 0; i < fields.size(); i++) {
> > >>>>>> final Map.Entry<String, Class> field = fields.get(i);
> > >>>>>> final String name = field.getKey();
> > >>>>>> - if (searchHitFields.fields().isEmpty()) {
> > >>>>>> - objects[i] =
> > >>> convert(searchHitFields.getSource().get(name),
> > >>>>> field.getValue());
> > >>>>>> + if (hit.fields().isEmpty()) {
> > >>>>>> + objects[i] = convert(hit.getSource().get(name),
> > >>>>> field.getValue());
> > >>>>>> + } else if (hit.fields().containsKey(name)) {
> > >>>>>> + objects[i] = convert(hit.field(name).getValue(),
> > >>>>> field.getValue());
> > >>>>>> } else {
> > >>>>>> - objects[i] =
> > >>>>> convert(searchHitFields.field(name).getValue(), field.getValue());
> > >>>>>> + throw new IllegalStateException(
> > >>>>>> + String.format(Locale.getDefault(), "No result
> > for
> > >>>>> %s", field));
> > >>>>>> }
> > >>>>>> }
> > >>>>>> return objects;
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java
> > >>>>>> index 668402b..46e3fc5 100644
> > >>>>>> ---
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java
> > >>>>>> +++
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java
> > >>>>>> @@ -22,6 +22,9 @@ import
> > org.apache.calcite.schema.impl.AbstractSchema;
> > >>>>>>
> > >>>>>> import com.carrotsearch.hppc.cursors.ObjectObjectCursor;
> > >>>>>>
> > >>>>>> +import com.google.common.annotations.VisibleForTesting;
> > >>>>>> +import com.google.common.base.Preconditions;
> > >>>>>> +
> > >>>>>> import com.google.common.collect.ImmutableList;
> > >>>>>> import com.google.common.collect.ImmutableMap;
> > >>>>>>
> > >>>>>> @@ -86,6 +89,16 @@ public class Elasticsearch2Schema extends
> > >>>>> AbstractSchema
> > >>>>>> }
> > >>>>>> }
> > >>>>>>
> > >>>>>> + /**
> > >>>>>> + * Allows schema to be instantiated from existing elastic
> search
> > >>>>> client.
> > >>>>>> + * This constructor is used in tests.
> > >>>>>> + */
> > >>>>>> + @VisibleForTesting
> > >>>>>> + Elasticsearch2Schema(Client client, String index) {
> > >>>>>> + this.client = Preconditions.checkNotNull(client, "client");
> > >>>>>> + this.index = Preconditions.checkNotNull(index, "index");
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> @Override protected Map<String, Table> getTableMap() {
> > >>>>>> final ImmutableMap.Builder<String, Table> builder =
> > >>>>> ImmutableMap.builder();
> > >>>>>>
> > >>>>>> @@ -120,7 +133,8 @@ public class Elasticsearch2Schema extends
> > >>>>> AbstractSchema
> > >>>>>>
> > >>>>>> final List<DiscoveryNode> nodes =
> > >>>>> ImmutableList.copyOf(transportClient.connectedNodes());
> > >>>>>> if (nodes.isEmpty()) {
> > >>>>>> - throw new RuntimeException("Cannot connect to any
> > elasticsearch
> > >>>>> nodes");
> > >>>>>> + throw new IllegalStateException("Cannot connect to any
> > >>>>> elasticsearch node: "
> > >>>>>> + + transportNodes);
> > >>>>>> }
> > >>>>>>
> > >>>>>> client = transportClient;
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java
> > >>>>>> index 636aa5f..2928835 100644
> > >>>>>> ---
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java
> > >>>>>> +++
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java
> > >>>>>> @@ -45,8 +45,15 @@ public class Elasticsearch2Table extends
> > >>>>> AbstractElasticsearchTable {
> > >>>>>> this.client = client;
> > >>>>>> }
> > >>>>>>
> > >>>>>> + /**
> > >>>>>> + * ES version 2.x. To access document attributes ES2 uses
> {@code
> > >>>>> _source.foo} syntax.
> > >>>>>> + */
> > >>>>>> + @Override protected String scriptedFieldPrefix() {
> > >>>>>> + return "_source";
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> @Override protected Enumerable<Object> find(String index,
> > List<String>
> > >>>>> ops,
> > >>>>>> - List<Map.Entry<String, Class>> fields) {
> > >>>>>> +
> > List<Map.Entry<String,
> > >>>>> Class>> fields) {
> > >>>>>> final String dbName = index;
> > >>>>>>
> > >>>>>> final String queryString = "{" + Util.toString(ops, "", ", ",
> "") +
> > >>>>> "}";
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java
> > >>>>>> new file mode 100644
> > >>>>>> index 0000000..287e094
> > >>>>>> --- /dev/null
> > >>>>>> +++
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java
> > >>>>>> @@ -0,0 +1,395 @@
> > >>>>>> +/*
> > >>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or
> > more
> > >>>>>> + * contributor license agreements. See the NOTICE file
> distributed
> > >>> with
> > >>>>>> + * this work for additional information regarding copyright
> > ownership.
> > >>>>>> + * The ASF licenses this file to you under the Apache License,
> > Version
> > >>>>> 2.0
> > >>>>>> + * (the "License"); you may not use this file except in
> compliance
> > >>> with
> > >>>>>> + * the License. You may obtain a copy of the License at
> > >>>>>> + *
> > >>>>>> + * http://www.apache.org/licenses/LICENSE-2.0
> > >>>>>> + *
> > >>>>>> + * Unless required by applicable law or agreed to in writing,
> > software
> > >>>>>> + * distributed under the License is distributed on an "AS IS"
> > BASIS,
> > >>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
> or
> > >>>>> implied.
> > >>>>>> + * See the License for the specific language governing
> permissions
> > and
> > >>>>>> + * limitations under the License.
> > >>>>>> + */
> > >>>>>> +package org.apache.calcite.adapter.elasticsearch2;
> > >>>>>> +
> > >>>>>> +import org.apache.calcite.jdbc.CalciteConnection;
> > >>>>>> +import org.apache.calcite.schema.SchemaPlus;
> > >>>>>> +import org.apache.calcite.schema.impl.ViewTable;
> > >>>>>> +import org.apache.calcite.schema.impl.ViewTableMacro;
> > >>>>>> +import org.apache.calcite.test.CalciteAssert;
> > >>>>>> +import org.apache.calcite.test.ElasticChecker;
> > >>>>>> +
> > >>>>>> +import com.google.common.io.LineProcessor;
> > >>>>>> +import com.google.common.io.Resources;
> > >>>>>> +
> > >>>>>> +import org.elasticsearch.action.bulk.BulkItemResponse;
> > >>>>>> +import org.elasticsearch.action.bulk.BulkRequestBuilder;
> > >>>>>> +import org.elasticsearch.action.bulk.BulkResponse;
> > >>>>>> +import org.elasticsearch.common.xcontent.XContentBuilder;
> > >>>>>> +import org.elasticsearch.common.xcontent.XContentFactory;
> > >>>>>> +
> > >>>>>> +import org.junit.BeforeClass;
> > >>>>>> +import org.junit.ClassRule;
> > >>>>>> +import org.junit.Test;
> > >>>>>> +
> > >>>>>> +import java.io.IOException;
> > >>>>>> +import java.nio.charset.StandardCharsets;
> > >>>>>> +import java.sql.Connection;
> > >>>>>> +import java.sql.DriverManager;
> > >>>>>> +import java.sql.SQLException;
> > >>>>>> +import java.util.Arrays;
> > >>>>>> +import java.util.Collections;
> > >>>>>> +import java.util.Locale;
> > >>>>>> +
> > >>>>>> +/**
> > >>>>>> + * Set of tests for ES adapter. Uses real instance via {@link
> > >>>>> EmbeddedElasticRule}. Document
> > >>>>>> + * source is local {@code zips-mini.json} file (located in the
> > >>>>> classpath).
> > >>>>>> + */
> > >>>>>> +public class ElasticSearch2AdapterTest {
> > >>>>>> +
> > >>>>>> + @ClassRule //init once for all tests
> > >>>>>> + public static final EmbeddedElasticRule NODE =
> > >>>>> EmbeddedElasticRule.create();
> > >>>>>> +
> > >>>>>> + private static final String ZIPS = "zips";
> > >>>>>> +
> > >>>>>> + /**
> > >>>>>> + * Used to create {@code zips} index and insert some data
> > >>>>>> + */
> > >>>>>> + @BeforeClass
> > >>>>>> + public static void setupInstance() throws Exception {
> > >>>>>> + // define mapping so fields are searchable (term query)
> > >>>>>> + XContentBuilder mapping =
> > >>>>> XContentFactory.jsonBuilder().startObject()
> > >>>>>> + .startObject("properties")
> > >>>>>> + .startObject("city").field("type", "string")
> > >>>>>> + .field("index", "not_analyzed").endObject()
> > >>>>>> + .startObject("state").field("type", "string")
> > >>>>>> + .field("index", "not_analyzed").endObject()
> > >>>>>> + .startObject("pop").field("type", "long").endObject()
> > >>>>>> + .endObject()
> > >>>>>> + .endObject();
> > >>>>>> +
> > >>>>>> + // create index
> > >>>>>> + NODE.client().admin().indices()
> > >>>>>> + .prepareCreate(ZIPS)
> > >>>>>> + .addMapping(ZIPS, mapping)
> > >>>>>> + .get();
> > >>>>>> +
> > >>>>>> + BulkRequestBuilder bulk =
> > >>>>> NODE.client().prepareBulk().setRefresh(true);
> > >>>>>> +
> > >>>>>> + // load records from file
> > >>>>>> +
> > >>>>>
> > >>>
> >
> Resources.readLines(ElasticSearch2AdapterTest.class.getResource("/zips-mini.json"),
> > >>>>>> + StandardCharsets.UTF_8, new LineProcessor<Void>() {
> > >>>>>> + @Override public boolean processLine(String line)
> > throws
> > >>>>> IOException {
> > >>>>>> + line = line.replaceAll("_id", "id"); // _id is a
> > >>>>> reserved attribute in ES
> > >>>>>> + bulk.add(NODE.client().prepareIndex(ZIPS,
> > >>>>> ZIPS).setSource(line));
> > >>>>>> + return true;
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Override public Void getResult() {
> > >>>>>> + return null;
> > >>>>>> + }
> > >>>>>> + });
> > >>>>>> +
> > >>>>>> + if (bulk.numberOfActions() == 0) {
> > >>>>>> + throw new IllegalStateException("No records to be
> indexed");
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + BulkResponse response = bulk.execute().get();
> > >>>>>> +
> > >>>>>> + if (response.hasFailures()) {
> > >>>>>> + throw new IllegalStateException(
> > >>>>>> + String.format(Locale.getDefault(), "Failed to
> > populate
> > >>>>> %s:\n%s", NODE.httpAddress(),
> > >>>>>> +
> > >>>>>
> Arrays.stream(response.getItems()).filter(BulkItemResponse::isFailed)
> > >>>>>> +
> > >>>>>
> > >>>
> >
> .map(BulkItemResponse::getFailureMessage).findFirst().orElse("<unknown>")));
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + private CalciteAssert.ConnectionFactory newConnectionFactory()
> {
> > >>>>>> + return new CalciteAssert.ConnectionFactory() {
> > >>>>>> + @Override public Connection createConnection() throws
> > >>>>> SQLException {
> > >>>>>> + final Connection connection =
> > >>>>> DriverManager.getConnection("jdbc:calcite:");
> > >>>>>> + final SchemaPlus root =
> > >>>>> connection.unwrap(CalciteConnection.class).getRootSchema();
> > >>>>>> +
> > >>>>>> + root.add("elastic", new
> Elasticsearch2Schema(NODE.client(),
> > >>>>> ZIPS));
> > >>>>>> +
> > >>>>>> + // add calcite view programmatically
> > >>>>>> + final String viewSql = "select cast(_MAP['city'] AS
> > >>>>> varchar(20)) AS \"city\", "
> > >>>>>> + + " cast(_MAP['loc'][0] AS float) AS
> > \"longitude\",\n"
> > >>>>>> + + " cast(_MAP['loc'][1] AS float) AS
> > \"latitude\",\n"
> > >>>>>> + + " cast(_MAP['pop'] AS integer) AS \"pop\", "
> > >>>>>> + + " cast(_MAP['state'] AS varchar(2)) AS
> \"state\",
> > "
> > >>>>>> + + " cast(_MAP['id'] AS varchar(5)) AS \"id\" "
> > >>>>>> + + "from \"elastic\".\"zips\"";
> > >>>>>> +
> > >>>>>> + ViewTableMacro macro = ViewTable.viewMacro(root, viewSql,
> > >>>>>> + Collections.singletonList("elastic"),
> > >>>>> Arrays.asList("elastic", "view"), false);
> > >>>>>> + root.add("ZIPS", macro);
> > >>>>>> +
> > >>>>>> + return connection;
> > >>>>>> + }
> > >>>>>> + };
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + private CalciteAssert.AssertThat calciteAssert() {
> > >>>>>> + return CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory());
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + /**
> > >>>>>> + * Tests using calcite view
> > >>>>>> + */
> > >>>>>> + @Test
> > >>>>>> + public void view() throws Exception {
> > >>>>>> + calciteAssert()
> > >>>>>> + .query("select * from zips where \"city\" =
> 'BROOKLYN'")
> > >>>>>> + .returns("city=BROOKLYN; longitude=-73.956985;
> > >>>>> latitude=40.646694; "
> > >>>>>> + + "pop=111396; state=NY; id=11226\n")
> > >>>>>> + .returnsCount(1);
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test
> > >>>>>> + public void emptyResult() {
> > >>>>>> + CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory())
> > >>>>>> + .query("select * from zips limit 0")
> > >>>>>> + .returnsCount(0);
> > >>>>>> +
> > >>>>>> + CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory())
> > >>>>>> + .query("select * from \"elastic\".\"zips\" where
> > >>>>> _MAP['Foo'] = '_MISSING_'")
> > >>>>>> + .returnsCount(0);
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test
> > >>>>>> + public void basic() throws Exception {
> > >>>>>> + CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory())
> > >>>>>> + .query("select * from \"elastic\".\"zips\" where
> > >>>>> _MAP['city'] = 'BROOKLYN'")
> > >>>>>> + .returnsCount(1);
> > >>>>>> +
> > >>>>>> + CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory())
> > >>>>>> + .query("select * from \"elastic\".\"zips\" where"
> > >>>>>> + + " _MAP['city'] in ('BROOKLYN',
> > 'WASHINGTON')")
> > >>>>>> + .returnsCount(2);
> > >>>>>> +
> > >>>>>> + // lower-case
> > >>>>>> + CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory())
> > >>>>>> + .query("select * from \"elastic\".\"zips\" where "
> > >>>>>> + + "_MAP['city'] in ('brooklyn', 'Brooklyn',
> > >>>>> 'BROOK') ")
> > >>>>>> + .returnsCount(0);
> > >>>>>> +
> > >>>>>> + // missing field
> > >>>>>> + CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory())
> > >>>>>> + .query("select * from \"elastic\".\"zips\" where
> > >>>>> _MAP['CITY'] = 'BROOKLYN'")
> > >>>>>> + .returnsCount(0);
> > >>>>>> +
> > >>>>>> + // limit works
> > >>>>>> + CalciteAssert.that()
> > >>>>>> + .with(newConnectionFactory())
> > >>>>>> + .query("select * from \"elastic\".\"zips\" limit 42")
> > >>>>>> + .returnsCount(42);
> > >>>>>> +
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testSort() {
> > >>>>>> + final String explain =
> > "PLAN=ElasticsearchToEnumerableConverter\n"
> > >>>>>> + + " ElasticsearchSort(sort0=[$4], dir0=[ASC])\n"
> > >>>>>> + + " ElasticsearchProject(city=[CAST(ITEM($0,
> > >>>>> 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE
> > >>>>> \"ISO-8859-1$en_US$primary\"], longitude=[CAST(ITEM(ITEM($0,
> 'loc'),
> > >>>>> 0)):FLOAT], latitude=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT],
> > >>>>> pop=[CAST(ITEM($0, 'pop')):INTEGER], state=[CAST(ITEM($0,
> > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE
> > >>>>> \"ISO-8859-1$en_US$primary\"], id=[CAST(ITEM($0, 'id')):VARCHAR(5)
> > >>>>> CHARACTER SET \"ISO-8859-1\" COLLATE
> > \"ISO-8859-1$en_US$primary\"])\n"
> > >>>>>> + + " ElasticsearchTableScan(table=[[elastic,
> > zips]])";
> > >>>>>> +
> > >>>>>> + calciteAssert()
> > >>>>>> + .query("select * from zips order by \"state\"")
> > >>>>>> + .returnsCount(10)
> > >>>>>> + .explainContains(explain);
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testSortLimit() {
> > >>>>>> + final String sql = "select \"state\", \"id\" from zips\n"
> > >>>>>> + + "order by \"state\", \"id\" offset 2 rows fetch
> next
> > 3
> > >>>>> rows only";
> > >>>>>> + calciteAssert()
> > >>>>>> + .query(sql)
> > >>>>>> + .returnsUnordered("state=AK; id=99801",
> > >>>>>> + "state=AL; id=35215",
> > >>>>>> + "state=AL; id=35401")
> > >>>>>> + .queryContains(
> > >>>>>> + ElasticChecker.elasticsearchChecker(
> > >>>>>> + "\"_source\" : [\"state\", \"id\"]",
> > >>>>>> + "\"sort\": [ {\"state\": \"asc\"},
> > >>> {\"id\":
> > >>>>> \"asc\"}]",
> > >>>>>> + "\"from\": 2",
> > >>>>>> + "\"size\": 3"));
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> +
> > >>>>>> +
> > >>>>>> + @Test public void testOffsetLimit() {
> > >>>>>> + final String sql = "select \"state\", \"id\" from zips\n"
> > >>>>>> + + "offset 2 fetch next 3 rows only";
> > >>>>>> + calciteAssert()
> > >>>>>> + .query(sql)
> > >>>>>> + .runs()
> > >>>>>> + .queryContains(
> > >>>>>> + ElasticChecker.elasticsearchChecker(
> > >>>>>> + "\"from\": 2",
> > >>>>>> + "\"size\": 3",
> > >>>>>> + "\"_source\" : [\"state\",
> \"id\"]"));
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testLimit() {
> > >>>>>> + final String sql = "select \"state\", \"id\" from zips\n"
> > >>>>>> + + "fetch next 3 rows only";
> > >>>>>> +
> > >>>>>> + calciteAssert()
> > >>>>>> + .query(sql)
> > >>>>>> + .runs()
> > >>>>>> + .queryContains(
> > >>>>>> + ElasticChecker.elasticsearchChecker(
> > >>>>>> + "\"size\": 3",
> > >>>>>> + "\"_source\" : [\"state\",
> \"id\"]"));
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testFilterSort() {
> > >>>>>> + final String sql = "select * from zips\n"
> > >>>>>> + + "where \"state\" = 'CA' and \"id\" >= '70000'\n"
> > >>>>>> + + "order by \"state\", \"id\"";
> > >>>>>> + final String explain =
> > "PLAN=ElasticsearchToEnumerableConverter\n"
> > >>>>>> + + " ElasticsearchSort(sort0=[$4], sort1=[$5],
> > dir0=[ASC],
> > >>>>> dir1=[ASC])\n"
> > >>>>>> + + " ElasticsearchProject(city=[CAST(ITEM($0,
> > >>>>> 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE
> > >>>>> \"ISO-8859-1$en_US$primary\"], longitude=[CAST(ITEM(ITEM($0,
> 'loc'),
> > >>>>> 0)):FLOAT], latitude=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT],
> > >>>>> pop=[CAST(ITEM($0, 'pop')):INTEGER], state=[CAST(ITEM($0,
> > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE
> > >>>>> \"ISO-8859-1$en_US$primary\"], id=[CAST(ITEM($0, 'id')):VARCHAR(5)
> > >>>>> CHARACTER SET \"ISO-8859-1\" COLLATE
> > \"ISO-8859-1$en_US$primary\"])\n"
> > >>>>>> + + "
> > >>> ElasticsearchFilter(condition=[AND(=(CAST(ITEM($0,
> > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE
> > >>>>> \"ISO-8859-1$en_US$primary\", 'CA'), >=(CAST(ITEM($0,
> > 'id')):VARCHAR(5)
> > >>>>> CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\",
> > >>>>> '70000'))])\n"
> > >>>>>> + + " ElasticsearchTableScan(table=[[elastic,
> > >>> zips]])";
> > >>>>>> + calciteAssert()
> > >>>>>> + .query(sql)
> > >>>>>> + .returnsOrdered("city=LOS ANGELES;
> > longitude=-118.258189;
> > >>>>> latitude=34.007856; "
> > >>>>>> + + "pop=96074; state=CA; id=90011",
> > >>>>>> + "city=BELL GARDENS; longitude=-118.17205;
> > >>>>> latitude=33.969177; "
> > >>>>>> + + "pop=99568; state=CA; id=90201",
> > >>>>>> + "city=NORWALK; longitude=-118.081767;
> > >>>>> latitude=33.90564; "
> > >>>>>> + + "pop=94188; state=CA; id=90650")
> > >>>>>> + .queryContains(
> > >>>>>> +
> ElasticChecker.elasticsearchChecker("\"query\"
> > : "
> > >>>>>> + +
> > >>>>> "{\"constant_score\":{\"filter\":{\"bool\":"
> > >>>>>> + +
> > >>>>> "{\"must\":[{\"term\":{\"state\":\"CA\"}},"
> > >>>>>> + +
> > >>>>> "{\"range\":{\"id\":{\"gte\":\"70000\"}}}]}}}}",
> > >>>>>> + "\"script_fields\":
> > >>>>> {\"longitude\":{\"script\":\"_source.loc[0]\"}, "
> > >>>>>> + +
> > >>>>> "\"latitude\":{\"script\":\"_source.loc[1]\"}, "
> > >>>>>> + + "\"city\":{\"script\":
> > >>>>> \"_source.city\"}, "
> > >>>>>> + + "\"pop\":{\"script\":
> > >>>>> \"_source.pop\"}, "
> > >>>>>> + + "\"state\":{\"script\":
> > >>>>> \"_source.state\"}, "
> > >>>>>> + + "\"id\":{\"script\": \"_
> > >>> source.id
> > >>>>> \"}}",
> > >>>>>> + "\"sort\": [ {\"state\": \"asc\"},
> > >>> {\"id\":
> > >>>>> \"asc\"}]"))
> > >>>>>> + .explainContains(explain);
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testFilterSortDesc() {
> > >>>>>> + final String sql = "select * from zips\n"
> > >>>>>> + + "where \"pop\" BETWEEN 95000 AND 100000\n"
> > >>>>>> + + "order by \"state\" desc, \"pop\"";
> > >>>>>> + calciteAssert()
> > >>>>>> + .query(sql)
> > >>>>>> + .limit(4)
> > >>>>>> + .returnsOrdered(
> > >>>>>> + "city=LOS ANGELES; longitude=-118.258189;
> > >>>>> latitude=34.007856; pop=96074; state=CA; id=90011",
> > >>>>>> + "city=BELL GARDENS; longitude=-118.17205;
> > >>>>> latitude=33.969177; pop=99568; state=CA; id=90201");
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testFilterRedundant() {
> > >>>>>> + final String sql = "select * from zips\n"
> > >>>>>> + + "where \"state\" > 'CA' and \"state\" < 'AZ' and
> > >>>>> \"state\" = 'OK'";
> > >>>>>> + calciteAssert()
> > >>>>>> + .query(sql)
> > >>>>>> + .runs()
> > >>>>>> + .queryContains(
> > >>>>>> + ElasticChecker.elasticsearchChecker(""
> > >>>>>> + + "\"query\" :
> > >>>>> {\"constant_score\":{\"filter\":{\"bool\":"
> > >>>>>> + +
> > >>>>> "{\"must\":[{\"term\":{\"state\":\"OK\"}}]}}}}",
> > >>>>>> + "\"script_fields\":
> > >>>>> {\"longitude\":{\"script\":\"_source.loc[0]\"}, "
> > >>>>>> + +
> > >>>>> "\"latitude\":{\"script\":\"_source.loc[1]\"}, "
> > >>>>>> + + "\"city\":{\"script\":
> > >>>>> \"_source.city\"}, "
> > >>>>>> + + "\"pop\":{\"script\":
> > >>>>> \"_source.pop\"}, \"state\":{\"script\": \"_source.state\"}, "
> > >>>>>> + +
> > "\"id\":{\"script\":
> > >>>>> \"_source.id\"}}"
> > >>>>>> + ));
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testInPlan() {
> > >>>>>> + final String[] searches = {
> > >>>>>> + "\"query\" :
> > >>>>> {\"constant_score\":{\"filter\":{\"bool\":{\"should\":"
> > >>>>>> + +
> > >>>>>
> > >>>
> >
> "[{\"bool\":{\"must\":[{\"term\":{\"pop\":96074}}]}},{\"bool\":{\"must\":[{\"term\":"
> > >>>>>> + + "{\"pop\":99568}}]}}]}}}}",
> > >>>>>> + "\"script_fields\":
> > >>>>> {\"longitude\":{\"script\":\"_source.loc[0]\"}, "
> > >>>>>> + +
> "\"latitude\":{\"script\":\"_source.loc[1]\"},
> > "
> > >>>>>> + + "\"city\":{\"script\": \"_source.city\"}, "
> > >>>>>> + + "\"pop\":{\"script\": \"_source.pop\"}, "
> > >>>>>> + + "\"state\":{\"script\": \"_source.state\"}, "
> > >>>>>> + + "\"id\":{\"script\": \"_source.id\"}}"
> > >>>>>> + };
> > >>>>>> +
> > >>>>>> + calciteAssert()
> > >>>>>> + .query("select * from zips where \"pop\" in (96074,
> > >>> 99568)")
> > >>>>>> + .returnsUnordered(
> > >>>>>> + "city=BELL GARDENS; longitude=-118.17205;
> > >>>>> latitude=33.969177; pop=99568; state=CA; id=90201",
> > >>>>>> + "city=LOS ANGELES; longitude=-118.258189;
> > >>>>> latitude=34.007856; pop=96074; state=CA; id=90011"
> > >>>>>> + )
> > >>>>>> +
> > >>>>> .queryContains(ElasticChecker.elasticsearchChecker(searches));
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testZips() {
> > >>>>>> + calciteAssert()
> > >>>>>> + .query("select \"state\", \"city\" from zips")
> > >>>>>> + .returnsCount(10);
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testProject() {
> > >>>>>> + final String sql = "select \"state\", \"city\", 0 as
> > \"zero\"\n"
> > >>>>>> + + "from zips\n"
> > >>>>>> + + "order by \"state\", \"city\"";
> > >>>>>> +
> > >>>>>> + calciteAssert()
> > >>>>>> + .query(sql)
> > >>>>>> + .limit(2)
> > >>>>>> + .returnsUnordered("state=AK; city=ANCHORAGE; zero=0",
> > >>>>>> + "state=AK; city=FAIRBANKS; zero=0")
> > >>>>>> + .queryContains(
> > >>>>>> +
> > >>>>> ElasticChecker.elasticsearchChecker("\"script_fields\": "
> > >>>>>> + + "{\"zero\":{\"script\":
> > \"0\"},
> > >>> "
> > >>>>>> + + "\"state\":{\"script\":
> > >>>>> \"_source.state\"}, "
> > >>>>>> + + "\"city\":{\"script\":
> > >>>>> \"_source.city\"}}",
> > >>>>>> + "\"sort\": [ {\"state\": \"asc\"},
> > >>>>> {\"city\": \"asc\"}]"));
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testFilter() {
> > >>>>>> + final String explain =
> > "PLAN=ElasticsearchToEnumerableConverter\n"
> > >>>>>> + + " ElasticsearchProject(state=[CAST(ITEM($0,
> > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE
> > >>>>> \"ISO-8859-1$en_US$primary\"], city=[CAST(ITEM($0,
> > 'city')):VARCHAR(20)
> > >>>>> CHARACTER SET \"ISO-8859-1\" COLLATE
> > \"ISO-8859-1$en_US$primary\"])\n"
> > >>>>>> + + " ElasticsearchFilter(condition=[=(CAST(ITEM($0,
> > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE
> > >>>>> \"ISO-8859-1$en_US$primary\", 'CA')])\n"
> > >>>>>> + + " ElasticsearchTableScan(table=[[elastic,
> > zips]])";
> > >>>>>> + calciteAssert()
> > >>>>>> + .query("select \"state\", \"city\" from zips where
> > >>>>> \"state\" = 'CA'")
> > >>>>>> + .limit(3)
> > >>>>>> + .returnsUnordered("state=CA; city=BELL GARDENS",
> > >>>>>> + "state=CA; city=LOS ANGELES",
> > >>>>>> + "state=CA; city=NORWALK")
> > >>>>>> + .explainContains(explain);
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> + @Test public void testFilterReversed() {
> > >>>>>> + calciteAssert()
> > >>>>>> + .query("select \"state\", \"city\" from zips where
> 'WI' <
> > >>>>> \"state\" order by \"city\"")
> > >>>>>> + .limit(2)
> > >>>>>> + .returnsUnordered("state=WV; city=BECKLEY",
> > >>>>>> + "state=WY; city=CHEYENNE");
> > >>>>>> + calciteAssert()
> > >>>>>> + .query("select \"state\", \"city\" from zips where
> > \"state\"
> > >>>>>> 'WI' order by \"city\"")
> > >>>>>> + .limit(2)
> > >>>>>> + .returnsUnordered("state=WV; city=BECKLEY",
> > >>>>>> + "state=WY; city=CHEYENNE");
> > >>>>>> + }
> > >>>>>> +
> > >>>>>> +}
> > >>>>>> +
> > >>>>>> +// End ElasticSearch2AdapterTest.java
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java
> > >>>>>>
> > ----------------------------------------------------------------------
> > >>>>>> diff --git
> > >>>>>
> > >>>
> >
> a/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java
> > >>>>>> new file mode 100644
> > >>>>>> index 0000000..4474add
> > >>>>>> --- /dev/null
> > >>>>>> +++
> > >>>>>
> > >>>
> >
> b/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java
> > >>>>>> @@ -0,0 +1,147 @@
> > >>>>>> +/*
> > >>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or
> > more
> > >>>>>> + * contributor license agreements. See the NOTICE file
> distributed
> > >>> with
> > >>>>>> + * this work for additional information regarding copyright
> > ownership.
> > >>>>>> + * The ASF licenses this file to you under the Apache License,
> > Version
> > >>>>> 2.0
> > >>>>>> + * (the "License"); you may not use this file except in
> compliance
> > >>> with
> > >>>>>> + * the License. You may obtain a copy of the License at
> > >>>>>> + *
> > >>>>>> + * http://www.apache.org/licenses/LICENSE-2.0
> > >>>>>> + *
> > >>>>>> + * Unless required by applicable law or agreed to in writing,
> > software
> > >>>>>> + * distributed under the License is distributed on an "AS IS"
> > BASIS,
> > >>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
> or
> > >>>>> implied.
> > >>>>>> + * See the License for the specific language governing
> permissions
> > and
> > >>>>>> + * limitations under the License.
> > >>>>>> + */
> > >>>>>> +package org.apache.calcite.adapter.elasticsearch2;
> > >>>>>> +
> > >>>>>> +import com.google.common.base.Preconditions;
> > >>>>>>
