> On July 27, 2015, 4:37 a.m., Amareshwari Sriramadasu wrote: > > pom.xml, line 51 > > <https://reviews.apache.org/r/36434/diff/6/?file=1020755#file1020755line51> > > > > Whats the license for jest? > > Amruth Sampath wrote: > its apache 2.0 https://github.com/searchbox-io/Jest/blob/master/LICENSE
No issue with licenses. Checked https://github.com/FasterXML/jackson-datatype-guava as well. It is also Apache License. > On July 27, 2015, 4:37 a.m., Amareshwari Sriramadasu wrote: > > src/site/apt/admin/esdriver-config.apt, line 20 > > <https://reviews.apache.org/r/36434/diff/6/?file=1020756#file1020756line20> > > > > Can you update config-server.apt and link this page from there? > > Amruth Sampath wrote: > Is this fine? > > * Driver configuration > > The supported drivers are configured through classnames in this > release. The configuration > for each driver can be specified in their site.xml file. For example, > HiveDriver's > configuration should be specified in hive-site.xml. Supported drivers > in this version > are org.apache.lens.driver.hive.HiveDriver, > org.apache.lens.driver.jdbc.JDBCDriver > and org.apache.lens.driver.es.ESDriver. > The configuration parameters and their description are explained in > their respective docs listed below > HiveDriver - {{{./hivedriver-config.html} here}} > JDBCDriver - {{{./jdbcdriver-config.html} here}} > ESDriver - {{{./esdriver-config.html} here}}. > For overridding configurations, use the appropriate site.xml as listed > below > hive - hivedriver-site.xml > jdbc - jdbc-driver-site.xml > es - esdriver-site.xml Looks fine. Seems existing doc wrt "For example, HiveDriver's configuration should be specified in hive-site.xml" seems wrong - should have been hivedriver-site.xml. Can you correct it ? - Amareshwari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36434/#review93069 ----------------------------------------------------------- On July 24, 2015, 7:32 a.m., Amruth Sampath wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36434/ > ----------------------------------------------------------- > > (Updated July 24, 2015, 7:32 a.m.) > > > Review request for lens, Amareshwari Sriramadasu, Jaideep dhok, and sharad > agarwal. > > > Bugs: LENS-252 > https://issues.apache.org/jira/browse/LENS-252 > > > Repository: lens > > > Description > ------- > > Elastic search driver for lens > ~~~~~~~~~~~~~~~~~~~~~~ > Elastic search accepts a nested json as a query and returns a json result. > The json result is nested for group by queries and simple for simple selects. > HQL -> ES json query > ~~~~~~~~~~~~~~~~~ > -> I have written a traversal (ASTTraverserForES)(specific for noSQL stores). > The traversal could be used for any purpose like query building/validation > etc. > -> The traversal will take in a query visitor (ASTVisitor.java) (for building > the query) and a criteria visitor (for building the where clause). I have > checked in concrete visitors for ElasticSearch that can build the elastic > search json query. > Elasticsearch client > ~~~~~~~~~~~~~~~ > -> There are multiple choices of elasticsearch client available. I've made > the client pluggable. > -> I've added one default HTTPClient implementation (Jest library - apache > 2). The choice of HTTP client over transport client was made because of the > version consistency requirement between the transport client and the ES > server. > -> Every client has to implement an execute method that takes in the query > and returns a LensResultSet. Hence the transformation of resultset must also > be done by the client implementation. > Elasticsearch Jest json response -> LensResultSet > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > The result set obtained could be a simple hit result (document response) or > facets response. > -> Facets response has a tree structure. Finding all the paths in the tree > will give us all the rows. Look at JestResultSetTransformer > (AggregateTransformer) > -> Hits response is straightforward to decode (TermTransformer) > Known issues/shortcommings > ~~~~~~~~~~~~~~~~~~~~~~~~ > -> Scrolling responses for aggregate queries (by design ES always returns the > complete bucket in a single json - there is no scroll facility) > -> Order by in aggregate queries. Fully functional order by queries can get > complex as the ordering by measure can happen only in the immediate parent > group by. 'Limit' is also blocked as it could be misleading to have limit > without order by. (Please note that order by and limit will still work in > queries without group bys) > -> *, count is not available as of now. > -> support for other UDFs. Right now common UDAFs like sum, min, max are > supported. We need a way to seamlessly translate a new UDF to elastic search > without code change > -> Query estimation > -> Session level config injection for properties like fetch size and group by > cardinality size? (Right now these configs are at driver level) > Have added a esdriver-default.xml for looking up default properties > > > Diffs > ----- > > lens-api/src/test/java/org/apache/lens/doc/TestGenerateConfigDoc.java > 5ef4186 > lens-driver-es/pom.xml PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/ASTTraverserForES.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriverConfig.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESQuery.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/ESClient.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/ESResultSet.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/jest/JestClientImpl.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/jest/JestResultSetTransformer.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/exceptions/ESClientException.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/exceptions/InvalidQueryException.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/ASTCriteriaVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/ASTVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/CriteriaVisitorFactory.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/ESVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESAggregateVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESCriteriaVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESCriteriaVisitorFactory.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESTermVisitor.java > PRE-CREATION > lens-driver-es/src/main/resources/esdriver-default.xml PRE-CREATION > lens-driver-es/src/test/java/org/apache/lens/driver/es/ESDriverTest.java > PRE-CREATION > lens-driver-es/src/test/java/org/apache/lens/driver/es/MockClientES.java > PRE-CREATION > > lens-driver-es/src/test/java/org/apache/lens/driver/es/QueryTranslationTest.java > PRE-CREATION > > lens-driver-es/src/test/java/org/apache/lens/driver/es/ResultSetTransformationTest.java > PRE-CREATION > > lens-driver-es/src/test/java/org/apache/lens/driver/es/ScrollingQueryTest.java > PRE-CREATION > lens-driver-es/src/test/resources/invalid-queries.data PRE-CREATION > lens-driver-es/src/test/resources/valid-queries.data PRE-CREATION > lens-server/pom.xml 35e7718 > pom.xml 15ac998 > src/site/apt/admin/esdriver-config.apt PRE-CREATION > > Diff: https://reviews.apache.org/r/36434/diff/ > > > Testing > ------- > > Added unit test cases for testing > - query translation > - result set translation > - Scrolling > > > Thanks, > > Amruth Sampath > >
