> On July 27, 2015, 4:37 a.m., Amareshwari Sriramadasu wrote: > > pom.xml, line 51 > > <https://reviews.apache.org/r/36434/diff/6/?file=1020755#file1020755line51> > > > > Whats the license for jest?
its apache 2.0 https://github.com/searchbox-io/Jest/blob/master/LICENSE > On July 27, 2015, 4:37 a.m., Amareshwari Sriramadasu wrote: > > src/site/apt/admin/esdriver-config.apt, line 20 > > <https://reviews.apache.org/r/36434/diff/6/?file=1020756#file1020756line20> > > > > Can you update config-server.apt and link this page from there? Is this fine? * Driver configuration The supported drivers are configured through classnames in this release. The configuration for each driver can be specified in their site.xml file. For example, HiveDriver's configuration should be specified in hive-site.xml. Supported drivers in this version are org.apache.lens.driver.hive.HiveDriver, org.apache.lens.driver.jdbc.JDBCDriver and org.apache.lens.driver.es.ESDriver. The configuration parameters and their description are explained in their respective docs listed below HiveDriver - {{{./hivedriver-config.html} here}} JDBCDriver - {{{./jdbcdriver-config.html} here}} ESDriver - {{{./esdriver-config.html} here}}. For overridding configurations, use the appropriate site.xml as listed below hive - hivedriver-site.xml jdbc - jdbc-driver-site.xml es - esdriver-site.xml - Amruth ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36434/#review93069 ----------------------------------------------------------- On July 24, 2015, 7:32 a.m., Amruth Sampath wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36434/ > ----------------------------------------------------------- > > (Updated July 24, 2015, 7:32 a.m.) > > > Review request for lens, Amareshwari Sriramadasu, Jaideep dhok, and sharad > agarwal. > > > Bugs: LENS-252 > https://issues.apache.org/jira/browse/LENS-252 > > > Repository: lens > > > Description > ------- > > Elastic search driver for lens > ~~~~~~~~~~~~~~~~~~~~~~ > Elastic search accepts a nested json as a query and returns a json result. > The json result is nested for group by queries and simple for simple selects. > HQL -> ES json query > ~~~~~~~~~~~~~~~~~ > -> I have written a traversal (ASTTraverserForES)(specific for noSQL stores). > The traversal could be used for any purpose like query building/validation > etc. > -> The traversal will take in a query visitor (ASTVisitor.java) (for building > the query) and a criteria visitor (for building the where clause). I have > checked in concrete visitors for ElasticSearch that can build the elastic > search json query. > Elasticsearch client > ~~~~~~~~~~~~~~~ > -> There are multiple choices of elasticsearch client available. I've made > the client pluggable. > -> I've added one default HTTPClient implementation (Jest library - apache > 2). The choice of HTTP client over transport client was made because of the > version consistency requirement between the transport client and the ES > server. > -> Every client has to implement an execute method that takes in the query > and returns a LensResultSet. Hence the transformation of resultset must also > be done by the client implementation. > Elasticsearch Jest json response -> LensResultSet > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > The result set obtained could be a simple hit result (document response) or > facets response. > -> Facets response has a tree structure. Finding all the paths in the tree > will give us all the rows. Look at JestResultSetTransformer > (AggregateTransformer) > -> Hits response is straightforward to decode (TermTransformer) > Known issues/shortcommings > ~~~~~~~~~~~~~~~~~~~~~~~~ > -> Scrolling responses for aggregate queries (by design ES always returns the > complete bucket in a single json - there is no scroll facility) > -> Order by in aggregate queries. Fully functional order by queries can get > complex as the ordering by measure can happen only in the immediate parent > group by. 'Limit' is also blocked as it could be misleading to have limit > without order by. (Please note that order by and limit will still work in > queries without group bys) > -> *, count is not available as of now. > -> support for other UDFs. Right now common UDAFs like sum, min, max are > supported. We need a way to seamlessly translate a new UDF to elastic search > without code change > -> Query estimation > -> Session level config injection for properties like fetch size and group by > cardinality size? (Right now these configs are at driver level) > Have added a esdriver-default.xml for looking up default properties > > > Diffs > ----- > > lens-api/src/test/java/org/apache/lens/doc/TestGenerateConfigDoc.java > 5ef4186 > lens-driver-es/pom.xml PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/ASTTraverserForES.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriverConfig.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESQuery.java > PRE-CREATION > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/ESClient.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/ESResultSet.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/jest/JestClientImpl.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/client/jest/JestResultSetTransformer.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/exceptions/ESClientException.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/exceptions/InvalidQueryException.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/ASTCriteriaVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/ASTVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/CriteriaVisitorFactory.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/ESVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESAggregateVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESCriteriaVisitor.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESCriteriaVisitorFactory.java > PRE-CREATION > > lens-driver-es/src/main/java/org/apache/lens/driver/es/translator/impl/ESTermVisitor.java > PRE-CREATION > lens-driver-es/src/main/resources/esdriver-default.xml PRE-CREATION > lens-driver-es/src/test/java/org/apache/lens/driver/es/ESDriverTest.java > PRE-CREATION > lens-driver-es/src/test/java/org/apache/lens/driver/es/MockClientES.java > PRE-CREATION > > lens-driver-es/src/test/java/org/apache/lens/driver/es/QueryTranslationTest.java > PRE-CREATION > > lens-driver-es/src/test/java/org/apache/lens/driver/es/ResultSetTransformationTest.java > PRE-CREATION > > lens-driver-es/src/test/java/org/apache/lens/driver/es/ScrollingQueryTest.java > PRE-CREATION > lens-driver-es/src/test/resources/invalid-queries.data PRE-CREATION > lens-driver-es/src/test/resources/valid-queries.data PRE-CREATION > lens-server/pom.xml 35e7718 > pom.xml 15ac998 > src/site/apt/admin/esdriver-config.apt PRE-CREATION > > Diff: https://reviews.apache.org/r/36434/diff/ > > > Testing > ------- > > Added unit test cases for testing > - query translation > - result set translation > - Scrolling > > > Thanks, > > Amruth Sampath > >
