----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42067/ -----------------------------------------------------------
(Updated Feb. 4, 2016, 2:21 a.m.) Review request for lens. Changes ------- Merged agin with master to fix "Patch does not apply. Build job: https://builds.apache.org/job/PreCommit-Lens-Build/266/"; Bugs: lens-901 https://issues.apache.org/jira/browse/lens-901 Repository: lens Description ------- support streaming results on lens Diffs (updated) ----- lens-api/src/main/java/org/apache/lens/api/query/QueryHandleWithResultSet.java a5da867 lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 915dac7 lens-driver-es/src/main/java/org/apache/lens/driver/es/client/ESResultSet.java 464b535 lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 149c6ab lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveInMemoryResultSet.java f8abd78 lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 82d7513 lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCResultSet.java 8b4da3f lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java b96cf88 lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8df389b lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java ed1fc43 lens-server-api/src/main/java/org/apache/lens/server/api/driver/InMemoryResultSet.java 0d64471 lens-server-api/src/main/java/org/apache/lens/server/api/driver/PartiallyFetchedInMemoryResultSet.java PRE-CREATION lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java 1269e45 lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java a20cf47 lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java e61398b lens-server/src/main/resources/lenssession-default.xml a321c3f lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java 4597f9d lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java d9b7679 lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java 6db990e src/site/apt/admin/session-config.apt e108a13 Diff: https://reviews.apache.org/r/42067/diff/ Testing ------- Three properties have been added to enable streaming * lens.query.prefetch.inmemory.resultset When set to true, specified number of rows of in-memory result set will be pre-fetched * lens.query.prefetch.inmemory.resultset.rows Specifies the number of rows to pre-fetch for in-memory result set when lens.query.prefetch.inmemory.resultset is set to true. Default value is 100 rows. * lens.query.prefetch.inmemory.resultset.ttl.millis Specifies the time in milli seconds starting from query submission time for which the pre-fetched in-memory result set will not be purged. This property is put to action only when lens.query.prefetch.inmemory.resultset is set to true and all rows of the in memory result set have been pre-fetched. The default value of this property is 60 secs. Streaming(pre-fetched) result set is available in case the above proerties are set and the result is a type of InMemoryResultSet. Further, the streming is enabled only when all result rows have been prefetched.Partial Streaming not supported as of now (Can be enabled in furtue is there is a clear use case for it) Also note QueryHandleWithResultSet will now hold result metadta (QueryResultSetMetadata) also. This info will be useful for client in case the streaming results are accesesd via execute_with_timeout API (REST End Point /queryapi/queries POST ) It is suggested to use this feature for queries that finish fast and return small number of rows. Streaming and persistence by server can happen parallely for streamed result sets. Will add more details on **Testing** done . Was able to build and run checkstyle. Also ran the added test cases for this feature. Thanks, Puneet Gupta
