Repository: zeppelin Updated Branches: refs/heads/master 709c5a70a -> 87705a932
[ZEPPELIN-1470] limiting results from jdbc ### What is this PR for? One thing we tracked down is that if you issue a large query on a very large table, it will simply try to load all results (and then cap them on Zeppelin's side), which seems suboptimal (and will freeze the server). Setting this on the JDBC level seems to solve the problem. ### What type of PR is it? Bug Fix ### Todos * [x] Tests ### How should this be tested? - Create or use a table with a very large number of rows. In our tests, I simply created a: ``` createdb zeppelin_test psql zeppelin_test create table too_many_rows(n int) ``` And added 5m rows to it. Making a paragraph like this will hang without setting a limit: ``` %zeppelin_test select * from too_many_rows ``` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Herval Freire <hfre...@twitter.com> Closes #2428 from herval/hfreire/limit-row-count and squashes the following commits: 4f66469 [Herval Freire] display truncation message b538c44 [Herval Freire] limiting results from jdbc Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/87705a93 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/87705a93 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/87705a93 Branch: refs/heads/master Commit: 87705a93244d24448cbd147f63263900241b4b2b Parents: 709c5a7 Author: Herval Freire <hfre...@twitter.com> Authored: Thu Jun 29 12:39:07 2017 -0700 Committer: Felix Cheung <felixche...@apache.org> Committed: Mon Jul 17 00:52:03 2017 -0700 ---------------------------------------------------------------------- .../src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java | 5 +++++ 1 file changed, 5 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/87705a93/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java ---------------------------------------------------------------------- diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java index 3483b4b..f3f4326 100644 --- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java +++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java @@ -683,6 +683,11 @@ public class JDBCInterpreter extends KerberosInterpreter { for (int i = 0; i < sqlArray.size(); i++) { String sqlToExecute = sqlArray.get(i); statement = connection.createStatement(); + + // fetch n+1 rows in order to indicate there's more rows available (for large selects) + statement.setFetchSize(getMaxResult()); + statement.setMaxRows(getMaxResult() + 1); + if (statement == null) { return new InterpreterResult(Code.ERROR, "Prefix not found."); }
