Yes, of course... I have opened a JIRA [1] and start working on it. [1] https://issues.apache.org/jira/browse/CAMEL-7700
Best, Christian ----------------- Software Integration Specialist Apache Member V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer Apache Incubator PMC Member https://www.linkedin.com/pub/christian-mueller/11/551/642 On Thu, Aug 14, 2014 at 3:22 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: > Hi > > IMHO you should only set that fetch limit if getMaxMessagesPerPoll > > 0. So the user needs to set that option explicit if he/she want to > limit it. > > > > On Wed, Aug 13, 2014 at 2:07 PM, Christian Müller > <christian.muel...@gmail.com> wrote: > > At present, we do not limit the fetch size on the JDBC driver level which > > will fix this issue (e.g. > > preparedStatement.setFetchSize(getMaxMessagesPerPoll())). > > The drawback is, we don't know how many rows are in the database which > will > > match our query. We only can assume if fetchSize is equal to > > maxMessagesPerPoll, there are more rows to read. > > > > From my point of view, we should go for this change. If nobody has > > objections, I will will do the needed changes later this week... > > > > Best, > > > > Christian > > ----------------- > > > > Software Integration Specialist > > > > Apache Member > > V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer > > Apache Incubator PMC Member > > > > https://www.linkedin.com/pub/christian-mueller/11/551/642 > > > > > > On Fri, Aug 8, 2014 at 10:34 PM, Matt Payne <pa...@mattpayne.org> wrote: > > > >> I am getting an java.lang.OutOfMemoryError: Java heap space when using > >> camel sql component with a query that returns a large number of rows. > >> > >> Using a small value for maxMessagesPerPoll=10 does not help[1]. > >> > >> When reading the source[2], I see: > >> 255 protected List<Map<String, Object>> queryForList(ResultSet rs) > throws > >> SQLException { > >> 256 ColumnMapRowMapper rowMapper = new ColumnMapRowMapper(); > >> 257 RowMapperResultSetExtractor<Map<String, Object>> mapper = > new > >> RowMapperResultSetExtractor<Map<String, Object>>(rowMapper); > >> 258 List<Map<String, Object>> data = mapper.extractData(rs); > >> 259 return data; > >> 260 } > >> > >> It seems that all of the result set, rs, is going to be read regardless > of > >> how large it is. When stepping through via eclipse's debugger this is > >> what I see happening. > >> > >> It's unclear how to ask camel sql component to only take X rows at a > time. > >> Is there a way to do this please? > >> > >> Thanks! --Matt Payne > >> > >> > >> [1] > >> > http://camel.465427.n5.nabble.com/Fetching-data-in-batches-td5718366.html > >> [2] > >> > >> > https://git-wip-us.apache.org/repos/asf?p=camel.git;a=blob;f=components/camel-sql/src/main/java/org/apache/camel/component/sql/SqlEndpoint.java#l255 > >> > > > > -- > Claus Ibsen > ----------------- > Red Hat, Inc. > Email: cib...@redhat.com > Twitter: davsclaus > Blog: http://davsclaus.com > Author of Camel in Action: http://www.manning.com/ibsen > hawtio: http://hawt.io/ > fabric8: http://fabric8.io/ >
