Steven Jacobs created ASTERIXDB-1926: ----------------------------------------
Summary: Buffer Cache error after 7 hours of Feed Ingestion Key: ASTERIXDB-1926 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1926 Project: Apache AsterixDB Issue Type: Bug Reporter: Steven Jacobs I have a single node, single partition cluster on one machine, and a script sending artificial tweets as fast as possible to a socket feed from a second machine. On the Asterix machine, I have a script performing a single record lookup every second. In both scripts, the key is randomly chosen, so in the read case the key may not exist, and in the write case, duplicate keys are being inserted. I'm using an upsert feed with no secondary keys or filters created. After 7 hours, all queries start to fail with the following (this happens every time for me): May 26, 2017 5:17:38 PM org.apache.asterix.api.http.server.RestApiServlet doHandle SEVERE: Unable to find free page in buffer cache after 1000 cycles (buffer cache undersized?) org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: Unable to find free page in buffer cache after 1000 cycles (buffer cache undersized?) at org.apache.asterix.metadata.declared.MetadataManagerUtil.getDatasetIndexes(MetadataManagerUtil.java:159) at org.apache.asterix.metadata.declared.MetadataProvider.getDatasetIndexes(MetadataProvider.java:364) at org.apache.asterix.optimizer.rules.PushFieldAccessRule.isAccessToIndexedField(PushFieldAccessRule.java:148) at org.apache.asterix.optimizer.rules.PushFieldAccessRule.propagateFieldAccessRec(PushFieldAccessRule.java:185) at org.apache.asterix.optimizer.rules.PushFieldAccessRule.rewritePost(PushFieldAccessRule.java:96) at org.apache.hyracks.algebricks.core.rewriter.base.AbstractRuleController.rewriteOperatorRef(AbstractRuleController.java:126) at org.apache.hyracks.algebricks.core.rewriter.base.AbstractRuleController.rewriteOperatorRef(AbstractRuleController.java:100) at org.apache.hyracks.algebricks.core.rewriter.base.AbstractRuleController.rewriteOperatorRef(AbstractRuleController.java:100) at org.apache.hyracks.algebricks.compiler.rewriter.rulecontrollers.SequentialFixpointRuleController.rewriteWithRuleCollection(SequentialFixpointRuleController.java:53) at org.apache.hyracks.algebricks.core.rewriter.base.HeuristicOptimizer.runOptimizationSets(HeuristicOptimizer.java:102) at org.apache.hyracks.algebricks.core.rewriter.base.HeuristicOptimizer.optimize(HeuristicOptimizer.java:82) at org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.optimize(HeuristicCompilerFactoryBuilder.java:90) at org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:263) at org.apache.asterix.app.translator.QueryTranslator.rewriteCompileQuery(QueryTranslator.java:1818) at org.apache.asterix.app.translator.QueryTranslator.lambda$handleQuery$2(QueryTranslator.java:2312) at org.apache.asterix.app.translator.QueryTranslator.createAndRunJob(QueryTranslator.java:2402) at org.apache.asterix.app.translator.QueryTranslator.deliverResult(QueryTranslator.java:2348) at org.apache.asterix.app.translator.QueryTranslator.handleQuery(QueryTranslator.java:2324) at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:362) at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:246) at org.apache.asterix.api.http.server.RestApiServlet.doHandle(RestApiServlet.java:206) at org.apache.asterix.api.http.server.RestApiServlet.getOrPost(RestApiServlet.java:176) at org.apache.asterix.api.http.server.RestApiServlet.get(RestApiServlet.java:160) at org.apache.hyracks.http.server.AbstractServlet.handle(AbstractServlet.java:74) at org.apache.hyracks.http.server.HttpRequestHandler.handle(HttpRequestHandler.java:70) at org.apache.hyracks.http.server.HttpRequestHandler.call(HttpRequestHandler.java:55) at org.apache.hyracks.http.server.HttpRequestHandler.call(HttpRequestHandler.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Here are the DDL's and the query: drop dataverse experiments if exists; create dataverse experiments; use dataverse experiments; create type TweetMessageType as closed { id: int64, message_text: string, created_at: string, country:string } create feed TweetFeed using socket_adapter ( ("sockets"="127.0.0.1:10001"), ("address-type"="IP"), ("type-name"="TweetMessageType"), ("upsert-feed"="true"), ("format"="adm") ); connect feed TweetFeed to dataset Tweets1; start feed TweetFeed; Query run every second: use dataverse experiments; for $i in dataset Tweets1 where $i.id = <random_int> return $i.country; -- This message was sent by Atlassian JIRA (v6.3.15#6346)