Hi All,
I am facing below error while accessing Manifoldcf. Requirement is to crawl
data from a website using Repository as "Web" and Output connector as
"Elastic Search"
Manifoldcf is configured inside a docker container and also postgres is
used a docker container.
When launching manifold getting below error
[image: image.png]
When checked logs:-
*1)sudo docker exec -it 0b872dfafc5c tail -1000
/usr/share/manifoldcf/example/logs/manifoldcf.log*
FATAL 2019-12-20T06:06:13,176 (Stuffer thread) - Error tossed: Timer
already cancelled.
java.lang.IllegalStateException: Timer already cancelled.
at java.util.Timer.sched(Timer.java:397) ~[?:1.8.0_232]
at java.util.Timer.schedule(Timer.java:193) ~[?:1.8.0_232]
at
org.postgresql.jdbc.PgConnection.addTimerTask(PgConnection.java:1113)
~[postgresql-42.1.3.jar:42.1.3]
at org.postgresql.jdbc.PgStatement.startTimer(PgStatement.java:887)
~[postgresql-42.1.3.jar:42.1.3]
at
org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:427)
~[postgresql-42.1.3.jar:42.1.3]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
~[postgresql-42.1.3.jar:42.1.3]
at
org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169)
~[postgresql-42.1.3.jar:42.1.3]
at
org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:136)
~[postgresql-42.1.3.jar:42.1.3]
at org.postgresql.jdbc.PgConnection.isValid(PgConnection.java:1311)
~[postgresql-42.1.3.jar:42.1.3]
at
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:92)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:126)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:75)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:797)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1457)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:146)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:204)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:837)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseTable.java:221)
~[mcf-core.jar:?]
at
org.apache.manifoldcf.crawler.jobs.Jobs.getActiveJobConnections(Jobs.java:736)
~[mcf-pull-agent.jar:?]
at
org.apache.manifoldcf.crawler.jobs.JobManager.getNextDocuments(JobManager.java:2869)
~[mcf-pull-agent.jar:?]
at
org.apache.manifoldcf.crawler.system.StufferThread.run(StufferThread.java:186)
[mcf-pull-agent.jar:?]
*2)sudo docker logs <CID> --tail 1000*
Exception in thread "PostgreSQL-JDBC-SharedTimer-1"
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.ArrayList.iterator(ArrayList.java:840)
at
java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1316)
at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
at java.net.InetAddress.getAllByName(InetAddress.java:1193)
at java.net.InetAddress.getAllByName(InetAddress.java:1127)
at java.net.InetAddress.getByName(InetAddress.java:1077)
at java.net.InetSocketAddress.<init>(InetSocketAddress.java:220)
at org.postgresql.core.PGStream.<init>(PGStream.java:66)
at
org.postgresql.core.QueryExecutorBase.sendQueryCancel(QueryExecutorBase.java:155)
at
org.postgresql.jdbc.PgConnection.cancelQuery(PgConnection.java:971)
at org.postgresql.jdbc.PgStatement.cancel(PgStatement.java:812)
at org.postgresql.jdbc.PgStatement$1.run(PgStatement.java:880)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
2019-12-19 18:09:05,848 Job start thread ERROR Unable to write to stream
logs/manifoldcf.log for appender MyFile
2019-12-19 18:09:05,848 Seeding thread ERROR Unable to write to stream
logs/manifoldcf.log for appender MyFile
2019-12-19 18:09:05,848 Job reset thread ERROR Unable to write to stream
logs/manifoldcf.log for appender MyFile
2019-12-19 18:09:05,848 Job notification thread ERROR Unable to write to
stream logs/manifoldcf.log for appender MyFile
2019-12-19 18:09:05,849 Seeding thread ERROR An exception occurred
processing Appender MyFile
org.apache.logging.log4j.core.appender.AppenderLoggingException: Error
flushing stream logs/manifoldcf.log
at
org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:159).
*Also tried the approach to clean up Database by truncating all manifoldcf
related tables, but still getting this error.*
Parameters defined in *postgresql conf *file is as suggested :- and
"max_pred_per_locks_transctions" is set to value "256".
[image: image.png]
