[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767116#comment-16767116
]
Markus Schuch commented on CONNECTORS-1581:
---
Was not able to reproduce. Everything was cleaned up properly after deleting
multiple jobs with >1.000.000 records, even when i poked around with database,
database connection, the manifoldcf process and the outputrepo.
It is possible, that the error occured while using an older version of the
mariadb jdbc driver (which we updated frequently in our setup), but since lack
of time i won't invest to explore this further.
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Assignee: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766037#comment-16766037
]
Karl Wright commented on CONNECTORS-1581:
-
It's possible that the problem is due to funkiness in MySQL. We've had a lot
of trouble lately because MySQL no longer seems to be properly enforcing
transaction integrity in at least some circumstances. OR it could be the
open-source MySQL driver we're using; maybe that needs an upgrade?
At any rate, removal of jobqueue rows MUST precede removal of job table rows;
there's a constraint in place in fact. So if you get to the point where that
constraint has been violated, you're pretty certain it's a database issue. :-(
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Assignee: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766020#comment-16766020
]
Markus Schuch commented on CONNECTORS-1581:
---
I checked for related data in all other tables with a jobid relation and found
no other records.
After deleting the records from the jobqueue the error disappeared.
Now i am testing to reproduce the situation by
- restarting / killing manifoldcf during cleanup
- restarting / killing output repository connection during cleanup
- disconnecting the database during cleanup
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Assignee: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765186#comment-16765186
]
Markus Schuch commented on CONNECTORS-1581:
---
{quote}How did you wind up in that situation though?{quote}
I do not know. We definetly did not temper with the database manually.
I will do some tests by removing some jobs within our setup to see if i can
reprocuce this.
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Assignee: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765178#comment-16765178
]
Karl Wright commented on CONNECTORS-1581:
-
Yes if the job ID doesn't show up anywhere it's safe to delete.
How did you wind up in that situation though?
Karl
On Mon, Feb 11, 2019 at 12:15 PM Markus Schuch (JIRA)
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Assignee: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765173#comment-16765173
]
Markus Schuch commented on CONNECTORS-1581:
---
{code}
select jobid, count(jobid), jobs.description from jobqueue left outer join jobs
on jobs.id = jobqueue.jobid group by jobid;
{code}
actually returns 1 row with documents not having a existing job.
These items belong to a job that we deleted some time ago. For some reason the
cleanup did not work properly.
[~kwri...@metacarta.com] Is it safe to delete those rows from the jobqeue table?
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Assignee: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763667#comment-16763667
]
Karl Wright commented on CONNECTORS-1581:
-
I am pretty concerned that the database layer is fundamentally not working
right. The fact that the set priority thread recovers argues that there is a
database failure that silently resolves. This is bizarre.
If the thread actually finally starts, then you should be good to go other than
the concerns expressed above.
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763567#comment-16763567
]
Markus Schuch commented on CONNECTORS-1581:
---
JDBC driver is mariadb javaclient 2.3.0.
I will enable {{org.apache.manifoldcf.db}} debug logging to see which job ids
are involved.
What is the impact of this error happening in the "set priority thread"?
The exception is logged several times but then it stops and startup seems to
complete without further problems.
We didn't see any problems so far.
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1581) [Set priority thread] Error tossed: null during startup
[
https://issues.apache.org/jira/browse/CONNECTORS-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763530#comment-16763530
]
Karl Wright commented on CONNECTORS-1581:
-
Here's the code that's throwing an NPE:
{code}
// Compute the list of connector instances we will need.
// This has a side effect of fetching all job descriptions too.
Set connectionNames = new HashSet();
for (int i = 0; i < descs.length; i++)
{
DocumentDescription dd = descs[i];
IJobDescription job = jobDescriptionMap.get(dd.getJobID());
if (job == null)
{
job = jobManager.load(dd.getJobID(),true);
jobDescriptionMap.put(dd.getJobID(),job);
}
connectionNames.add(job.getConnectionName());
}
{code}
The problem is, apparently, that jobManager.load() is coming back null.
I have no idea why this would happen but clearly the problem has to do with the
database implementation, perhaps the mysql driver being used?
> [Set priority thread] Error tossed: null during startup
> ---
>
> Key: CONNECTORS-1581
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1581
> Project: ManifoldCF
> Issue Type: Bug
> Environment: •ManifoldCF 2.12, running in a Docker Container
> based on Redhat Linux, OpenJDK 8
> • AWS RDS Database (Aurora MySQL -> 5.6 compatible, utf8 (collation
> utf8_bin))
> • Single Process Setup
>Reporter: Markus Schuch
>Priority: Major
>
> We see the following {{NullPointerException}} at startup:
> {code}
> [Set priority thread] FATAL org.apache.manifoldcf.crawlerthreads- Error
> tossed: null
> java.lang.NullPointerException
> at
> org.apache.manifoldcf.crawler.system.ManifoldCF.writeDocumentPriorities(ManifoldCF.java:1202)
> at
> org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:141)
> {code}
> What could be the cause of that?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
