Setting up MainfoldCF in Eclipse
My Name is Naveen and i work for searchblox. i would like to contribute to ManifoldCF connectors. Since i am new to ManifoldCF i would like to know some basic things. I read the documentation and i was not able to find How to setup ManifoldCF for development in eclipse. Please guide me or point me to some tutorials. Thanks and Regards, Naveen A.N
Re: Setting up MainfoldCF in Eclipse
Hi Naveen, I don't believe we have a checked-in eclipse settings file anywhere, but a number of developers use it with ManifoldCF. I am not sure how they do this, though. I suspect they are building via eclipse using the Maven build, which while supported is definitely not very close to the official ant build. Karl On Mon, Jun 1, 2015 at 9:04 AM, Naveen A.N anav...@searchblox.com wrote: My Name is Naveen and i work for searchblox. i would like to contribute to ManifoldCF connectors. Since i am new to ManifoldCF i would like to know some basic things. I read the documentation and i was not able to find How to setup ManifoldCF for development in eclipse. Please guide me or point me to some tutorials. Thanks and Regards, Naveen A.N
[jira] [Commented] (CONNECTORS-1162) Apache Kafka Output Connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567847#comment-14567847 ] Tugba Dogan commented on CONNECTORS-1162: - Hi, I have just created a module for kafka connector. Now, I'm planning to install my own Kafka instance and start working on it. You can review the commit if you would like: https://github.com/tugbadogan/manifoldcf/commit/9ab74719083abfc9a5ec9884efdd30e730ad84ac Apache Kafka Output Connector - Key: CONNECTORS-1162 URL: https://issues.apache.org/jira/browse/CONNECTORS-1162 Project: ManifoldCF Issue Type: Wish Affects Versions: ManifoldCF 1.8.1, ManifoldCF 2.0.1 Reporter: Rafa Haro Assignee: Karl Wright Labels: gsoc, gsoc2015 Fix For: ManifoldCF 1.10, ManifoldCF 2.2 Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design. A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients. Apache Kafka is being used for a number of uses cases. One of them is to use Kafka as a feeding system for streaming BigData processes, both in Apache Spark or Hadoop environment. A Kafka output connector could be used for streaming or dispatching crawled documents or metadata and put them in a BigData processing pipeline -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CONNECTORS-1191) ManifoldCFException: Unexpected job status encountered
[
https://issues.apache.org/jira/browse/CONNECTORS-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arcadius Ahouansou updated CONNECTORS-1191:
---
Attachment: unexpected-jobqueue.png
stuffer-thread-manifoldcf.log
Hi [~kwri...@metacarta.com]
Please stuffer-thread-manifoldcf.log is the full log copied from the server.
It does not seem to contain any error, however, the job status screen shows
some jobqueue status errors
ManifoldCFException: Unexpected job status encountered
--
Key: CONNECTORS-1191
URL: https://issues.apache.org/jira/browse/CONNECTORS-1191
Project: ManifoldCF
Issue Type: Bug
Components: Web connector
Affects Versions: ManifoldCF 2.0.2
Environment: - Debian 7.8 x86_64 GNU/Linux
- Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
- Server version: 5.5.41-0+wheezy1 (Debian)
Reporter: Arcadius Ahouansou
Assignee: Karl Wright
Priority: Critical
Fix For: ManifoldCF 1.9, ManifoldCF 2.1
Attachments: CONNECTORS-1191-2.patch, CONNECTORS-1191.patch,
manifoldcf2.0.2.log, stuffer-thread-manifoldcf.log, unexpected-jobqueue.png
Hello.
I am running the latest ManifoldCF 2.0.2 and my log is filled of
{code}
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected job
status encountered: 34
at
org.apache.manifoldcf.crawler.jobs.Jobs.returnJobToActive(Jobs.java:2073)
at
org.apache.manifoldcf.crawler.jobs.JobManager.resetJobs(JobManager.java:8261)
at
org.apache.manifoldcf.crawler.system.JobResetThread.run(JobResetThread.java:91)
{code}
I have attached full log for more detail.
Note that I am running against MySQL.
Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CONNECTORS-1191) ManifoldCFException: Unexpected job status encountered
[
https://issues.apache.org/jira/browse/CONNECTORS-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567982#comment-14567982
]
Karl Wright commented on CONNECTORS-1191:
-
Hi Arcadius,
The log contains absolutely no log data for worker threads. What happened to
it? It also seems to cover a very short time period from today. Perhaps the
logs rolled?
ManifoldCFException: Unexpected job status encountered
--
Key: CONNECTORS-1191
URL: https://issues.apache.org/jira/browse/CONNECTORS-1191
Project: ManifoldCF
Issue Type: Bug
Components: Web connector
Affects Versions: ManifoldCF 2.0.2
Environment: - Debian 7.8 x86_64 GNU/Linux
- Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
- Server version: 5.5.41-0+wheezy1 (Debian)
Reporter: Arcadius Ahouansou
Assignee: Karl Wright
Priority: Critical
Fix For: ManifoldCF 1.9, ManifoldCF 2.1
Attachments: CONNECTORS-1191-2.patch, CONNECTORS-1191.patch,
manifoldcf2.0.2.log, stuffer-thread-manifoldcf.log, unexpected-jobqueue.png
Hello.
I am running the latest ManifoldCF 2.0.2 and my log is filled of
{code}
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected job
status encountered: 34
at
org.apache.manifoldcf.crawler.jobs.Jobs.returnJobToActive(Jobs.java:2073)
at
org.apache.manifoldcf.crawler.jobs.JobManager.resetJobs(JobManager.java:8261)
at
org.apache.manifoldcf.crawler.system.JobResetThread.run(JobResetThread.java:91)
{code}
I have attached full log for more detail.
Note that I am running against MySQL.
Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CONNECTORS-1191) ManifoldCFException: Unexpected job status encountered
[
https://issues.apache.org/jira/browse/CONNECTORS-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568080#comment-14568080
]
Arcadius Ahouansou commented on CONNECTORS-1191:
Hi [~kwri...@metacarta.com], exactly, log rotated :-)
I do have a manifoldcf.log.1, but that content is almost the same as the one
posted.
Seems like I have lost the interesting piece as by default, the
RollingFIleAppender will keep only 1 rolled log.
I have changed logging.properties to keep 10 rolled log. Hopefully, that would
be enough.
Thanks.
Arcadius.
ManifoldCFException: Unexpected job status encountered
--
Key: CONNECTORS-1191
URL: https://issues.apache.org/jira/browse/CONNECTORS-1191
Project: ManifoldCF
Issue Type: Bug
Components: Web connector
Affects Versions: ManifoldCF 2.0.2
Environment: - Debian 7.8 x86_64 GNU/Linux
- Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
- Server version: 5.5.41-0+wheezy1 (Debian)
Reporter: Arcadius Ahouansou
Assignee: Karl Wright
Priority: Critical
Fix For: ManifoldCF 1.9, ManifoldCF 2.1
Attachments: CONNECTORS-1191-2.patch, CONNECTORS-1191.patch,
manifoldcf2.0.2.log, stuffer-thread-manifoldcf.log, unexpected-jobqueue.png
Hello.
I am running the latest ManifoldCF 2.0.2 and my log is filled of
{code}
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected job
status encountered: 34
at
org.apache.manifoldcf.crawler.jobs.Jobs.returnJobToActive(Jobs.java:2073)
at
org.apache.manifoldcf.crawler.jobs.JobManager.resetJobs(JobManager.java:8261)
at
org.apache.manifoldcf.crawler.system.JobResetThread.run(JobResetThread.java:91)
{code}
I have attached full log for more detail.
Note that I am running against MySQL.
Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CONNECTORS-1119) Notifications on job completion
[ https://issues.apache.org/jira/browse/CONNECTORS-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568179#comment-14568179 ] colin commented on CONNECTORS-1119: --- Hi Karl, This looks great - question - this sends a pre-configured message. Can the job status be included? Thanks, Colin Notifications on job completion --- Key: CONNECTORS-1119 URL: https://issues.apache.org/jira/browse/CONNECTORS-1119 Project: ManifoldCF Issue Type: Wish Reporter: colin Assignee: Karl Wright Fix For: ManifoldCF 1.9, ManifoldCF 2.1 Currently the status of jobs can be checked via the rest API or the crawler ui. However there doesn't appear to be a way to send alerts based on jobs completing successfully or otherwise. Is there some way of adding an e-mail distribution I may have missed? or is it a matter of overriding the noteJobComplete method in the output connector? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CONNECTORS-1119) Notifications on job completion
[ https://issues.apache.org/jira/browse/CONNECTORS-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568179#comment-14568179 ] colin edited comment on CONNECTORS-1119 at 6/1/15 10:50 PM: Hi Karl, This looks great - question - this sends a pre-configured message. Can the job completion status be included? e.g. success / failure Thanks, Colin was (Author: thomasobrian): Hi Karl, This looks great - question - this sends a pre-configured message. Can the job status be included? Thanks, Colin Notifications on job completion --- Key: CONNECTORS-1119 URL: https://issues.apache.org/jira/browse/CONNECTORS-1119 Project: ManifoldCF Issue Type: Wish Reporter: colin Assignee: Karl Wright Fix For: ManifoldCF 1.9, ManifoldCF 2.1 Currently the status of jobs can be checked via the rest API or the crawler ui. However there doesn't appear to be a way to send alerts based on jobs completing successfully or otherwise. Is there some way of adding an e-mail distribution I may have missed? or is it a matter of overriding the noteJobComplete method in the output connector? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CONNECTORS-1207) Continuous crawl in filesystem give NPE while deleting some files from the directory
Naveen.A.N created CONNECTORS-1207: -- Summary: Continuous crawl in filesystem give NPE while deleting some files from the directory Key: CONNECTORS-1207 URL: https://issues.apache.org/jira/browse/CONNECTORS-1207 Project: ManifoldCF Issue Type: Bug Components: SearchBlox connector Affects Versions: ManifoldCF 1.9 Reporter: Naveen.A.N On a continuous crawl for File system if some of the files are deleted from the directory the documents are not deleted from searchblox. There was a error in the manifoldcf.log FATAL 2015-06-01 23:24:48,404 (Worker thread '29') - Error tossed: null java.lang.NullPointerException -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [jira] [Created] (CONNECTORS-1207) Continuous crawl in filesystem give NPE while deleting some files from the directory
I don’t see any issues related to Manifoldcf or any other exceptions in searchblox log. Naveen. On Jun 2, 2015, at 10:50 AM, Will Martin wmar...@synthostech.com wrote: Can you provide the searchblox log? -Original Message- From: Naveen.A.N (JIRA) [mailto:j...@apache.org] Sent: Tuesday, June 02, 2015 1:18 AM To: dev@manifoldcf.apache.org Subject: [jira] [Created] (CONNECTORS-1207) Continuous crawl in filesystem give NPE while deleting some files from the directory Naveen.A.N created CONNECTORS-1207: -- Summary: Continuous crawl in filesystem give NPE while deleting some files from the directory Key: CONNECTORS-1207 URL: https://issues.apache.org/jira/browse/CONNECTORS-1207 Project: ManifoldCF Issue Type: Bug Components: SearchBlox connector Affects Versions: ManifoldCF 1.9 Reporter: Naveen.A.N On a continuous crawl for File system if some of the files are deleted from the directory the documents are not deleted from searchblox. There was a error in the manifoldcf.log FATAL 2015-06-01 23:24:48,404 (Worker thread '29') - Error tossed: null java.lang.NullPointerException -- This message was sent by Atlassian JIRA (v6.3.4#6332)
RE: [jira] [Created] (CONNECTORS-1207) Continuous crawl in filesystem give NPE while deleting some files from the directory
Can you provide the searchblox log? -Original Message- From: Naveen.A.N (JIRA) [mailto:j...@apache.org] Sent: Tuesday, June 02, 2015 1:18 AM To: dev@manifoldcf.apache.org Subject: [jira] [Created] (CONNECTORS-1207) Continuous crawl in filesystem give NPE while deleting some files from the directory Naveen.A.N created CONNECTORS-1207: -- Summary: Continuous crawl in filesystem give NPE while deleting some files from the directory Key: CONNECTORS-1207 URL: https://issues.apache.org/jira/browse/CONNECTORS-1207 Project: ManifoldCF Issue Type: Bug Components: SearchBlox connector Affects Versions: ManifoldCF 1.9 Reporter: Naveen.A.N On a continuous crawl for File system if some of the files are deleted from the directory the documents are not deleted from searchblox. There was a error in the manifoldcf.log FATAL 2015-06-01 23:24:48,404 (Worker thread '29') - Error tossed: null java.lang.NullPointerException -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Setting up MainfoldCF in Eclipse
Hi Naveen, As per the chat we had you can use pom to setup your project in IDE. Then configure your tomcat to remote debug. Then you can debug and find you issue and fix it accordingly. Then AFAIK you can create a jira and provide your fix as a patch. Thanks, Tishan On Mon, Jun 1, 2015 at 6:21 AM, Karl Wright daddy...@gmail.com wrote: Hi Naveen, I don't believe we have a checked-in eclipse settings file anywhere, but a number of developers use it with ManifoldCF. I am not sure how they do this, though. I suspect they are building via eclipse using the Maven build, which while supported is definitely not very close to the official ant build. Karl On Mon, Jun 1, 2015 at 9:04 AM, Naveen A.N anav...@searchblox.com wrote: My Name is Naveen and i work for searchblox. i would like to contribute to ManifoldCF connectors. Since i am new to ManifoldCF i would like to know some basic things. I read the documentation and i was not able to find How to setup ManifoldCF for development in eclipse. Please guide me or point me to some tutorials. Thanks and Regards, Naveen A.N -- Regards, Tishan
[jira] [Commented] (CONNECTORS-1119) Notifications on job completion
[ https://issues.apache.org/jira/browse/CONNECTORS-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568327#comment-14568327 ] Karl Wright commented on CONNECTORS-1119: - There are quite a number of reasons for a job stopping. Here are few of them: - Aborted due to error - Aborted due to manual command - Finished successfully - Paused due to manual command - Stopped due to leaving a schedule window At the moment, a notification connector distinguishes only between successful completion, and all other possibilities. Adding more power to the abstraction and to the connector is possible but will require significant connector API changes. You will want to open a new ticket, and I'll try to get to it before the next release in August. Notifications on job completion --- Key: CONNECTORS-1119 URL: https://issues.apache.org/jira/browse/CONNECTORS-1119 Project: ManifoldCF Issue Type: Wish Reporter: colin Assignee: Karl Wright Fix For: ManifoldCF 1.9, ManifoldCF 2.1 Currently the status of jobs can be checked via the rest API or the crawler ui. However there doesn't appear to be a way to send alerts based on jobs completing successfully or otherwise. Is there some way of adding an e-mail distribution I may have missed? or is it a matter of overriding the noteJobComplete method in the output connector? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
