[jira] [Commented] (CONNECTORS-423) Elastic Search connector should handle document metadata
[ https://issues.apache.org/jira/browse/CONNECTORS-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230332#comment-13230332 ] Piergiorgio Lucidi commented on CONNECTORS-423: --- I created a branch to work on this issue: CONNECTORS-423. Today I finished these tasks: - added the connector-label parameter in the Ant script - added the support for indexing binaries using the Attachment Plugin - added metadata support I have to debug all this stuff and I have to change the integration test module to add the Attachment Plugin in the ElasticSearch instance used during the test. Elastic Search connector should handle document metadata Key: CONNECTORS-423 URL: https://issues.apache.org/jira/browse/CONNECTORS-423 Project: ManifoldCF Issue Type: Improvement Components: Elastic Search connector Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Piergiorgio Lucidi Fix For: ManifoldCF 0.5 The Elastic Search connector does not currently handle metadata, but it should. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-424) Add RepositoryDocument.getFileName
[ https://issues.apache.org/jira/browse/CONNECTORS-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229148#comment-13229148 ] Karl Wright commented on CONNECTORS-424: Looks good to me. Have you tested this with non-ASCII file names yet? I think that would be worthwhile before committing. Add RepositoryDocument.getFileName -- Key: CONNECTORS-424 URL: https://issues.apache.org/jira/browse/CONNECTORS-424 Project: ManifoldCF Issue Type: Improvement Components: Framework agents process Affects Versions: ManifoldCF 0.5 Reporter: Shinichiro Abe Priority: Minor Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-424.patch When posting files to Solr via file system connector or share drive connector, the stream_name of metadata is hard-coded docname. I want to set the stream_name a right file name, it is useful for searching documents by file name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-424) Add RepositoryDocument.getFileName
[ https://issues.apache.org/jira/browse/CONNECTORS-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229237#comment-13229237 ] Shinichiro Abe commented on CONNECTORS-424: --- Yes, I've tested CJK character file name at least. These file names are not garbled. I'll commit after checking a little bit. Add RepositoryDocument.getFileName -- Key: CONNECTORS-424 URL: https://issues.apache.org/jira/browse/CONNECTORS-424 Project: ManifoldCF Issue Type: Improvement Components: Framework agents process Affects Versions: ManifoldCF 0.5 Reporter: Shinichiro Abe Priority: Minor Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-424.patch When posting files to Solr via file system connector or share drive connector, the stream_name of metadata is hard-coded docname. I want to set the stream_name a right file name, it is useful for searching documents by file name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-425) Elastic Search documentation missing images
[ https://issues.apache.org/jira/browse/CONNECTORS-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229768#comment-13229768 ] Karl Wright commented on CONNECTORS-425: I disabled the missing images. This is probably OK for the 0.5 release, so I'm bumping down the priority and triaging it for next release. Elastic Search documentation missing images --- Key: CONNECTORS-425 URL: https://issues.apache.org/jira/browse/CONNECTORS-425 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Priority: Blocker Fix For: ManifoldCF next The ElasticSearch documentation refers to images that never actually were committed. Looking back at the ticket, they were never included in any uploaded patch either. These are: [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-connection-parameters.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-user.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-job-parameters.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-history-report.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-connection-parameters.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-user.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-job-parameters.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://en_US/images/en_US/elasticsearch-history-report.PNG. (No context info available) [exec] X [0] ja_JP/images/ja_JP/elasticsearch-connection-parameters_ja_JP.PNG BROKEN: No pipeline matched request: ja_ JP/images/ja_JP/elasticsearch-connection-parameters_ja_JP.PNG [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-connection-parameters_ja_JP.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-user_ja_JP.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-job-parameters_ja_JP.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-history-report_ja_JP.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-connection-parameters_ja_JP.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-user_ja_JP.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-job-parameters_ja_JP.PNG. (No context info available) [exec] ERROR - Image not found. URI: cocoon://ja_JP/images/ja_JP/elasticsearch-history-report_ja_JP.PNG. (No context info available) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13227446#comment-13227446 ] Karl Wright commented on CONNECTORS-288: Luca and I ironed this out. Basically the problem was that the id (the URL-encoded documentURI) needs to be part of the URL for both the index and the delete operation. Once that was done, the test passes. I'm now in the process of merging the branch into trunk. That should be completed this evening, with luck. Once complete, the only outstanding issue I can see is that metadata is not being indexed. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13227517#comment-13227517 ] Piergiorgio Lucidi commented on CONNECTORS-288: --- I agree with Karl, we should merge this branch into trunk and then I think that it should work correctly. Anyway we can fix new issues quickly without any specific issue related to the CMIS server. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13227269#comment-13227269 ] Karl Wright commented on CONNECTORS-288: Hi Luca, An output connector cannot assume anything about a URL. The URL may come from any repository connector, and they all have different forms. For example, file system and SharePoint URLs are basically full paths. You would not want to confuse a/b/cd with x/y/cd, would you? So please don't think that the only kind of URL an Elastic Search connector is ever going to see will be from CMIS. So the resolution of CONNECTORS-417 is immaterial here. The change I made, which instead of using the last part of the URL path as a file name, tries to use the entire URL and encode it, so that the whole thing is interpreted as a file name. I changed the delete code and also the index code to make this consistent. But it did not work, as I have said earlier. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13227272#comment-13227272 ] Luca Stancapiano commented on CONNECTORS-288: - hi karl It's wrong anyway. In that code you append an url to an other url. If the url is: http://localhost:9090/index/generictype/ In that url you append this: http://localhost:8543/chemistry/filename so the final url become: http://localhost:9090/index/generictype/http://localhost:8543/chemistry/filename It has no sense. So the curl operation 'ld become: curl -XDELETE http://localhost:9090/index/generictype/http://localhost:8543/chemistry/filename instead of: curl -XDELETE http://localhost:9090/index/generictype/filename Let me know, please An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226945#comment-13226945
]
Luca Stancapiano commented on CONNECTORS-288:
-
Okdeleting the properties.xml after each test I can now run and fix the
test. The problem of the delete is tied to a last update:
class org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchDelete:row
34
{code}
String fileName = java.net.URLEncoder.encode(documentURI,utf-8);
{code}
it must be a fileNome, not a Uri because it is the id to use for the delete.
The delete method in the elasticsearch connector is as a curl as below:
{code}
curl -XDELETE http://localhost:9200/index/generictype/null
{code}
where gtgt is the 'fileName' variable so it'ld become so:
{code}
curl -XDELETE
http://localhost:9200/index/generictype/http%3A%2F%2Flocalhost%3A9090%2Fchemistry-opencmis-server-inmemory%2Fatom%2F139%2Fnull
{code}
If you modify the row as it was before so:
{code}
String fileName = FilenameUtils.getName(documentURI);
{code}
the delete works.
Done it there is a new problem tied to
https://issues.apache.org/jira/browse/CONNECTORS-417:
our test create multiple versions of the document called 'null'
When the delete is called, the test succeeds to delete the first 'null'
document because the dlete operation delete all versions for the document. So,
after, the 'null' document is not more there but it try to delete anyhow and it
goes in a loop.
I suppose this problem is done because the resolution of
https://issues.apache.org/jira/browse/CONNECTORS-417 is not committed in the
branch. Let me know for it
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-421) Test infrastructure's cleanup methods cause agents process to fail to start
[ https://issues.apache.org/jira/browse/CONNECTORS-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223441#comment-13223441 ] Luca Stancapiano commented on CONNECTORS-421: - we could to move the stopAgentsRun variable from the org.apache.manifoldcf.agents.system.ManifoldCF class to the parent org.apache.manifoldcf.core.system.ManifoldCF class and hereditate it to the child. Let me see Test infrastructure's cleanup methods cause agents process to fail to start --- Key: CONNECTORS-421 URL: https://issues.apache.org/jira/browse/CONNECTORS-421 Project: ManifoldCF Issue Type: Bug Components: Tests Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Priority: Minor Fix For: ManifoldCF 0.5 The testing infrastructure attempts to clean up from a previous run of a test. If properties.xml is left around, however, this cleanup will succeed but prevent the agents process from starting in the test. That's because the addShutdownHook() is called, and when resetEnvironment() is called agentStop() is thus invoked, and once agentStop() is invoked agentStart() will just exit next time it is called. The fix is to make resetEnvironment() reset the agents stop flag so that starting agents will then work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-421) Test infrastructure's cleanup methods cause agents process to fail to start
[ https://issues.apache.org/jira/browse/CONNECTORS-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223447#comment-13223447 ] Luca Stancapiano commented on CONNECTORS-421: - If I reset the variable in the resetEnvironment() simply I get a shutdown and the test never start Test infrastructure's cleanup methods cause agents process to fail to start --- Key: CONNECTORS-421 URL: https://issues.apache.org/jira/browse/CONNECTORS-421 Project: ManifoldCF Issue Type: Bug Components: Tests Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Priority: Minor Fix For: ManifoldCF 0.5 The testing infrastructure attempts to clean up from a previous run of a test. If properties.xml is left around, however, this cleanup will succeed but prevent the agents process from starting in the test. That's because the addShutdownHook() is called, and when resetEnvironment() is called agentStop() is thus invoked, and once agentStop() is invoked agentStart() will just exit next time it is called. The fix is to make resetEnvironment() reset the agents stop flag so that starting agents will then work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221632#comment-13221632
]
Luca Stancapiano commented on CONNECTORS-288:
-
I continue to get the problem of WorkerThread. It never starts so the test
never calls the api of elasticsearch connector. I expose my last running of the
test
In the org.apache.manifoldcf.crawler.tests.BaseITDerby there is:
{code}
@Before
public void setUp()
throws Exception
{
super.setUp();
mcfInstance.start();
}
{code}
The setUp method starts a clean of the services. In this clean is started the
org.apache.manifoldcf.agents.system.ManifoldCF.doCleanup() method:
{code}
public void doCleanup()
throws ManifoldCFException
{
// Shutting down in this way must prevent startup from taking place.
synchronized (runningHash)
{
stopAgentsRun = true;
}
IThreadContext tc = ThreadContextFactory.make();
stopAgents(tc);
}
{code}
since this time the stopAgentsRun variable is true!!
So , when Manifold starts we have this control in the
org.apache.manifoldcf.agents.system.ManifoldCF class:
{code}
public static void startAgents(IThreadContext threadContext)
throws ManifoldCFException
{
// Get agent manager
IAgentManager manager = AgentManagerFactory.make(threadContext);
String[] classes = manager.getAllAgents();
ManifoldCFException problem = null;
synchronized (runningHash)
{
// DO NOT permit this method to do anything if stopAgents() has ever been
called for this JVM!
// (If it has, it means that the JVM is trying to shut down.)
if (stopAgentsRun)
return;
.
{code}
stopAgentsRun is always true so the agents never be launched!!!
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-420) Velocity support should include a model where there is one template for all languages, and tags that you can use for i18n strings
[
https://issues.apache.org/jira/browse/CONNECTORS-420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219970#comment-13219970
]
Karl Wright commented on CONNECTORS-420:
I believe this is part of a broader class of problem in that we really want our
Velocity templates to be able to work with objects, not just strings. This may
include portions of the Configuration and Specification structures. In other
words, I think we need to generalize our use of Velocity so that we can
properly place important objects in the Velocity context. This will
potentially include not only structures that are parts of Configuration and
Specification, but also ResourceBundles that allow internationalization.
The issue then becomes how to handle the context-sensitive rendering of
resources, etc. It is possible to call static class methods from within
Velocity as described here:
http://velocity.apache.org/engine/devel/developer-guide.html
Specifically:
{code}
context.put(Math, Math.class);
{code}
This would then allow us to present the encoder class in this way:
{code}
context.put(Encoder, org.apache.manifoldcf.ui.util.Encoder.class);
{code}
... and the template would look something like:
{code}
input type=hidden name=foo value=$Encoder.attributeEscape($foovalue)/
{code}
... instead of:
{code}
input type=hidden name=foo value=$foovalue_A/
{code}
Velocity support should include a model where there is one template for all
languages, and tags that you can use for i18n strings
-
Key: CONNECTORS-420
URL: https://issues.apache.org/jira/browse/CONNECTORS-420
Project: ManifoldCF
Issue Type: Improvement
Components: Framework core
Affects Versions: ManifoldCF 0.5
Reporter: Karl Wright
Assignee: Karl Wright
Fix For: ManifoldCF 0.5
The current way we use Velocity templates is to have a different template for
each language. That's very wasteful and leads to code duplication. We need
to support a way of having one template for all languages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-420) Velocity support should include a model where there is one template for all languages, and tags that you can use for i18n strings
[ https://issues.apache.org/jira/browse/CONNECTORS-420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13220386#comment-13220386 ] Karl Wright commented on CONNECTORS-420: Just tried it out - works exactly as expected. I'm going to internationalize/localize the editConfiguration.html element of the CMIS connector as an example, and then we can merge the branch back in. Velocity support should include a model where there is one template for all languages, and tags that you can use for i18n strings - Key: CONNECTORS-420 URL: https://issues.apache.org/jira/browse/CONNECTORS-420 Project: ManifoldCF Issue Type: Improvement Components: Framework core Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 The current way we use Velocity templates is to have a different template for each language. That's very wasteful and leads to code duplication. We need to support a way of having one template for all languages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13218282#comment-13218282 ] Piergiorgio Lucidi commented on CONNECTORS-288: --- I added a fix to enable the connector without any index defined in ElasticSearch: we have to discuss if we need the check for the index specified in the configuration or not. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13218349#comment-13218349
]
Piergiorgio Lucidi commented on CONNECTORS-288:
---
{quote}
For example, for the CMIS path
'http://localhost:9090/chemistry-opencmis-server-inmemory/atom/138/null'
(Piergiorgio, what is that 'null' at the end?? Sounds like it may be a bug to
me.), the output connector turns this into filename 'null'.
{quote}
Karl, you're right. I found a bug in the CMIS Connector, that URL is generated
by the CMIS Connector, if there are no version label defined for the node, the
OpenCMIS InMemory Server returns null, that is the last part of the documentURI.
From the CMIS Connector code:
{code}
String version = document.getVersionLabel();
String endpoint = protocol+://+server+:+port+path;
String documentURI = endpoint+/+id+/+version;
activities.ingestDocument(id, version, documentURI, rd);
{code}
That's why we have that null in the documentURI, and maybe it could solve the
problem about the delete feature in the integration test, because there is an
unique filename null that doesn't mean nothing.
So I have to create a new ticket and I have to create a patch for the issue.
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13217490#comment-13217490
]
Piergiorgio Lucidi commented on CONNECTORS-288:
---
I tried to execute the test now and I think that now tests are runned correctly
but it seems that it can't delete the job from Manifold at the end of the test:
{code}
sanityCheck(org.apache.manifoldcf.elasticsearch_tests.APISanityDerbyIT):
ManifoldCF did not delete in the allotted time of 12 milliseconds
sanityCheck(org.apache.manifoldcf.elasticsearch_tests.APISanityDerbyIT):
Can't delete repository connection 'CMIS Connection': existing jobs refer to it
{code}
This is the last part of the test, so we are near to have a complete
integration test implementation.
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13217519#comment-13217519
]
Karl Wright commented on CONNECTORS-288:
bq. I tried to execute the test now and I think that now tests are runned
correctly but it seems that it can't delete the job from Manifold at the end of
the test:
Right, the problem is that the job deletion hangs, because it's trying to
delete the documents from the index and something goes wrong with that. I
posted earlier the manifoldcf.log output associated with this failure:
{code}
ERROR 2012-02-26 16:09:35,903 (Document delete thread '7') - Exception tossed:
Server/page not found
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Server/page not found
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection.call(ElasticSearchConnection.java:111)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchDelete.init(ElasticSearchDelete.java:35)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.removeDocument(ElasticSearchConnector.java:378)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentDeleteMultiple(IncrementalIngester.java:748)
at
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:130)
{code}
The issue is that the Server/page not found error seems to occur
intermittently on many different requests. These are usually retried, but at
the end during the delete phase they wait 5 minutes before being retried, which
is why the test fails. The real problem is that we should not be getting
intermittent random errors at all, which is why I think we need to look at data
that is kept around in the connector from request to request, namely the cached
data structures. I am certain these are the source of the problem.
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216825#comment-13216825 ] Luca Stancapiano commented on CONNECTORS-288: - An other thing that I can note is that the org.apache.manifoldcf.crawler.system.WorkerThread and the org.apache.manifoldcf.crawler.system.StartupThread are not active when the test start. I suppose they support the jobs, when they start An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216827#comment-13216827 ] Karl Wright commented on CONNECTORS-288: From what I can see, the connector IS called, but it just throws an exception when it sets up its session. I can instrument the connector if you like in order to prove this to you. If you want to see this, just browse to localhost:8346/mcf-crawler-ui while the test is running. View the output connection. You will see the exception I've already reported. WorkerThread and StartupThread will not become active until the agents process starts. In a test, this happens during a @Before method. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216830#comment-13216830 ] Karl Wright commented on CONNECTORS-288: Instrumentation yields the following: [junit] org.apache.manifoldcf.core.interfaces.ManifoldCFException: Server/pa ge not found [junit] at org.apache.manifoldcf.agents.output.elasticsearch.ElasticSear chConnection.call(ElasticSearchConnection.java:111) [junit] at org.apache.manifoldcf.agents.output.elasticsearch.ElasticSear chAction.init(ElasticSearchAction.java:37) [junit] at org.apache.manifoldcf.agents.output.elasticsearch.ElasticSear chConnector.check(ElasticSearchConnector.java:389) I'm instrumenting the ElasticSearchAction constructor now to see what URL it thinks it is using. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216840#comment-13216840 ] Karl Wright commented on CONNECTORS-288: So when I added a System.out.println of the URL in ElasticSearchAction, I no longer get any errors; the check() response is OK. (That is wrong, by the way; it should return super.check() instead, which is Connection working.) The instrumented URL output looks like this: [junit] URL is 'http://localhost:9200/index/_optimize' [junit] URL is 'http://localhost:9200/index/_status' [junit] URL is 'http://localhost:9200/index/_optimize' [junit] URL is 'http://localhost:9200/index/_optimize' [junit] URL is 'http://localhost:9200/index/_optimize' ... followed by the 12 ms timeout. Some conclusions: (1) We should fix the check() method; (2) The fact that check() succeeds sometimes and fails others is quite disconcerting; clearly the connector is doing something pretty wrong. I also looked more deeply at the code itself. The addOrReplaceDocument() method uses a synchronizer to permit only one thread to index at a time. This does not seem correct to me, and it is thus probable that the problem stems from improper understanding of the ManifoldCF threading model. Each connector instance should be working with its own ElasticSearchIndex object and its own HttpClient method so that all of the threads can operate independently without collision. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216851#comment-13216851
]
Karl Wright commented on CONNECTORS-288:
Looking at the actual test run, the history reports the following at the end:
{code}
02-26-2012 16:09:25.129 Indexation (ElasticSearch)
http://localhost:9200/index/_optimize
OK 0 69
02-26-2012 16:09:24.939 job end 1330290457146(Test Job)
0 1
02-26-2012 16:09:14.909 Deletion (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/138/null
OK 0 7
02-26-2012 16:09:07.787 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/140/null
OK 27 6
02-26-2012 16:09:07.778 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/139/null
OK 27 7
02-26-2012 16:09:07.769 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/137/null
OK 27 15
02-26-2012 16:09:05.278 job start 1330290457146(Test Job)
0 1
02-26-2012 16:08:55.020 Indexation (ElasticSearch)
http://localhost:9200/index/_optimize
OK 0 93
02-26-2012 16:08:54.926 job end 1330290457146(Test Job)
0 1
02-26-2012 16:08:47.678 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/138/null
OK 27 10
02-26-2012 16:08:47.666 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/140/null
OK 27 6
02-26-2012 16:08:47.652 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/139/null
OK 27 11
02-26-2012 16:08:47.646 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/137/null
OK 27 13
02-26-2012 16:08:45.192 job start 1330290457146(Test Job)
0 1
02-26-2012 16:08:34.940 Indexation (ElasticSearch)
http://localhost:9200/index/_optimize
OK 0 75
02-26-2012 16:08:34.917 job end 1330290457146(Test Job)
0 1
02-26-2012 16:08:29.502 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/139/null
OK 27 10
02-26-2012 16:08:29.491 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/137/null
OK 27 8
02-26-2012 16:08:29.412 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/140/null
OK 27 66
02-26-2012 16:08:29.404 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/138/null
OK 27 68
02-26-2012 16:08:25.097 job start 1330290457146(Test Job)
0 1
02-26-2012 16:08:24.846 Indexation (ElasticSearch)
http://localhost:9200/index/_optimize
OK 0 88
02-26-2012 16:08:14.890 job stop1330290457146(Test Job)
0 1
02-26-2012 16:08:09.041 Optimize (ElasticSearch)
http://localhost:9090/chemistry-opencmis-server-inmemory/atom...
/138/null
OK 27 868
02-26-2012 16:08:04.900 job start 1330290457146(Test Job)
0 1
{code}
The job at the end is stuck in the Cleaning up state, which indicates that it
is trying to delete the documents from the index, but is not succeeding for
some reason.
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216966#comment-13216966
]
Karl Wright commented on CONNECTORS-288:
Just checked the manifoldcf.log file from the test crawl. Here's a snippet:
{code}
ERROR 2012-02-26 16:08:09,921 (Worker thread '4') - Exception tossed:
org.apache.manifoldcf.core.interfaces.ManifoldCFException:
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection.call(ElasticSearchConnection.java:111)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchIndex.init(ElasticSearchIndex.java:100)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.addOrReplaceDocument(ElasticSearchConnector.java:357)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1579)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:504)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:370)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1577)
at
org.apache.manifoldcf.crawler.connectors.cmis.CmisRepositoryConnector.processDocuments(CmisRepositoryConnector.java:1162)
at
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:561)
ERROR 2012-02-26 16:09:35,903 (Document delete thread '7') - Exception tossed:
Server/page not found
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Server/page not found
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection.call(ElasticSearchConnection.java:111)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchDelete.init(ElasticSearchDelete.java:35)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.removeDocument(ElasticSearchConnector.java:378)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentDeleteMultiple(IncrementalIngester.java:748)
at
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:130)
ERROR 2012-02-26 16:09:36,908 (Document delete thread '9') - Exception tossed:
Server/page not found
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Server/page not found
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection.call(ElasticSearchConnection.java:111)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchDelete.init(ElasticSearchDelete.java:35)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.removeDocument(ElasticSearchConnector.java:378)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentDeleteMultiple(IncrementalIngester.java:748)
at
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:130)
ERROR 2012-02-26 16:09:37,907 (Document delete thread '8') - Exception tossed:
Server/page not found
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Server/page not found
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection.call(ElasticSearchConnection.java:111)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchDelete.init(ElasticSearchDelete.java:35)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.removeDocument(ElasticSearchConnector.java:378)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentDeleteMultiple(IncrementalIngester.java:748)
at
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:130)
{code}
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216971#comment-13216971 ] Karl Wright commented on CONNECTORS-288: More code review: The architecture you are using to cache specifications could use some improvement, I think. The method getOutputDescription() is not meant to perform a blind conversion of the output specification to a string, but to include only those parameters that, if changed, would change what was indexed. Furthermore, it is expected that the format of the string be such that it is quickly unpackable, so that no caching should be necessary even if parameters need to be parsed from the string. To help, there are a set of pack/unpack methods available for your use from the base class that are reasonably performant and meant for this purpose. See the Solr connector for an idea how these are used. Or, you can continue to use JSON, but when you go back and forth to JSON I suspect you're doing more work than the pack/unpack methods would do. If you do decide to cache things for whatever reason, I would urge you to use the ICacheManager construct, since that will be guaranteed to be maintained over the long run. Ideally, your code when done should not have any synchronize blocks in it at all, since synchronization is managed largely by the framework. Another subject we should talk about is managing the HTTP connection pool. I noted that you put pool management into one of the subclasses (ElasticSearchConnection). The problem with that is that you want the lifetime of the pool to be the lifetime of the ElasticSearchConnector class instance, otherwise the pool is not going to do you much good. So I would move the MultiThreadedHTTPConnectionManager instance to the main ElasticSearchConnector class, and provide an ElasticSearchConnector method that fetches an HttpClient object from that instance - or just pass it in when you construct ElasticSearchConnection. Also, don't forget to hook up the poll() method to the MultiThreadedHTTPConnectionManager instance so that connections will be closed when idle. See the SharePoint connector for an idea how this is done. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216472#comment-13216472 ] Karl Wright commented on CONNECTORS-288: I got the UI test working, and made some progress on the IT test. Now it runs but the connection that it sets up fails with Threw exception: 'Server/page not found'. I'll look further but this may be more up Luca's alley to figure out. You can figure out what is going on by opening a ManifoldCF UI browser window connected to localhost:8346/mcf-crawler-ui while the test is running. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216481#comment-13216481 ] Luca Stancapiano commented on CONNECTORS-288: - I was working it me too but it seems you resolved first than me I can continue on it...you can leave it to me , don't worry :) An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216504#comment-13216504 ] Luca Stancapiano commented on CONNECTORS-288: - how do you start the test? I use inside mcf-elasticsearch-test project: mvn integration-test and I continue to get the same error starting the test: org.apache.manifoldcf.core.interfaces.ManifoldCFException: ManifoldCF did not terminate in the allotted time of 12 milliseconds at org.apache.manifoldcf.elasticsearch_tests.APISanityDerbyIT.waitJobInactive(APISanityDerbyIT.java:576) at org.apache.manifoldcf.elasticsearch_tests.APISanityDerbyIT.sanityCheck(APISanityDerbyIT.java:412) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) and I confirm that the elasticsearch connector is never called An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215658#comment-13215658 ] Luca Stancapiano commented on CONNECTORS-288: - The problem is that the test expects a json response by the elasticsearch connector plugin but no request is done on it. I expect that the request is done through the startJob call An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215737#comment-13215737 ] Karl Wright commented on CONNECTORS-288: I don't follow you, but maybe a little later I can delve into this further. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214658#comment-13214658 ] Luca Stancapiano commented on CONNECTORS-288: - The test try to connect here: http://localhost:8346/mcf-api-service/json/outputconnections/ElasticSearch%20Connection but there is no active port during the test. Maybe because Manifold goes late to start? An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214670#comment-13214670 ] Karl Wright commented on CONNECTORS-288: The testing infrastructure starts jetty and deploys the war files during test setup. If that hasn't happened then the test is not defined correctly. I'm happy to give this a look, but I can't run with Maven here at work. Is the test expected to work under Ant yet? An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214680#comment-13214680 ] Karl Wright commented on CONNECTORS-288: Just verified that we're not there under ant yet. So let me ask where all the test dependencies come from, so I can code them in Ant. The pom includes: repositories repository idsonatype/id urlhttps://oss.sonatype.org/content/repositories/releases/url /repository /repositories The dependencies look like: dependency groupIdorg.elasticsearch/groupId artifactIdelasticsearch/artifactId version0.18.7/version /dependency dependency groupIdorg.apache.lucene/groupId artifactIdlucene-core/artifactId version3.5.0/version /dependency dependency groupIdorg.apache.lucene/groupId artifactIdlucene-analyzers/artifactId version3.5.0/version /dependency dependency groupIdorg.apache.lucene/groupId artifactIdlucene-snowball/artifactId version3.0.3/version /dependency dependency groupIdorg.apache.lucene/groupId artifactIdlucene-fast-vector-highlighter/artifactId version3.0.3/version /dependency dependency groupIdorg.apache.lucene/groupId artifactIdlucene-highlighter/artifactId version2.4.0/version /dependency dependency groupIdorg.apache.lucene/groupId artifactIdlucene-queries/artifactId version2.4.0/version /dependency I take it the ElasticSearch dependency come from the sonatype repository? An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214682#comment-13214682
]
Piergiorgio Lucidi commented on CONNECTORS-288:
---
{quote}
I take it the ElasticSearch dependency come from the sonatype repository?
{quote}
I suggest to download the ElasticSearch binary package and then you can take
easily all the dependencies from the lib folder, it's more easy. Here the
address of the binary package:
https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.18.7.tar.gz
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214776#comment-13214776 ] Karl Wright commented on CONNECTORS-288: bq. I suggest to download the ElasticSearch binary package and then you can take easily all the dependencies from the lib folder Sounds reasonable. Hopefully will be able to look at this more tonight. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214783#comment-13214783 ] Luca Stancapiano commented on CONNECTORS-288: - I tried the test using only lucene-core 3.5.0 and elasticsearch 0.18.7 as dependencies and I get the same error. Maybe the other libraries are useless An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214874#comment-13214874 ] Luca Stancapiano commented on CONNECTORS-288: - The problem seems to be done by a configuration of the test. Manifold expects the same name for the name of outputConnection: org.apache.manifoldcf.elasticsearch_tests.APISanityIT: 291 and the output_connection field of the job: org.apache.manifoldcf.elasticsearch_tests.APISanityIT: 367 I corrected it and it goes on. If you want I can continue to test it An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214883#comment-13214883 ] Karl Wright commented on CONNECTORS-288: Please continue! It's a busy day here and I won't be able to look at this again until evening, if even then... An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215177#comment-13215177 ] Karl Wright commented on CONNECTORS-288: Committed fixes that tie elasticsearch download and unpack into the build system, and point the elasticsearch IT tests at the unpacked download. Test now fails with an API error; expects 201 but sees 400. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-414) Incorrect path in executecommand.sh, start-webapps.sh and start-database.sh
[ https://issues.apache.org/jira/browse/CONNECTORS-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215457#comment-13215457 ] Karl Wright commented on CONNECTORS-414: Looks good except for a minor thing: $JAVA_HOME/bin/java should be $JAVA_HOME/bin/java in two cases. Incorrect path in executecommand.sh, start-webapps.sh and start-database.sh --- Key: CONNECTORS-414 URL: https://issues.apache.org/jira/browse/CONNECTORS-414 Project: ManifoldCF Issue Type: Bug Components: Examples Affects Versions: ManifoldCF 0.5 Reporter: Shinichiro Abe Assignee: Shinichiro Abe Priority: Minor Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-414.patch Currently the scripts above include an incorrect path. I couldn't run each script at multiprocess-examples. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13213840#comment-13213840 ] Piergiorgio Lucidi commented on CONNECTORS-288: --- I'm sorry for this delay, I think that we only need to add all the needed ElasticSearch dependencies to run these integration tests: * ElasticSearch dependencies * Lucene dependencies I hope during these days to finish the implementation of integration tests. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13213889#comment-13213889
]
Piergiorgio Lucidi commented on CONNECTORS-288:
---
I updated the code with the following changes:
- updated the Lucene dependencies to the 3.5 version on the pom.xml
- an initial fix for the integration tests implementation: now Manifold,
OpenCMIS and ElasticSearch servers started correctly
So now it is possible to execute with Maven the integration tests for this
connector and now it returns this exception:
{code}
OpenCMIS InMemory server is starting...
2012-02-22 20:12:25.157:INFO::jetty-6.1.26
2012-02-22 20:12:25.164:INFO::Extract
../dependency/chemistry-opencmis-server-inmemory.war to
/var/folders/PT/PTaGVJVfF8ChNLT0YTb4tk+++TI/-Tmp-/Jetty_0_0_0_0_9090_chemistry.opencmis.server.inmemory.war__chemistry.opencmis.server.inmemory__80aygt/webapp
22-feb-2012 20.12.27
com.sun.xml.ws.transport.http.servlet.WSServletContextListener
contextInitialized
INFO: WSSERVLET12: JAX-WS context listener initializing
22-feb-2012 20.12.33 com.sun.xml.ws.transport.http.servlet.WSServletDelegate
init
INFO: WSSERVLET14: JAX-WS servlet initializing
2012-02-22 20:12:33.390:INFO::Started SocketConnector@0.0.0.0:9090
OpenCMIS InMemory server is started listening on port 9090
ElasticSearch is starting...
ElasticSearch is started on port 9200
PooledConnection.guardConnection(): found closed Connection. Statement
information follows. Attempting to recover.
PooledConnection.guardConnection: statement was null
PooledConnection.guardConnection(): Recovered connection
java.lang.Exception: API http error; expected 201, saw 400:
at
org.apache.manifoldcf.crawler.tests.ManifoldCFInstance.performAPIPutOperation(ManifoldCFInstance.java:314)
at
org.apache.manifoldcf.crawler.tests.ManifoldCFInstance.performAPIPutOperationViaNodes(ManifoldCFInstance.java:377)
at
org.apache.manifoldcf.crawler.tests.BaseITDerby.performAPIPutOperationViaNodes(BaseITDerby.java:166)
at
org.apache.manifoldcf.elasticsearch_tests.APISanityIT.sanityCheck(APISanityIT.java:345)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
{code}
I think that probably there is a wrong value
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214017#comment-13214017 ] Luca Stancapiano commented on CONNECTORS-288: - yessurely there is something too more in the parameters sent through the test case. I take it An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-412) Velocity params in Open Search Server Connector
[ https://issues.apache.org/jira/browse/CONNECTORS-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211295#comment-13211295 ] Karl Wright commented on CONNECTORS-412: It is a different problem that was already there in the code. I will open a different ticket for it. Velocity params in Open Search Server Connector --- Key: CONNECTORS-412 URL: https://issues.apache.org/jira/browse/CONNECTORS-412 Project: ManifoldCF Issue Type: Improvement Components: OpenSearchServer connector Affects Versions: ManifoldCF 0.5 Environment: velocity Reporter: Luca Stancapiano Assignee: Karl Wright Labels: opensearchserver, velocity Fix For: ManifoldCF 0.5 Attachments: manifold-opensearchserver-velocity-patch, manifold-opensearchserver-velocity-patch I see opensearchserver is the last connector that uses the custom script in the html pages instead of velocity. I updated it with velocity as for the other projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-412) Velocity params in Open Search Server Connector
[ https://issues.apache.org/jira/browse/CONNECTORS-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211300#comment-13211300 ] Karl Wright commented on CONNECTORS-412: The new ticket is CONNECTORS-413. I committed a fix for it too. Velocity params in Open Search Server Connector --- Key: CONNECTORS-412 URL: https://issues.apache.org/jira/browse/CONNECTORS-412 Project: ManifoldCF Issue Type: Improvement Components: OpenSearchServer connector Affects Versions: ManifoldCF 0.5 Environment: velocity Reporter: Luca Stancapiano Assignee: Karl Wright Labels: opensearchserver, velocity Fix For: ManifoldCF 0.5 Attachments: manifold-opensearchserver-velocity-patch, manifold-opensearchserver-velocity-patch I see opensearchserver is the last connector that uses the custom script in the html pages instead of velocity. I updated it with velocity as for the other projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-413) OpenSearchServer connector hidden tabs use wrong variable variant so escaping is not right
[ https://issues.apache.org/jira/browse/CONNECTORS-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211315#comment-13211315 ] Luca Stancapiano commented on CONNECTORS-413: - I've found this thing also in: http://svn.apache.org/repos/asf/incubator/lcf/trunk/connectors/opensearchserver/connector/src/main/resources/org/apache/manifoldcf/agents/output/opensearchserver/editSpecification_en_US.html and: http://svn.apache.org/repos/asf/incubator/lcf/trunk/connectors/opensearchserver/connector/src/main/resources/org/apache/manifoldcf/agents/output/opensearchserver/editSpecification_ja_JP.html should it be changed too? OpenSearchServer connector hidden tabs use wrong variable variant so escaping is not right -- Key: CONNECTORS-413 URL: https://issues.apache.org/jira/browse/CONNECTORS-413 Project: ManifoldCF Issue Type: Bug Components: OpenSearchServer connector Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 The hidden tabs for the OpenSearchServer connector use the _B variant of Velocity variables in some places when they should be using the _A variant. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-413) OpenSearchServer connector hidden tabs use wrong variable variant so escaping is not right
[ https://issues.apache.org/jira/browse/CONNECTORS-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211324#comment-13211324 ] Luca Stancapiano commented on CONNECTORS-413: - Ok, clean OpenSearchServer connector hidden tabs use wrong variable variant so escaping is not right -- Key: CONNECTORS-413 URL: https://issues.apache.org/jira/browse/CONNECTORS-413 Project: ManifoldCF Issue Type: Bug Components: OpenSearchServer connector Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 The hidden tabs for the OpenSearchServer connector use the _B variant of Velocity variables in some places when they should be using the _A variant. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211343#comment-13211343 ] Karl Wright commented on CONNECTORS-288: Committed latest patch. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-velocity-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-410) Rumor has it that the Documentum DFC is now using dfc.properties instead of dmcl.ini
[ https://issues.apache.org/jira/browse/CONNECTORS-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210885#comment-13210885 ] Karl Wright commented on CONNECTORS-410: Couldn't get confirmation as to the necessity of the DOCUMENTUM environment variable, so I went ahead and made the change as if it was still required. Rumor has it that the Documentum DFC is now using dfc.properties instead of dmcl.ini Key: CONNECTORS-410 URL: https://issues.apache.org/jira/browse/CONNECTORS-410 Project: ManifoldCF Issue Type: Bug Components: Documentum connector Affects Versions: ManifoldCF 0.4, ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 The Documentum sidecar process scripts make sure that the environment variable DOCUMENTUM is properly set for DFC by looking for dmcl.ini (for Windows) at that location. I've been told that that is no longer valid; we want to look for dfc.properties instead. This is trivial to change once we get confirmation that this is indeed a correct description of the problem, and also whether the DOCUMENTUM environment variable is even necessary any more. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211039#comment-13211039 ] Luca Stancapiano commented on CONNECTORS-288: - I see still the ElasticSearchSchema class in the SVN: http://svn.apache.org/repos/asf/incubator/lcf/branches/CONNECTORS-288/connectors/elasticsearch/connector/src/main/java/org/apache/manifoldcf/agents/output/elasticsearch/ElasticSearchSchema.java That class should be deleted. It is empty An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211051#comment-13211051 ] Karl Wright commented on CONNECTORS-288: Removed An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-412) Velocity params in Open Search Server Connector
[ https://issues.apache.org/jira/browse/CONNECTORS-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211187#comment-13211187 ] Luca Stancapiano commented on CONNECTORS-412: - sorry...I didn't understand... should I do something about it? Velocity params in Open Search Server Connector --- Key: CONNECTORS-412 URL: https://issues.apache.org/jira/browse/CONNECTORS-412 Project: ManifoldCF Issue Type: Improvement Components: OpenSearchServer connector Affects Versions: ManifoldCF 0.5 Environment: velocity Reporter: Luca Stancapiano Assignee: Karl Wright Labels: opensearchserver, velocity Fix For: ManifoldCF 0.5 Attachments: manifold-opensearchserver-velocity-patch, manifold-opensearchserver-velocity-patch I see opensearchserver is the last connector that uses the custom script in the html pages instead of velocity. I updated it with velocity as for the other projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-411) Execution of integration tests from Maven is broken
[
https://issues.apache.org/jira/browse/CONNECTORS-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210396#comment-13210396
]
Piergiorgio Lucidi commented on CONNECTORS-411:
---
I solved this issue, maybe a mistake during a commit, for some reasons in all
the POMs the execution snippet of the failsafe plugin was removed:
{code}
executions
execution
idintegration-test/id
goals
goalintegration-test/goal
/goals
/execution
execution
idverify/id
goals
goalverify/goal
/goals
/execution
/executions
{code}
I added the snippet and now I'm executing all the integration tests to be sure
to solve the problem and then I think that I can commit the patch.
Execution of integration tests from Maven is broken
---
Key: CONNECTORS-411
URL: https://issues.apache.org/jira/browse/CONNECTORS-411
Project: ManifoldCF
Issue Type: Bug
Components: Build
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Fix For: ManifoldCF 0.5
Original Estimate: 1h
Remaining Estimate: 1h
It seems that there some problems trying to run integration tests using Maven.
Trying to execute integration tests from the 0.4-incubating package, it
works, but trying from the trunk integration tests are not considered.
I think that we can find differences between 0.4-incubating branch and the
trunk to solve this issue.
I'm going in deep on this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209673#comment-13209673 ] Karl Wright commented on CONNECTORS-288: I modified the ant build to properly build and run the tests. Unfortunately the compilation of the IT tests fails. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209841#comment-13209841 ] Karl Wright commented on CONNECTORS-288: Committed latest patch. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-409) mcf-ui-core.jar doesn't exist in dist/example/lib
[ https://issues.apache.org/jira/browse/CONNECTORS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208276#comment-13208276 ] Karl Wright commented on CONNECTORS-409: Yes, please go ahead. Looks like it succeeded for me by random chance. mcf-ui-core.jar doesn't exist in dist/example/lib - Key: CONNECTORS-409 URL: https://issues.apache.org/jira/browse/CONNECTORS-409 Project: ManifoldCF Issue Type: Bug Components: Build Affects Versions: ManifoldCF 0.5 Reporter: Shinichiro Abe Assignee: Shinichiro Abe Priority: Minor Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-409.patch Currently mcf-ui-core.jar isn't copied into dist/example/lib. So I couldn't register info by using crawler-ui at all. I fix framework/build.xml. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-409) mcf-ui-core.jar doesn't exist in dist/example/lib
[ https://issues.apache.org/jira/browse/CONNECTORS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208289#comment-13208289 ] Shinichiro Abe commented on CONNECTORS-409: --- r1244405. Thank you. mcf-ui-core.jar doesn't exist in dist/example/lib - Key: CONNECTORS-409 URL: https://issues.apache.org/jira/browse/CONNECTORS-409 Project: ManifoldCF Issue Type: Bug Components: Build Affects Versions: ManifoldCF 0.5 Reporter: Shinichiro Abe Assignee: Shinichiro Abe Priority: Minor Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-409.patch Currently mcf-ui-core.jar isn't copied into dist/example/lib. So I couldn't register info by using crawler-ui at all. I fix framework/build.xml. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208698#comment-13208698 ] Piergiorgio Lucidi commented on CONNECTORS-288: --- I added an initial version of integration tests. I also found that now Maven doesn't execute integration tests... that sounds very strange to me :( An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-406) Some ManifoldCF code doesn't use the MultiThreadedHttpConnectionManager
[ https://issues.apache.org/jira/browse/CONNECTORS-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207702#comment-13207702 ] Karl Wright commented on CONNECTORS-406: r1243916 fixes the script engine. Some ManifoldCF code doesn't use the MultiThreadedHttpConnectionManager --- Key: CONNECTORS-406 URL: https://issues.apache.org/jira/browse/CONNECTORS-406 Project: ManifoldCF Issue Type: Improvement Components: Framework agents process, OpenSearchServer connector, Scripting client Affects Versions: ManifoldCF 0.5 Reporter: Luca Stancapiano Fix For: ManifoldCF 0.5 I signal three projects not using the http multithread manager: mcf-opensearch-connector pull-agent (tests) scripting-engine -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207822#comment-13207822 ] Karl Wright commented on CONNECTORS-288: If you want to construct a test that uses File System as the source, you would need to import ../ifs-test-build.xml from your test directory's build.xml to include the right stuff. (This is used by the Solr Connector's UI test, but there is no actual integration test you can base your test on yet, so you will be blazing a new path.) An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207826#comment-13207826 ] Piergiorgio Lucidi commented on CONNECTORS-288: --- Ok so probably we could follow the CMIS way to be sure to have some sections of integration tests implemented and taken from another test module. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206872#comment-13206872
]
Karl Wright commented on CONNECTORS-288:
Tried building this morning, but got a build error:
{code}
[javac]
C:\wip\mcf\CONNECTORS-288\connectors\elasticsearch\connector\src\main\java\org\apache\manifoldcf\agents\output\elasticsearch\ElasticSearchConnector.java:382:
cannot access
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchSchema
[javac] bad class file:
C:\wip\mcf\CONNECTORS-288\connectors\elasticsearch\connector\src\main\java\org\apache\manifoldcf\agents\output\elasticsearch\ElasticSearchSchema.java
[javac] file does not contain class
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchSchema
[javac] Please remove or make sure it appears in the correct subdirectory
of the classpath.
[javac] ElasticSearchSchema oss = new
ElasticSearchSchema(getConfigParameters(null));
[javac] ^
[javac] 1 error
{code}
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207010#comment-13207010 ] Karl Wright commented on CONNECTORS-288: Committed this latest patch. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207015#comment-13207015 ] Karl Wright commented on CONNECTORS-288: It now builds too. I guess the main thing we still need is an integration test and a UI test. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207253#comment-13207253 ] Luca Stancapiano commented on CONNECTORS-288: - yesand the velocity part too An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207569#comment-13207569 ] Piergiorgio Lucidi commented on CONNECTORS-288: --- For the integration tests module the ElasticSearch Community replied to my message: https://groups.google.com/group/elasticsearch/msg/f48036470f3931b7 So it seems that it is possible to create an instance of ElasticSearch using the Java API and then we can configure the connector against it. We can try this way, otherwise we can follow the standard way (download, extract and start). An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[
https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206413#comment-13206413
]
Piergiorgio Lucidi commented on CONNECTORS-288:
---
{quote}1 - indent{quote}
{quote}2 - add the velocity parameters{quote}
Luca, I could work on this if you want ;)
Let me know!
{quote}
4 - integration tests
{quote}
Here I can help Luca and I can implement the integration test module, in this
way Luca could continue working directly on the connector.
I published a post in the ElasticSearch group to have a suggestion about
integration tests:
http://groups.google.com/group/elasticsearch/browse_thread/thread/5d568a7f5803acd0
This just to have their opinion about how to implement generic integration
tests.
Anyway I have an idea: we could download the latest stable distribution,
extract it and then start the search server with the bash command. But I would
like to understand if we can use the Java API directly as I saw in their
integration tests code, it could be better to control the instance parameters.
I'm looking forward for their reply and let us see how we could continue :)
An ElasticSearch connector would be helpful
---
Key: CONNECTORS-288
URL: https://issues.apache.org/jira/browse/CONNECTORS-288
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.5
Reporter: Piergiorgio Lucidi
Assignee: Piergiorgio Lucidi
Labels: elasticsearch
Fix For: ManifoldCF next
Attachments: manifold-elasticsearch-patch,
manifold-elasticsearch-patch, manifold-elasticsearch-patch,
manifoldcf-elasticsearch-project-patct
Original Estimate: 120h
Remaining Estimate: 120h
An ElasticSearch connector could be very useful to spread the use of
ManifoldCF
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206456#comment-13206456 ] Karl Wright commented on CONNECTORS-288: Could you do an svn update and then resubmit the latest patch? I'm getting conflicts; it looks like you haven't updated. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206458#comment-13206458 ] Karl Wright commented on CONNECTORS-288: Committed latest patch. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-327) Might be able to save some binary space by only delivering the war files in one place
[ https://issues.apache.org/jira/browse/CONNECTORS-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206080#comment-13206080 ] Karl Wright commented on CONNECTORS-327: r1243033 fixes a regression caused by the previous commit. Might be able to save some binary space by only delivering the war files in one place - Key: CONNECTORS-327 URL: https://issues.apache.org/jira/browse/CONNECTORS-327 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Priority: Minor Fix For: ManifoldCF 0.5 The war files are currently placed in both the multiprocess example and the single process example. We could have them delivered instead to the main dist area, and referenced by the examples using ... In order to make this reasonably configurable, however, we'd also probably want to add properties.xml properties in the single process example that provides paths to the wars explicitly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-327) Might be able to save some binary space by only delivering the war files in one place
[ https://issues.apache.org/jira/browse/CONNECTORS-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206081#comment-13206081 ] Karl Wright commented on CONNECTORS-327: r1243040. Might be able to save some binary space by only delivering the war files in one place - Key: CONNECTORS-327 URL: https://issues.apache.org/jira/browse/CONNECTORS-327 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Priority: Minor Fix For: ManifoldCF 0.5 The war files are currently placed in both the multiprocess example and the single process example. We could have them delivered instead to the main dist area, and referenced by the examples using ... In order to make this reasonably configurable, however, we'd also probably want to add properties.xml properties in the single process example that provides paths to the wars explicitly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206229#comment-13206229 ] Karl Wright commented on CONNECTORS-288: Latest patch committed. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-327) Might be able to save some binary space by only delivering the war files in one place
[ https://issues.apache.org/jira/browse/CONNECTORS-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205374#comment-13205374 ] Karl Wright commented on CONNECTORS-327: r1242749 is a refactoring of the way the Jetty loader finds war files. This permits the relative war file locations to be listed in properties.xml. Might be able to save some binary space by only delivering the war files in one place - Key: CONNECTORS-327 URL: https://issues.apache.org/jira/browse/CONNECTORS-327 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Priority: Minor Fix For: ManifoldCF 0.5 The war files are currently placed in both the multiprocess example and the single process example. We could have them delivered instead to the main dist area, and referenced by the examples using ... In order to make this reasonably configurable, however, we'd also probably want to add properties.xml properties in the single process example that provides paths to the wars explicitly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-327) Might be able to save some binary space by only delivering the war files in one place
[ https://issues.apache.org/jira/browse/CONNECTORS-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206049#comment-13206049 ] Karl Wright commented on CONNECTORS-327: r1243011 fixes lingering inconsistencies with the documentation and the build results, and also standardizes the names of directories that contain jars and the locations of scripts. Might be able to save some binary space by only delivering the war files in one place - Key: CONNECTORS-327 URL: https://issues.apache.org/jira/browse/CONNECTORS-327 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Priority: Minor Fix For: ManifoldCF 0.5 The war files are currently placed in both the multiprocess example and the single process example. We could have them delivered instead to the main dist area, and referenced by the examples using ... In order to make this reasonably configurable, however, we'd also probably want to add properties.xml properties in the single process example that provides paths to the wars explicitly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-402) Separate the proprietary / non-proprietary deliverables for each connector
[ https://issues.apache.org/jira/browse/CONNECTORS-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204496#comment-13204496 ] Karl Wright commented on CONNECTORS-402: I decided to implement the second solution. So now the outstanding issues are down to (2), (3), and (4). Separate the proprietary / non-proprietary deliverables for each connector -- Key: CONNECTORS-402 URL: https://issues.apache.org/jira/browse/CONNECTORS-402 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.4 Reporter: Piergiorgio Lucidi Assignee: Karl Wright Fix For: ManifoldCF 0.5 Original Estimate: 24h Remaining Estimate: 24h As we have spoken today this ticket is dedicated to try to work on the build process to separate proprietary and non-proprietary libraries in the release package. In this way, for example, we could have a standard way to release a specific library for the connector (mcf-alfresco-connector) without release proprietary libraries and allow users to add proprietary libraries in a specific folder connector-lib-proprietary that could be added to the ManifoldCF classloader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-402) Separate the proprietary / non-proprietary deliverables for each connector
[ https://issues.apache.org/jira/browse/CONNECTORS-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204758#comment-13204758 ] Karl Wright commented on CONNECTORS-402: Fixed the tests, I think. Separate the proprietary / non-proprietary deliverables for each connector -- Key: CONNECTORS-402 URL: https://issues.apache.org/jira/browse/CONNECTORS-402 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.4 Reporter: Piergiorgio Lucidi Assignee: Karl Wright Fix For: ManifoldCF 0.5 Original Estimate: 24h Remaining Estimate: 24h As we have spoken today this ticket is dedicated to try to work on the build process to separate proprietary and non-proprietary libraries in the release package. In this way, for example, we could have a standard way to release a specific library for the connector (mcf-alfresco-connector) without release proprietary libraries and allow users to add proprietary libraries in a specific folder connector-lib-proprietary that could be added to the ManifoldCF classloader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205062#comment-13205062 ] Karl Wright commented on CONNECTORS-288: I committed the latest patch. One thing I noticed is that you are not using HTTP connection pooling. In a real system this would lead to problems. I'd suggest you have a look at MultiThreadedHttpConnectionManager; there are examples online that demonstrate reasonable usage. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-402) Separate the proprietary / non-proprietary deliverables for each connector
[ https://issues.apache.org/jira/browse/CONNECTORS-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205132#comment-13205132 ] Karl Wright commented on CONNECTORS-402: The code in the branch is complete; just the documentation needs updating. I'll try and tackle that tomorrow. Separate the proprietary / non-proprietary deliverables for each connector -- Key: CONNECTORS-402 URL: https://issues.apache.org/jira/browse/CONNECTORS-402 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.4 Reporter: Piergiorgio Lucidi Assignee: Karl Wright Fix For: ManifoldCF 0.5 Original Estimate: 24h Remaining Estimate: 24h As we have spoken today this ticket is dedicated to try to work on the build process to separate proprietary and non-proprietary libraries in the release package. In this way, for example, we could have a standard way to release a specific library for the connector (mcf-alfresco-connector) without release proprietary libraries and allow users to add proprietary libraries in a specific folder connector-lib-proprietary that could be added to the ManifoldCF classloader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-401) Update the website with some material about the project
[ https://issues.apache.org/jira/browse/CONNECTORS-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13203798#comment-13203798 ] Piergiorgio Lucidi commented on CONNECTORS-401: --- We only need to update the Japanese translation for the following pages of the website: * books-and-presentations * index Update the website with some material about the project --- Key: CONNECTORS-401 URL: https://issues.apache.org/jira/browse/CONNECTORS-401 Project: ManifoldCF Issue Type: Improvement Components: Site Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Fix For: ManifoldCF 0.5 Original Estimate: 1h Remaining Estimate: 1h We have to update the website to add the following sections: * more details about each specific release in the news section * add a Books and Presentations page in the About section to add links for slides, books and podcasts -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-402) Separate the proprietary / non-proprietary deliverables for each connector
[ https://issues.apache.org/jira/browse/CONNECTORS-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204068#comment-13204068 ] Karl Wright commented on CONNECTORS-402: I created a branch CONNECTORS-402 for this work. Problems still outstanding: (1) Since JDBC drivers must be built into the war files, they cannot be delivered into connector-lib. This means we have no ability to prebuild the JDBC connector to include oracle, mssql, sybase, or mysql support. We need a way of suppressing inclusion of these JDBC jars when building for release, even though we will likely have run the ant download-dependencies target in the release branch. (2) The tests all need to be run and checked, since the place where proprietary materials are placed has now become lib-proprietary across the board. (3) The how-to-build-and-deploy documentation needs significant changes. (4) The process components of the FileNet and Documentum connectors are where the proprietary jars wind up; we need a solution for these. Separate the proprietary / non-proprietary deliverables for each connector -- Key: CONNECTORS-402 URL: https://issues.apache.org/jira/browse/CONNECTORS-402 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.4 Reporter: Piergiorgio Lucidi Assignee: Karl Wright Fix For: ManifoldCF 0.5 Original Estimate: 24h Remaining Estimate: 24h As we have spoken today this ticket is dedicated to try to work on the build process to separate proprietary and non-proprietary libraries in the release package. In this way, for example, we could have a standard way to release a specific library for the connector (mcf-alfresco-connector) without release proprietary libraries and allow users to add proprietary libraries in a specific folder connector-lib-proprietary that could be added to the ManifoldCF classloader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204115#comment-13204115 ] Karl Wright commented on CONNECTORS-288: Committed second patch to the branch. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifold-elasticsearch-patch, manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-402) Separate the proprietary / non-proprietary deliverables for each connector
[ https://issues.apache.org/jira/browse/CONNECTORS-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204126#comment-13204126 ] Karl Wright commented on CONNECTORS-402: I can think of two models that might resolve (1) and (5). That model would involve having a build-image target pretty much everywhere there's a build target today. Then the image targets would depend on build-image rather than on build. This is not ideal because if you forget and do a build first, or you try to run tests, etc. you will construct an image that includes things that should not be shipped. But this is the way it works today to a large extent anyhow. The alternative is to just exclude things when building the image. That is, during the build process where there are two differing results (depending on whether the result is included in the release image) we build BOTH of them, and exclude the proprietary one when we zip up the binary for distribution. This means that there would be two versions of each example, for instance - a non-proprietary version, and a proprietary one. Similarly, there would be two connectors.xml files; one non-proprietary, and one proprietary. Separate the proprietary / non-proprietary deliverables for each connector -- Key: CONNECTORS-402 URL: https://issues.apache.org/jira/browse/CONNECTORS-402 Project: ManifoldCF Issue Type: Improvement Components: Build Affects Versions: ManifoldCF 0.4 Reporter: Piergiorgio Lucidi Assignee: Karl Wright Fix For: ManifoldCF 0.5 Original Estimate: 24h Remaining Estimate: 24h As we have spoken today this ticket is dedicated to try to work on the build process to separate proprietary and non-proprietary libraries in the release package. In this way, for example, we could have a standard way to release a specific library for the connector (mcf-alfresco-connector) without release proprietary libraries and allow users to add proprietary libraries in a specific folder connector-lib-proprietary that could be added to the ManifoldCF classloader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13202370#comment-13202370 ] Karl Wright commented on CONNECTORS-288: I just built the connector using ant. First thing I noticed was the dist/connectors.xml file: !-- Add your output connectors here -- outputconnector name=OpenSearchServer class=org.apache.manifoldcf.agents.output.elasticsearch.OpenSearchServerConnector/ outputconnector name=OpenSearchServer class=org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector/ You can't have two connectors with the same name ;-). Also the class name looks suspicious. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13202374#comment-13202374 ] Piergiorgio Lucidi commented on CONNECTORS-288: --- Luca will work on this task during this week, I think that he will solve all the issues ;) An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13202415#comment-13202415 ] Luca Stancapiano commented on CONNECTORS-288: - yes no problem I'll work for it in this week. I will write here and in the mailing list for problems An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200692#comment-13200692 ] Tommaso Teofili commented on CONNECTORS-288: Nice thing Luce, looking forward to a patch for it :) An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Fix For: ManifoldCF next Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200881#comment-13200881 ] Luca Stancapiano commented on CONNECTORS-288: - I add here a first patch. It is a skeleton for project and docs. So it is not still working but I prefer maintain an history of the work. The next thing to do is to add json calls. An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200931#comment-13200931 ] Karl Wright commented on CONNECTORS-288: I've created a branch for this work and committed the first patch against it: branches/CONNECTORS-288. Please submit subsequent diffs against this branch. Thanks! An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Affects Versions: ManifoldCF 0.5 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Labels: elasticsearch Fix For: ManifoldCF next Attachments: manifoldcf-elasticsearch-project-patct Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-398) Alfresco Connector UI Japanese translation would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200970#comment-13200970 ] Karl Wright commented on CONNECTORS-398: The resource switching based on locale is already in the code, so the new patch just needs to be committed. Alfresco Connector UI Japanese translation would be helpful --- Key: CONNECTORS-398 URL: https://issues.apache.org/jira/browse/CONNECTORS-398 Project: ManifoldCF Issue Type: Improvement Components: Alfresco connector Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-398.patch Original Estimate: 4h Remaining Estimate: 4h A Japanese translation of the Alfresco Connector could be very useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-398) Alfresco Connector UI Japanese translation would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200974#comment-13200974 ] Karl Wright commented on CONNECTORS-398: r1240865. Alfresco Connector UI Japanese translation would be helpful --- Key: CONNECTORS-398 URL: https://issues.apache.org/jira/browse/CONNECTORS-398 Project: ManifoldCF Issue Type: Improvement Components: Alfresco connector Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-398.patch Original Estimate: 4h Remaining Estimate: 4h A Japanese translation of the Alfresco Connector could be very useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-288) An ElasticSearch connector would be helpful
[ https://issues.apache.org/jira/browse/CONNECTORS-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200568#comment-13200568 ] Luca Stancapiano commented on CONNECTORS-288: - I find it interesting.ready to start! An ElasticSearch connector would be helpful --- Key: CONNECTORS-288 URL: https://issues.apache.org/jira/browse/CONNECTORS-288 Project: ManifoldCF Issue Type: New Feature Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Fix For: ManifoldCF next Original Estimate: 120h Remaining Estimate: 120h An ElasticSearch connector could be very useful to spread the use of ManifoldCF -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-396) CMIS Connector UI Japanese translation
[ https://issues.apache.org/jira/browse/CONNECTORS-396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198400#comment-13198400 ] Hitoshi Ozawa commented on CONNECTORS-396: -- I won't mind but please wait until this weekend. I'm really tied up right now with a short term project. CMIS Connector UI Japanese translation -- Key: CONNECTORS-396 URL: https://issues.apache.org/jira/browse/CONNECTORS-396 Project: ManifoldCF Issue Type: Improvement Components: CMIS connector Affects Versions: ManifoldCF 0.3 Reporter: Piergiorgio Lucidi Assignee: Piergiorgio Lucidi Fix For: ManifoldCF 0.5 Original Estimate: 1h Remaining Estimate: 1h I updated the Japanese templates with the latest changes and we need to translate some terms in the Japanese language. Maybe Hitoshi could help us here ;) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-387) Japanese fonts are missing from the PDF files generated for the site
[ https://issues.apache.org/jira/browse/CONNECTORS-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196768#comment-13196768 ] Karl Wright commented on CONNECTORS-387: I'd hoped to modify output.xmap to do the right thing for condition font inclusion, but this is not going to work because the conditional here would just change font registration, and we need to change font specification. We really need conditionals in the forrest.properties.xml to make the right thing happen. Nevertheless I am certain that the parameters created by forrest.parameters.xml are substituted somewhere to specify the font selection for FOP. That file can be replaced by our own version with multilanguage support the same way I replace the other files in Forrest. Will keep looking... Japanese fonts are missing from the PDF files generated for the site Key: CONNECTORS-387 URL: https://issues.apache.org/jira/browse/CONNECTORS-387 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 Japanese fonts are missing from the PDFs generated for the site. This page describes how to fix that: http://forrest.apache.org/pluginDocs/plugins_0_90/org.apache.forrest.plugin.output.pdf/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-387) Japanese fonts are missing from the PDF files generated for the site
[ https://issues.apache.org/jira/browse/CONNECTORS-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196775#comment-13196775 ] Karl Wright commented on CONNECTORS-387: The right file for conditionalizating font choice is clearly plugins/org.apache.forrest.plugin.output.pdf/resources/stylesheets/document-to-fo.xsl. Unfortunately I don't see any evidence that the stylesheet has access to the path of the current file, and what its variable name would be called. It may even be that the stylesheet is applied once, not on every PDF converted, which would be a hard thing to work around without more significant changes to the Forrest code. I'm also not a stylesheet pro so I'd love an example of a conditional stylesheet selection based on a regexp match. Japanese fonts are missing from the PDF files generated for the site Key: CONNECTORS-387 URL: https://issues.apache.org/jira/browse/CONNECTORS-387 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 Japanese fonts are missing from the PDFs generated for the site. This page describes how to fix that: http://forrest.apache.org/pluginDocs/plugins_0_90/org.apache.forrest.plugin.output.pdf/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-387) Japanese fonts are missing from the PDF files generated for the site
[ https://issues.apache.org/jira/browse/CONNECTORS-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195994#comment-13195994 ] Karl Wright commented on CONNECTORS-387: Looks good, and the properly-generated PDFs are already live. I've rearranged the site to give more equal emphasis to english and japanese, and also allow for the crawler UI help like to take you to the right help page for your language. The only downside here is that there's now need of a launch tab which does nothing more than tell you to click on the appropriate language tab. I'm hoping Google will pick this up OK. Hitoshi, could you synch up trunk and do ant download-dependencies doc, and see that the site navigation for the site at dist/doc now makes sense in Japanese? The only outstanding work for this ticket now is trying to get different fonts used to generate PDFs for different parts of the site. Japanese fonts are missing from the PDF files generated for the site Key: CONNECTORS-387 URL: https://issues.apache.org/jira/browse/CONNECTORS-387 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 Japanese fonts are missing from the PDFs generated for the site. This page describes how to fix that: http://forrest.apache.org/pluginDocs/plugins_0_90/org.apache.forrest.plugin.output.pdf/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-387) Japanese fonts are missing from the PDF files generated for the site
[ https://issues.apache.org/jira/browse/CONNECTORS-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196453#comment-13196453 ] Hitoshi Ozawa commented on CONNECTORS-387: -- Looks very good! Sorry, that I wasn't too much of a help, but I'm tied up with a short term customer project. Japanese fonts are missing from the PDF files generated for the site Key: CONNECTORS-387 URL: https://issues.apache.org/jira/browse/CONNECTORS-387 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 Japanese fonts are missing from the PDFs generated for the site. This page describes how to fix that: http://forrest.apache.org/pluginDocs/plugins_0_90/org.apache.forrest.plugin.output.pdf/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-387) Japanese fonts are missing from the PDF files generated for the site
[ https://issues.apache.org/jira/browse/CONNECTORS-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195780#comment-13195780 ] Karl Wright commented on CONNECTORS-387: Made some progress. Turns out the the symbols-project-v10.ent file is never read. Furthermore, since none of the paths can be relative, I realized we're better off just making a local copy of Forrest and modifying the PDF plugin's output.xmap. I've written Ant targets which do all of these things, and it is largely successful, except for the very last step, which is generating the PDFs themselves. Then, for each PDF I get the following: BROKEN: For TrueType collection you must specify which font to select (-ttcname) This is the same error I see when I use FOP to generate a metrics file, unless I include -ttcname xxx on the command line to specify the font. The IPA font collection has 2 fonts in it which explains why it is needed. But the font tag in the config.xml does not seem to have any way of describing the font name, so once again I'm stuck. I may need to go hacking around inside FOP to get this to work now. Japanese fonts are missing from the PDF files generated for the site Key: CONNECTORS-387 URL: https://issues.apache.org/jira/browse/CONNECTORS-387 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 Japanese fonts are missing from the PDFs generated for the site. This page describes how to fix that: http://forrest.apache.org/pluginDocs/plugins_0_90/org.apache.forrest.plugin.output.pdf/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-387) Japanese fonts are missing from the PDF files generated for the site
[ https://issues.apache.org/jira/browse/CONNECTORS-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195802#comment-13195802 ] Karl Wright commented on CONNECTORS-387: It looks like someone tried to add support for .ttc's in FOP 1.0. To use it you are supposed to set the attribute sub-font to the font name in the collection. But this does not seem to have any effect. This could be because Forrest's FOP version is 0.95 at the moment. But replacing the 0.95 fop jar with the 1.0 fop jar still doesn't get us past this problem. Japanese fonts are missing from the PDF files generated for the site Key: CONNECTORS-387 URL: https://issues.apache.org/jira/browse/CONNECTORS-387 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 Japanese fonts are missing from the PDFs generated for the site. This page describes how to fix that: http://forrest.apache.org/pluginDocs/plugins_0_90/org.apache.forrest.plugin.output.pdf/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-387) Japanese fonts are missing from the PDF files generated for the site
[ https://issues.apache.org/jira/browse/CONNECTORS-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195812#comment-13195812 ] Karl Wright commented on CONNECTORS-387: r1237395 Japanese fonts are missing from the PDF files generated for the site Key: CONNECTORS-387 URL: https://issues.apache.org/jira/browse/CONNECTORS-387 Project: ManifoldCF Issue Type: Bug Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 Japanese fonts are missing from the PDFs generated for the site. This page describes how to fix that: http://forrest.apache.org/pluginDocs/plugins_0_90/org.apache.forrest.plugin.output.pdf/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-393) SharePoint connector does not accept empty username/password
[ https://issues.apache.org/jira/browse/CONNECTORS-393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195813#comment-13195813 ] Karl Wright commented on CONNECTORS-393: r1237395 includes a fix for this. SharePoint connector does not accept empty username/password Key: CONNECTORS-393 URL: https://issues.apache.org/jira/browse/CONNECTORS-393 Project: ManifoldCF Issue Type: Bug Components: SharePoint connector Affects Versions: ManifoldCF 0.4, ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 The SharePoint connector does not currently allow an empty username, and thus no authentication. The problem is in the UI Javascript. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
