[jira] [Commented] (CONNECTORS-1636) ElasticSearch Connector not working with ingest pipeline processor attachment
[
https://issues.apache.org/jira/browse/CONNECTORS-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037123#comment-17037123
]
Rohit Batta commented on CONNECTORS-1636:
-
Hi Karl,
I have fixed it by validating pipeline and processing base64 in case of
pipeline uses attachment processor.
I am facing one issue now if I add some other fields to form by declaring those
to `ElasticSearchParam.java`, the form is not rendered.
{code:html}
$Encoder.bodyEscape($ResourceBundle.getString('ElasticSearchConnector.ReferenceIndexNameColon'))
{code}
Error logs prints
{code:txt}
// Some comments here
org.apache.velocity.exception.MethodInvocationException: Invocation of method
'attributeEscape' in class java.lang.Class threw exception
java.lang.NullPointerException at editConfiguration_Parameters.html[line 103,
column 64]
at
org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:243)
at
org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:187)
at
org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
at
org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:369)
at
org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at
org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:106)
at
org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at org.apache.velocity.Template.merge(Template.java:356)
at org.apache.velocity.Template.merge(Template.java:260)
at
org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:354)
at
org.apache.manifoldcf.ui.i18n.Messages.outputResourceWithVelocity(Messages.java:159)
at
org.apache.manifoldcf.agents.output.elasticsearch.Messages.outputResourceWithVelocity(Messages.java:136)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.outputResource(ElasticSearchConnector.java:269)
at
org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.outputConfigurationBody(ElasticSearchConnector.java:294)
at
org.apache.manifoldcf.core.interfaces.ConnectorFactory.outputThisConfigurationBody(ConnectorFactory.java:83)
at
org.apache.manifoldcf.agents.interfaces.OutputConnectorFactory.outputConfigurationBody(OutputConnectorFactory.java:107)
at org.apache.jsp.editoutput_jsp._jspService(editoutput_jsp.java:694)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at
[jira] [Commented] (CONNECTORS-1636) ElasticSearch Connector not working with ingest pipeline processor attachment
[
https://issues.apache.org/jira/browse/CONNECTORS-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037006#comment-17037006
]
Karl Wright commented on CONNECTORS-1636:
-
Patches welcome.
ElasticSearch changes so quickly it's very hard to keep up.
> ElasticSearch Connector not working with ingest pipeline processor attachment
> -
>
> Key: CONNECTORS-1636
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1636
> Project: ManifoldCF
> Issue Type: Bug
> Components: Elastic Search connector
>Affects Versions: ManifoldCF 2.15
>Reporter: Rohit Batta
>Assignee: Karl Wright
>Priority: Major
> Labels: ManifoldCF, connector, elasticsearch, manifoldcf
> Fix For: ManifoldCF next
>
>
> While using Apache manifoldcf elasticsearch connector for elasticsearch
> version 6.6.x, I found that connector is not working as expected for pipeline
> processor "attachment".
> The processor requires Base64 String to process input stream to content.
> It is working for "mapper-attachment" plugin but that plugin is deprecated in
> newer versions of elasticsearch.
> In case elasticsearch pipeline is used and mapper-attachment is set to false.
> then the content is processed as byte Array to index document, which is not
> correct type for indexing to elasticsearch.
>
> {code:java}
> if (!useMapperAttachments && inputStream != null) {
> if (contentAttributeName != null) {
> Reader r = new InputStreamReader(inputStream, Consts.UTF_8);
> if (needComma) {
> pw.print(",");
> }
> pw.append(jsonStringEscape(contentAttributeName)).append(" : \"");
> char[] buffer = new char[65536];
> while (true) {
> int amt = r.read(buffer, 0, buffer.length);
> if (amt == -1)
> break;
> for (int j = 0; j < amt; j++) {
> final char x = buffer[j];
> if (x == '\n')
> pw.append('\\').append('n');
> else if (x == '\r')
> pw.append('\\').append('r');
> else if (x == '\t')
> pw.append('\\').append('t');
> else if (x == '\b')
> pw.append('\\').append('b');
> else if (x == '\f')
> pw.append('\\').append('f');
> else if (x < 32) {
> pw.append("\\u").append(String.format(Locale.ROOT,
> "%04x", (int) x));
> } else {
> if (x == '\"' || x == '\\' || x == '/')
> pw.append('\\');
> pw.append(x);
> }
> }
> }
> pw.append("\"");
> needComma = true;
> }
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
