[
https://issues.apache.org/jira/browse/METRON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197520#comment-16197520
]
ASF GitHub Bot commented on METRON-1230:
----------------------------------------
Github user mmiklavc commented on a diff in the pull request:
https://github.com/apache/metron/pull/785#discussion_r143550601
--- Diff:
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/topology/MergeAndShadeTransformer.java
---
@@ -0,0 +1,90 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.topology;
+
+import com.google.common.base.Splitter;
+import org.apache.storm.daemon.JarTransformer;
+import org.apache.storm.hack.StormShadeTransformer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.lang.invoke.MethodHandles;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.jar.JarEntry;
+import java.util.jar.JarInputStream;
+import java.util.jar.JarOutputStream;
+
+/**
+ * This is a storm jar transformer that will add in additional jars pulled
from an
+ * environment variable. The jars will be merged with the main uber jar
and then
+ * the resulting jar will be shaded and relocated according to the
StormShadeTransformer.
+ *
+ */
+public class MergeAndShadeTransformer implements JarTransformer {
+ public static final String EXTRA_JARS_ENV = "EXTRA_JARS";
+ private static final Logger LOG =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ StormShadeTransformer _underlyingTransformer = new
StormShadeTransformer();
+ @Override
+ public void transform(InputStream input, OutputStream output) throws
IOException {
+ String extraJars = System.getenv().get(EXTRA_JARS_ENV);
+ if(extraJars == null || extraJars.length() == 0) {
+ _underlyingTransformer.transform(input, output);
+ return;
+ }
+ File tmpFile = File.createTempFile("metron", "jar");
+ tmpFile.deleteOnExit();
+ Set<String> entries = new HashSet<>();
+ try (JarOutputStream jout = new JarOutputStream(new
BufferedOutputStream(new FileOutputStream(tmpFile)))) {
+ try (JarInputStream jin = new JarInputStream(new
BufferedInputStream(input))){
+ copy(jin, jout, entries);
+ }
+ for (String fileStr : Splitter.on(",").split(extraJars)) {
+ File f = new File(fileStr);
+ if (!f.exists()) {
+ continue;
+ }
+ LOG.info("Merging jar {} from {}", f.getName(),
f.getAbsolutePath());
+ try (JarInputStream jin = new JarInputStream(new
BufferedInputStream(new FileInputStream(f)))) {
+ copy(jin, jout, entries);
+ }
+ }
+ }
+ _underlyingTransformer.transform(new BufferedInputStream(new
FileInputStream(tmpFile)), output);
+ }
+
+ private void copy(JarInputStream jin, JarOutputStream jout, Set<String>
entries) throws IOException {
+ byte[] buffer = new byte[1024];
+ for(JarEntry entry = jin.getNextJarEntry(); entry != null; entry =
jin.getNextJarEntry()) {
+ if(entries.contains(entry.getName())) {
+ continue;
+ }
+ if(LOG.isDebugEnabled()) {
--- End diff --
isDebugEnabled is no longer necessary with SLF4J - the token replacement is
disabled under the hood if debugging is not enabled.
> As a stopgap prior to METRON-777, add more simplistic sideloading of custom
> Parsers
> -----------------------------------------------------------------------------------
>
> Key: METRON-1230
> URL: https://issues.apache.org/jira/browse/METRON-1230
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
>
> Until we get 777 in, it'd be nice to have a simple ability using the normal
> storm functionality to enable users to provide custom parsers without forking
> Metron. This should be done via simply creating a jar with their code (and
> bundled dependencies) and have it picked up and available to the REST Service
> (and consequently the management UI) as well as the start_parser_topology.sh
> command.
> This should be minimal movement as we have a more robust solution coming with
> METRON-777.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
