Hi Eric, I also faced errors when setting up Metron in AWS. You can try installing the cluster using Ambari.I faced issues in playbook due to Ansible version issues.
Thanks, Hema On Fri, 13 Sep, 2019, 4:30 PM Otto Fowler, <[email protected]> wrote: > I believe that the EC2 script has been tested before every release. The > issue here is probably the build. > > We need the real errors, maybe you can post it as a gist or something. > > Also, maybe you can log into the machine and see if you can just build > metron ( without the ansible and deploy stuff. > > > > > On September 13, 2019 at 06:57:30, Otto Fowler ([email protected]) > wrote: > > So you are using > https://github.com/apache/metron/tree/master/metron-deployment/amazon-ec2 > ? > > > > On September 12, 2019 at 16:27:43, Eric Jacksch ([email protected]) wrote: > > Greetings, > > I've been trying to deploy in AWS to ec2 instances using the playbook. > > The VPC is created, instances spun up, etc, but I get hundreds (or > thousands) of lines of errors during the build. > > Can anyone help? > > I'm running the playbook on another EC2 instance, so it is certainly > possible that I am missing a dependency, etc... > > Thanks! > > > > TASK [metron-builder : Build Metron] > > ********************************************************************************************** > failed: [ec2-35-183-121-120.ca-central-1.compute.amazonaws.com -> > localhost] (item=mvn package -DskipTests -T 2C -P HDP-2.5.0.0,mpack) > => {"changed": true, "cmd": "mvn package -DskipTests -T 2C -P > HDP-2.5.0.0,mpack", "delta": "0:01:12.526650", "end": "2019-09-12 > 19:58:48.231138", "failed": true, "item": "mvn package -DskipTests -T > 2C -P HDP-2.5.0.0,mpack", "msg": "non-zero return code", "rc": 1, > "start": "2019-09-12 19:57:35.704488", "stderr": "warning: No > SupportedSourceVersion annotation found on > org.adrianwalker.multilinestring.MultilineProcessor, returning > RELEASE_6.\nwarning: Supported source version 'RELEASE_6' from > annotation processor > 'org.adrianwalker.multilinestring.MultilineProcessor' less than > -source '1.8'\n2 warnings\nwarning: No SupportedSourceVersion > annotation found on > org.adrianwalker.multilinestring.MultilineProcessor, returning > RELEASE_6.\nwarning: Supported source version 'RELEASE_6' from > annotation processor > 'org.adrianwalker.multilinestring.MultilineProcessor' less than > -source > '1.8'\n/home/ec2-user/metron/metron-platform/metron-test-utilities/src/main/java/org/apache/metron/test/bolt/BaseBoltTest.java:87: > warning: [unchecked] unchecked method invocation: method copyOf in > class ImmutableSet is applied to given types\n ImmutableSet keys = > ImmutableSet.copyOf(message.keySet());\n > ^\n required: Collection<? extends E>\n found: Set\n > where E is a type-variable:\n E extends Object declared in method > <E>copyOf(Collection<? extends > > E>)\n/home/ec2-user/metron/metron-platform/metron-test-utilities/src/main/java/org/apache/metron/test/bolt/BaseBoltTest.java:87: > warning: [unchecked] unchecked conversion\n ImmutableSet keys = > ImmutableSet.copyOf(message.keySet());\n > ^\n required: Collection<? extends E>\n > found: Set\n where E is a type-variable:\n E extends Object > declared in method <E>copyOf(Collection<? extends > > E>)\n/home/ec2-user/metron/metron-platform/metron-test-utilities/src/main/java/org/apache/metron/test/utils/KafkaLoader.java:67: > warning: [unchecked] unchecked call to send(ProducerRecord<K,V>) as a > member of the raw type KafkaProducer\n kafkaProducer.send(new > ProducerRecord<String, String>(topic, line));\n > ^\n where K,V are type-variables:\n K extends Object declared > in class KafkaProducer\n V extends Object declared in class > KafkaProducer\n5 > > warnings\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:174: > warning: [unchecked] unchecked conversion\n functionRestConfig > = getArg(1, Map.class, args);\n ^\n > required: Map<String,Object>\n found: > > Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:176: > warning: [unchecked] unchecked conversion\n queryParameters = > getArg(2, Map.class, args);\n ^\n > required: Map<String,Object>\n found: > > Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:181: > warning: [unchecked] unchecked cast\n Map<String, Object> > globalRestConfig = (Map<String, Object>) > getGlobalConfig(context).get(STELLAR_REST_SETTINGS);\n > > ^\n required: Map<String,Object>\n found: > > Object\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:182: > warning: [unchecked] unchecked cast\n Map<String, Object> > getRestConfig = (Map<String, Object>) > getGlobalConfig(context).get(STELLAR_REST_GET_SETTINGS);\n > > ^\n required: Map<String,Object>\n found: > > Object\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:183: > warning: [unchecked] unchecked generic array creation for varargs > parameter of type Map<String,Object>[]\n RestConfig restConfig = > buildRestConfig(globalRestConfig, getRestConfig, > functionRestConfig);\n > > ^\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:254: > warning: [unchecked] unchecked conversion\n functionRestConfig > = getArg(2, Map.class, args);\n ^\n > required: Map<String,Object>\n found: > > Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:256: > warning: [unchecked] unchecked conversion\n queryParameters = > getArg(3, Map.class, args);\n ^\n > required: Map<String,Object>\n found: > > Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:261: > warning: [unchecked] unchecked cast\n Map<String, Object> > globalRestConfig = (Map<String, Object>) > getGlobalConfig(context).get(STELLAR_REST_SETTINGS);\n > > ^\n required: Map<String,Object>\n found: > > Object\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:262: > warning: [unchecked] unchecked cast\n Map<String, Object> > postRestConfig = (Map<String, Object>) > getGlobalConfig(context).get(STELLAR_REST_POST_SETTINGS);\n > > ^\n required: Map<String,Object>\n found: > > Object\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:263: > warning: [unchecked] unchecked generic array creation for varargs > parameter of type Map<String,Object>[]\n RestConfig restConfig = > buildRestConfig(globalRestConfig, postRestConfig, > functionRestConfig);\n > > ^\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/RestFunctions.java:347: > warning: [unchecked] unchecked generic array creation for varargs > parameter of type Map<String,Object>[]\n RestConfig restConfig = > buildRestConfig(getGlobalConfig(context));\n > > > ^\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarParser.java:3045: > warning: [MissingOverride] sempred overrides method in Recognizer; > expected @Override\n\tpublic boolean sempred(RuleContext _localctx, > int ruleIndex, int predIndex) {\n\t ^\n (see > http://errorprone.info/bugpattern/MissingOverride)\n Did you mean > '@Override public boolean sempred(RuleContext _localctx, int > ruleIndex, int predIndex) > > {'?\n/home/ec2-user/metron/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/SerDeUtils.java:136: > warning: [unchecked] newInstantiatorOf(Class) in > DefaultInstantiatorStrategy implements <T>newInstantiatorOf(Class<T>) > in InstantiatorStrategy\n public ObjectInstantiator > newInstantiatorOf(final Class type) {\n > ^\n return type requires unchecked conversion from ObjectInstantiator > to ObjectInstantiator<T>\n where T is a type-variable:\n T extends > Object declared in method <T>newInstantiatorOf(Class<T>)\n13 > warnings\nwarning: No SupportedSourceVersion annotation found on > org.adrianwalker.multilinestring.MultilineProcessor, returning > RELEASE_6.\nwarning: Supported source version 'RELEASE_6' from > annotation processor > 'org.adrianwalker.multilinestring.MultilineProcessor' less than > -source > '1.8'\n/home/ec2-user/metron/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java:654: > warning: [unchecked] unchecked method invocation: method run in class > StellarProcessorUtils is applied to given types\n > assertThat(run(query, vars), instanceOf(Map.class));\n > ^\n required: String,Map<String,Object>\n found: > > String,Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java:654: > warning: [unchecked] unchecked conversion\n assertThat(run(query, > vars), instanceOf(Map.class));\n ^\n > required: Map<String,Object>\n found: > > Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java:656: > warning: [unchecked] unchecked method invocation: method run in class > StellarProcessorUtils is applied to given types\n > assertThat(run(query, vars), equalTo(\"bar\"));\n ^\n > required: String,Map<String,Object>\n found: > > String,Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java:656: > warning: [unchecked] unchecked conversion\n assertThat(run(query, > vars), equalTo(\"bar\"));\n ^\n required: > Map<String,Object>\n found: > > Map\n/home/ec2-user/metron/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java:662: > warning: [unchecked] unchecked call to put(K,V) as a member of the raw > type HashMap\n put(\"mymap\", null);\n ^\n where K,V are > type-variables:\n K extends Object declared in class HashMap\n V > extends Object declared in class > > HashMap\n/home/ec2-user/metron/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/fu > > <snip...many, many, many lines omitted> > > (Wall Clock)", "[INFO] Finished at: 2019-09-12T19:58:47Z", "[INFO] > Final Memory: 738M/2811M", "[INFO] > ------------------------------------------------------------------------", > "[ERROR] Failed to execute goal > org.apache.maven.plugins:maven-compiler-plugin:3.5.1:compile > (default-compile) on project metron-profiler-spark: Compilation > failure -> [Help 1]", "[ERROR] ", "[ERROR] To see the full stack trace > of the errors, re-run Maven with the -e switch.", "[ERROR] Re-run > Maven using the -X switch to enable full debug logging.", "[ERROR] ", > "[ERROR] For more information about the errors and possible solutions, > please read the following articles:", "[ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException";, > "[ERROR] ", "[ERROR] After correcting the problems, you can resume the > build with the command", "[ERROR] mvn <goals> -rf > :metron-profiler-spark", "[INFO] ", "[INFO] --- > maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ > metron-data-management ---"]} > > NO MORE HOSTS LEFT > > **************************************************************************************************************** > to retry, use: --limit > @/home/ec2-user/metron/metron-deployment/amazon-ec2/playbook.retry > > PLAY RECAP > ************************************************************************************************************************ > ec2-15-222-1-43.ca-central-1.compute.amazonaws.com : ok=14 changed=8 > unreachable=0 failed=0 > ec2-15-222-9-90.ca-central-1.compute.amazonaws.com : ok=14 changed=8 > unreachable=0 failed=0 > ec2-35-182-158-204.ca-central-1.compute.amazonaws.com : ok=14 > changed=8 unreachable=0 failed=0 > ec2-35-183-110-49.ca-central-1.compute.amazonaws.com : ok=14 > changed=8 unreachable=0 failed=0 > ec2-35-183-121-120.ca-central-1.compute.amazonaws.com : ok=15 > changed=9 unreachable=0 failed=1 > ec2-35-183-121-202.ca-central-1.compute.amazonaws.com : ok=14 > changed=8 unreachable=0 failed=0 > ec2-35-183-210-180.ca-central-1.compute.amazonaws.com : ok=14 > changed=8 unreachable=0 failed=0 > ec2-99-79-193-27.ca-central-1.compute.amazonaws.com : ok=14 > changed=8 unreachable=0 failed=0 > ec2-99-79-52-74.ca-central-1.compute.amazonaws.com : ok=14 changed=8 > unreachable=0 failed=0 > ec2-99-79-60-74.ca-central-1.compute.amazonaws.com : ok=14 changed=8 > unreachable=0 failed=0 > localhost : ok=20 changed=15 unreachable=0 failed=0 > >
