Shouldn't require additional authorization unless something changed recently. Did you try creating a PR from your branch in your fork on github.com?
Thanks On Tue, Jan 15, 2019 at 6:59 PM Aaron Bossert <aa...@punchcyber.com> wrote: > When I try to do a pull request from Intellij, I get a permissions > error...I have not done this much, so it could be something really dumb on > my part...do I need you to authorize my account in some way? > > On Tue, Jan 15, 2019 at 3:20 PM Aaron Bossert <aa...@punchcyber.com> > wrote: > > > OK, I have my local clone working and updated for Hadoop 3.1... > > > > On Tue, Jan 15, 2019 at 1:55 PM Aaron Bossert <aa...@punchcyber.com> > > wrote: > > > >> OK, folks. I have imported the Master branch of apex-core....I updated > >> the dependency for Hadoop to 3.1.0 and here is first round of > problems. Of > >> those listed below, all but the last two constants have equivalents in > the > >> newer version that were easy enough to do a find and replace > >> with...however, the last two are shown in 2.6.1 as deprecated, but no > >> mention of a replacement...Perhaps someone more familiar with the Apex > code > >> base can chime in on these two constants (highlighted in red-ish toward > the > >> bottom and copied just below for convenience)? > >> > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramClientUtils.java:[831,71] > >> cannot find symbol > >> symbol: variable HADOOP_SSL_ENABLED_KEY > >> location: class org.apache.hadoop.fs.CommonConfigurationKeysPublic > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramClientUtils.java:[831,125] > >> cannot find symbol > >> symbol: variable HADOOP_SSL_ENABLED_DEFAULT > >> location: class org.apache.hadoop.fs.CommonConfigurationKeysPublic > >> > >> > >> > >> [INFO] ------------------------------------------------------------- > >> [WARNING] COMPILATION WARNING : > >> [INFO] ------------------------------------------------------------- > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[120,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/StreamingContainerUmbilicalProtocol.java:[37,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/StreamingContainerUmbilicalProtocol.java:[38,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/StreamingContainerUmbilicalProtocol.java:[38,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PTOperator.java:[44,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/OperatorStatus.java:[29,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/OperatorStatus.java:[30,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/OperatorStatus.java:[30,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[67,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[68,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[68,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[69,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[69,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[70,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[70,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/RequestFactory.java:[26,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/RequestFactory.java:[26,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[70,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[71,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[71,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/OperatorContext.java:[33,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/OperatorContext.java:[33,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/cli/ApexCli.java:[146,16] > >> sun.misc.Signal is internal proprietary API and may be removed in a > future > >> release > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/cli/ApexCli.java:[147,16] > >> sun.misc.SignalHandler is internal proprietary API and may be removed > in a > >> future release > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/org/apache/apex/engine/EmbeddedAppLauncherImpl.java:[32,27] > >> com.datatorrent.api.LocalMode in com.datatorrent.api has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StramLocalCluster.java:[45,27] > >> com.datatorrent.api.LocalMode in com.datatorrent.api has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StramLocalCluster.java:[45,27] > >> com.datatorrent.api.LocalMode in com.datatorrent.api has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java:[63,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java:[64,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java:[64,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/StramToNodeGetPropertyRequest.java:[33,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/org/apache/apex/engine/api/plugin/DAGExecutionPlugin.java:[29,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/org/apache/apex/engine/api/plugin/DAGExecutionPlugin.java:[29,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/org/apache/apex/engine/plugin/AbstractDAGExecutionPluginContext.java:[32,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/StramToNodeSetPropertyRequest.java:[30,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Slider.java:[31,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/OperatorResponse.java:[23,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/debug/TupleRecorderCollection.java:[41,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/debug/TupleRecorderCollection.java:[42,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/debug/TupleRecorderCollection.java:[42,27] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/OperatorDeployInfo.java:[78,15] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/OperatorDeployInfo.java:[199,17] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/OperatorDeployInfo.java:[270,17] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/BaseContext.java:[70,15] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/api/StreamingContainerUmbilicalProtocol.java:[188,22] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java:[1523,19] > >> > <T>setAttribute(com.datatorrent.api.Operator,com.datatorrent.api.Attribute<T>,T) > >> in com.datatorrent.api.DAG has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java:[230,15] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java:[309,17] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java:[453,17] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java:[925,17] > >> setCounters(java.lang.Object) in com.datatorrent.api.Context has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PTOperator.java:[258,25] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerAgent.java:[465,11] > >> getSchemePrefix(org.apache.hadoop.yarn.conf.YarnConfiguration) in > >> com.datatorrent.stram.util.ConfigUtils has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/OperatorStatus.java:[85,38] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/OperatorStatus.java:[86,15] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/OperatorStatus.java:[160,15] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[210,21] > >> processStats(com.datatorrent.api.StatsListener.BatchedOperatorStats) in > >> com.datatorrent.api.StatsListener has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[212,16] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[212,47] > >> processStats(com.datatorrent.api.StatsListener.BatchedOperatorStats) in > >> com.datatorrent.api.StatsListener has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[218,7] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[218,33] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[231,73] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[242,21] > >> processStats(com.datatorrent.api.StatsListener.BatchedOperatorStats) in > >> com.datatorrent.api.StatsListener has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[234,19] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[236,50] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[244,22] > >> processStats(com.datatorrent.api.StatsListener.BatchedOperatorStats) in > >> com.datatorrent.api.StatsListener has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[250,22] > >> processStats(com.datatorrent.api.StatsListener.BatchedOperatorStats) in > >> com.datatorrent.api.StatsListener has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[254,67] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[815,16] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[820,12] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[825,52] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java:[1845,13] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[418,22] > >> SystemClock() in org.apache.hadoop.yarn.util.SystemClock has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[438,18] > >> SystemClock() in org.apache.hadoop.yarn.util.SystemClock has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[492,35] > >> getSchemePrefix(org.apache.hadoop.yarn.conf.YarnConfiguration) in > >> com.datatorrent.stram.util.ConfigUtils has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[497,69] > >> getMemory() in org.apache.hadoop.yarn.api.records.Resource has been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[508,42] > >> getSchemePrefix(org.apache.hadoop.yarn.conf.YarnConfiguration) in > >> com.datatorrent.stram.util.ConfigUtils has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[1048,9] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[1557,14] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[1720,62] > >> COUNTERS_AGGREGATOR in com.datatorrent.api.Context.OperatorContext has > been > >> deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[1721,44] > >> counters in com.datatorrent.api.Stats.OperatorStats has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[2593,74] > >> counters in com.datatorrent.api.Stats.OperatorStats has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java:[3142,72] > >> SystemClock() in org.apache.hadoop.yarn.util.SystemClock has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[129,30] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[356,16] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[409,7] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[485,43] > >> com.datatorrent.api.Operator.CheckpointListener in > >> com.datatorrent.api.Operator has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[486,23] > >> com.datatorrent.api.Operator.CheckpointListener in > >> com.datatorrent.api.Operator has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[572,37] > >> com.datatorrent.api.Operator.CheckpointListener in > >> com.datatorrent.api.Operator has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[573,17] > >> com.datatorrent.api.Operator.CheckpointListener in > >> com.datatorrent.api.Operator has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[636,16] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[657,16] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[659,75] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[666,51] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/Node.java:[667,66] > >> com.datatorrent.api.StatsListener in com.datatorrent.api has been > deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/OperatorContext.java:[147,10] > >> counters in com.datatorrent.api.Stats.OperatorStats has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/OperatorContext.java:[147,26] > >> counters in com.datatorrent.stram.api.BaseContext has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/engine/OperatorContext.java:[148,9] > >> counters in com.datatorrent.stram.api.BaseContext has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/ClassPathResolvers.java:[171,20] > >> writeStringToFile(java.io.File,java.lang.String,boolean) in > >> org.apache.commons.io.FileUtils has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/ClassPathResolvers.java:[193,16] > >> copy(java.io.InputStream,java.io.Writer) in > org.apache.commons.io.IOUtils > >> has been deprecated > >> [WARNING] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/ClassPathResolvers.java:[198,21] > >> readFileToString(java.io.File) in org.apache.commons.io.FileUtils has > been > >> deprecated > >> [INFO] 100 warnings > >> [INFO] ------------------------------------------------------------- > >> [INFO] ------------------------------------------------------------- > >> [ERROR] COMPILATION ERROR : > >> [INFO] ------------------------------------------------------------- > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java:[166,94] > >> cannot find symbol > >> symbol: variable DELEGATION_TOKEN_RENEW_INTERVAL_DEFAULT > >> location: class org.apache.hadoop.yarn.conf.YarnConfiguration > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java:[167,87] > >> cannot find symbol > >> symbol: variable DELEGATION_TOKEN_MAX_LIFETIME_DEFAULT > >> location: class org.apache.hadoop.yarn.conf.YarnConfiguration > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramAppLauncher.java:[615,120] > >> cannot find symbol > >> symbol: variable DELEGATION_TOKEN_MAX_LIFETIME_KEY > >> location: class org.apache.hadoop.yarn.conf.YarnConfiguration > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramAppLauncher.java:[615,173] > >> cannot find symbol > >> symbol: variable DELEGATION_TOKEN_MAX_LIFETIME_DEFAULT > >> location: class org.apache.hadoop.yarn.conf.YarnConfiguration > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramAppLauncher.java:[618,123] > >> cannot find symbol > >> symbol: variable DELEGATION_TOKEN_RENEW_INTERVAL_KEY > >> location: class org.apache.hadoop.yarn.conf.YarnConfiguration > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramAppLauncher.java:[618,178] > >> cannot find symbol > >> symbol: variable DELEGATION_TOKEN_RENEW_INTERVAL_DEFAULT > >> location: class org.apache.hadoop.yarn.conf.YarnConfiguration > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramClientUtils.java:[831,71] > >> cannot find symbol > >> symbol: variable HADOOP_SSL_ENABLED_KEY > >> location: class org.apache.hadoop.fs.CommonConfigurationKeysPublic > >> [ERROR] > >> > /Users/mbossert/IdeaProjects/apex-core/engine/src/main/java/com/datatorrent/stram/client/StramClientUtils.java:[831,125] > >> cannot find symbol > >> symbol: variable HADOOP_SSL_ENABLED_DEFAULT > >> location: class org.apache.hadoop.fs.CommonConfigurationKeysPublic > >> > >> On Sun, Jan 13, 2019 at 6:03 PM Aaron Bossert <aa...@punchcyber.com> > >> wrote: > >> > >>> Thanks! > >>> > >>> Sent from my iPhone > >>> > >>> > On Jan 13, 2019, at 18:00, Thomas Weise <t...@apache.org> wrote: > >>> > > >>> > For the upgrade it is probably best when you change the version > number > >>> in > >>> > [1] and then work through the build issues. > >>> > > >>> > I would expect a number of dependency changes since the client > >>> classpath > >>> > now has the Hadoop transitive dependencies shaded away [2]. > >>> > > >>> > Regarding the development process, see [3]. > >>> > > >>> > Thomas > >>> > > >>> > [1] > >>> > > >>> > https://github.com/apache/apex-core/blob/1f5425dc5ac1d3452a3b37db0e4c61485af9c6e1/pom.xml#L109 > >>> > [2] https://hadoop.apache.org/docs/r3.0.0/ > >>> > [3] http://apex.apache.org/contributing.html > >>> > > >>> > > >>> > > >>> >> On Fri, Jan 11, 2019 at 5:14 PM Aaron Bossert <aa...@punchcyber.com > > > >>> wrote: > >>> >> > >>> >> at first blush, I would think, that the YARN node tagging and native > >>> Docker > >>> >> support would be particularly intriguing...the GPU node support > would > >>> be a > >>> >> good implementation trick to be able to put operators that do deep > >>> learning > >>> >> tasks... > >>> >> > >>> >> On Fri, Jan 11, 2019 at 2:42 PM Pramod Immaneni < > >>> pramod.imman...@gmail.com > >>> >>> > >>> >> wrote: > >>> >> > >>> >>> I agree we should just try changing the dependency first. I am in > >>> favor > >>> >> of > >>> >>> upgrading. It will be good to explore if and how we can leverage > >>> some of > >>> >>> the new features in 3.x. > >>> >>> > >>> >>>> On Fri, Jan 11, 2019 at 11:38 AM Vlad Rozov <vro...@apache.org> > >>> wrote: > >>> >>>> > >>> >>>> Hi Aaron, > >>> >>>> > >>> >>>> I’d recommend to start with upgrading Hadoop dependency to 3.x, > >>> >>>> recompiling and running unit test. Also, before you put > significant > >>> >>> effort > >>> >>>> into Hadoop 3.x upgrade, let’s discuss and vote on dev@apex. Once > >>> >>>> everyone agrees and vote passes, please open JIRA for tracking the > >>> >>> changes. > >>> >>>> > >>> >>>> Thank you, > >>> >>>> > >>> >>>> Vlad > >>> >>>> > >>> >>>>> On Jan 10, 2019, at 07:58, Aaron Bossert <aa...@punchcyber.com> > >>> >> wrote: > >>> >>>>> > >>> >>>>> As requested by Thomas, I am hoping we can kick off a discussion > >>> >> about > >>> >>> a > >>> >>>>> short-term goal of adding support for Hadoop 3.X, though, if I am > >>> not > >>> >>>>> mistaken, there was a pretty large change from 3.0 to 3.1...but > >>> will > >>> >>>>> confirm, in which case, the ask is to support 3.1+ specifically. > >>> >>>>> > >>> >>>>> I program in Scala for the most part, but have not problem > >>> switching > >>> >> to > >>> >>>>> pure Java as needed. That being said, I am very unfamiliar with > >>> the > >>> >>> Apex > >>> >>>>> code base and also unsure of the scope and level of effort > required > >>> >> to > >>> >>>> get > >>> >>>>> this done. I look to the group for a few pointers on where to > >>> begin > >>> >> as > >>> >>>>> well as code style and such...I don't think I will need a lot of > >>> >>>>> hand-holding, just a good shove in the right direction... > >>> >>>>> > >>> >>>>> Thanks again for the discussion and support on this...for > >>> background > >>> >>> here > >>> >>>>> is my personal and professional motivation for pressing forward: > >>> >>>>> > >>> >>>>> - Doing research work for clients as well as internal R&D for > >>> >>> streaming > >>> >>>>> data processing > >>> >>>>> - very specific requirement to support the ability to alter > >>> >>>>> topologies programatically and manually on the fly, hence the > >>> >>>> interest in > >>> >>>>> Apex, which is the only platform I know of that has this > >>> support > >>> >>>> built > >>> >>>>> in...I don't want to re-invent that wheel unless absolutely > >>> >>>> necessary > >>> >>>>> - General interest in supporting the open-source community, both > >>> >> as a > >>> >>>>> company and individually > >>> >>>>> > >>> >>>>> -- > >>> >>>>> > >>> >>>>> M. Aaron Bossert > >>> >>>>> (202) 594-1703 > >>> >>>>> Punch Cyber Analytics Group > >>> >>>> > >>> >>>> -- > >>> >>> Thanks, > >>> >>> Pramod > >>> >>> http://ts.la/pramod3443 > >>> >>> > >>> >> > >>> >> > >>> >> -- > >>> >> > >>> >> M. Aaron Bossert > >>> >> (202) 594-1703 > >>> >> Punch Cyber Analytics Group > >>> >> > >>> > >> > >> > >> -- > >> > >> M. Aaron Bossert > >> (202) 594-1703 > >> Punch Cyber Analytics Group > >> > >> > >> > > > > -- > > > > M. Aaron Bossert > > (202) 594-1703 > > Punch Cyber Analytics Group > > > > > > > > -- > > M. Aaron Bossert > (202) 594-1703 > Punch Cyber Analytics Group > -- Thanks, Pramod http://ts.la/pramod3443
