[GitHub] twill pull request #53: (TWILL-230) Get resource report based on the caller ...

2017-04-03 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/53#discussion_r109552373
  
--- Diff: 
twill-yarn/src/main/java/org/apache/twill/yarn/YarnTwillController.java ---
@@ -322,7 +314,46 @@ private boolean hasRun(YarnApplicationState state) {
 
   @Override
   public ResourceReport getResourceReport() {
-// in case the user calls this before starting, return null
+// Only has resource report if the app is running.
+if (state() != State.RUNNING) {
+  return null;
+}
+ResourceReportClient resourcesClient = getResourcesClient();
 return (resourcesClient == null) ? null : resourcesClient.get();
   }
+
+  /**
+   * Returns the {@link ResourceReportClient} for fetching resource report 
from the AM.
+   * It first consults the RM for the tracking URL and get the resource 
report from there.
+   */
+  @Nullable
+  private ResourceReportClient getResourcesClient() {
+YarnApplicationReport report = processController.getReport();
+List urls = new ArrayList<>(2);
--- End diff --

Why will this url list has initial capacity of 2?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill issue #49: (TWILL-229) Default to use logback_template.xml to configur...

2017-03-30 Thread yaojiefeng
Github user yaojiefeng commented on the issue:

https://github.com/apache/twill/pull/49
  
I see. LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill issue #49: (TWILL-229) Default to use logback_template.xml to configur...

2017-03-30 Thread yaojiefeng
Github user yaojiefeng commented on the issue:

https://github.com/apache/twill/pull/49
  
Changes LGTM, may I ask where do we set the logback if it is overriden by 
user?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #18: Fix reset log level for runnables started up later

2016-12-13 Thread yaojiefeng
GitHub user yaojiefeng opened a pull request:

https://github.com/apache/twill/pull/18

Fix reset log level for runnables started up later

There is a bug when we try to reset log levels for some runnables which are 
started up later, e.g, restart or instance changes, after some runtime log 
level changes. For now, when we start up the `TwillContainerService`, we set 
log levels based on the what we try to set in `TwillPreparer` + runtime log 
levels. But we do not memorize the old log level for loggers changed at 
runtime. So we we reset, we will not get back to the state we want.
This PR fixes this by memorizing the log level changes coming from runtime. 
And remove when we call reset.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yaojiefeng/twill 
fix/remember-the-runtime-log-level-change-during-startup

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/twill/pull/18.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #18


commit ae01446bfd9c02982428b8b25ec7fe40b33c89f6
Author: yaojiefeng <yao...@cask.co>
Date:   2016-12-13T12:03:55Z

fix reset log level for runnables




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill issue #14: TWILL-138 Runtime log level change

2016-12-13 Thread yaojiefeng
Github user yaojiefeng commented on the issue:

https://github.com/apache/twill/pull/14
  
Thanks! @hsaputra 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill issue #17: Change initial application submission log message from DEBU...

2016-12-12 Thread yaojiefeng
Github user yaojiefeng commented on the issue:

https://github.com/apache/twill/pull/17
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-12-10 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r91845224
  
--- Diff: twill-api/src/main/java/org/apache/twill/api/TwillController.java 
---
@@ -89,4 +90,45 @@
* @return A {@link Future} that will be completed when the restart 
operation has been done.
*/
   Future restartInstances(String runnable, int instanceId, int... 
moreInstanceIds);
+
+  /**
+   * Set the log levels for requested logger names for Twill applications 
running in a container.
+   * The log level for a logger name can be {@code null} except for the 
root logger, which will reset the log level for
+   * the specified logger.
+   *
+   * @param logLevels The {@link Map} contains the requested logger names 
and log levels that need to be set.
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done. It will carry the
+   * {@link Map} of log levels as the result.
+   */
+  Future<Map<String, LogEntry.Level>> setLogLevels(Map<String, 
LogEntry.Level> logLevels);
+
+  /**
+   * Set the log levels for requested logger names for a {@link 
TwillRunnable}.
+   * The log level for a logger name can be {@code null} except for the 
root logger, which will reset the log level for
+   * the specified logger.
+   *
+   * @param runnableName The name of the runnable to set the log level.
+   * @param logLevelsForRunnable The {@link Map} contains the requested 
logger name and log level that
+   * need to be changed.
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done. It will carry the
+   * {@link Map} of log levels as the result.
+   */
+  Future<Map<String, LogEntry.Level>> setLogLevels(String runnableName,
+   Map<String, 
LogEntry.Level> logLevelsForRunnable);
+
+  /**
+   * Reset the log levels of all runnables to the default log level, 
{@code LogEntry.Level.INFO}.
+   *
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done.
+   */
+  Future resetLogLevels();
--- End diff --

Changed the value in `Future` to be the loggerNames passed through this 
methods. Empty means resetting for all loggers


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-12-03 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r90767592
  
--- Diff: 
twill-yarn/src/main/java/org/apache/twill/internal/container/TwillContainerService.java
 ---
@@ -51,29 +63,36 @@
   private final ClassLoader classLoader;
   private final BasicTwillContext context;
   private final ContainerLiveNodeData containerLiveNodeData;
+  private final Map<String, LogEntry.Level> oldLogLevels;
+  private final Map<String, LogEntry.Level> defaultLogLevels;
   private ExecutorService commandExecutor;
   private TwillRunnable runnable;
 
   public TwillContainerService(BasicTwillContext context, ContainerInfo 
containerInfo, ZKClient zkClient,
RunId runId, TwillRunnableSpecification 
specification, ClassLoader classLoader,
-   Location applicationLocation) {
+   Location applicationLocation, Map<String, 
LogEntry.Level> defaultLogLevels,
+   Map<String, LogEntry.Level> logLevels) {
 super(zkClient, runId, applicationLocation);
 
 this.specification = specification;
 this.classLoader = classLoader;
-this.containerLiveNodeData = createLiveNodeData(containerInfo);
+this.defaultLogLevels = ImmutableMap.copyOf(defaultLogLevels);
+this.oldLogLevels = Collections.synchronizedMap(new 
HashMap<>(defaultLogLevels));
--- End diff --

When you reset, you cannot reset the log level for a particular logger. It 
will reset the log levels for all loggers which are changed during runtime. So 
if you call reset, it will reset the log level of ROOT to INFO, and the 
particular logger to the log level when the runnable starts. If it was null 
initially, its log level will be following the ROOT log level, INFO


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-12-03 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r90767420
  
--- Diff: twill-api/src/main/java/org/apache/twill/api/TwillController.java 
---
@@ -89,4 +90,45 @@
* @return A {@link Future} that will be completed when the restart 
operation has been done.
*/
   Future restartInstances(String runnable, int instanceId, int... 
moreInstanceIds);
+
+  /**
+   * Set the log levels for requested logger names for Twill applications 
running in a container.
+   * The log level for a logger name can be {@code null} except for the 
root logger, which will reset the log level for
+   * the specified logger.
+   *
+   * @param logLevels The {@link Map} contains the requested logger names 
and log levels that need to be set.
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done. It will carry the
+   * {@link Map} of log levels as the result.
+   */
+  Future<Map<String, LogEntry.Level>> setLogLevels(Map<String, 
LogEntry.Level> logLevels);
+
+  /**
+   * Set the log levels for requested logger names for a {@link 
TwillRunnable}.
+   * The log level for a logger name can be {@code null} except for the 
root logger, which will reset the log level for
+   * the specified logger.
+   *
+   * @param runnableName The name of the runnable to set the log level.
+   * @param logLevelsForRunnable The {@link Map} contains the requested 
logger name and log level that
+   * need to be changed.
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done. It will carry the
+   * {@link Map} of log levels as the result.
+   */
+  Future<Map<String, LogEntry.Level>> setLogLevels(String runnableName,
+   Map<String, 
LogEntry.Level> logLevelsForRunnable);
+
+  /**
+   * Reset the log levels of all runnables to the default log level, 
{@code LogEntry.Level.INFO}.
+   *
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done.
+   */
+  Future resetLogLevels();
+
+  /**
+   * Reset the log levels of the given runnable to the log levels when .
--- End diff --

My bad for not updating the comments. The `resetLogLevels` will reset the 
log level of a runnable to the time when it starts up. This means it will reset 
the log level to what `logback.xml` is + the log level requested from 
`TwillPreparer`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-11-14 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r87877029
  
--- Diff: twill-api/src/main/java/org/apache/twill/api/TwillController.java 
---
@@ -89,4 +90,30 @@
* @return A {@link Future} that will be completed when the restart 
operation has been done.
*/
   Future restartInstances(String runnable, int instanceId, int... 
moreInstanceIds);
+
+  /**
+   * Set the log levels for requested logger names for Twill applications 
running in a container.
+   * The log level for a logger name can be {@code null} except for the 
root logger, which will reset the log level for
--- End diff --

We check it using `Preconditions.checkArgument`, so an exception will be 
thrown for null log level for root logger.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-11-09 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r87254498
  
--- Diff: 
twill-yarn/src/main/java/org/apache/twill/internal/ServiceMain.java ---
@@ -264,6 +262,10 @@ protected String getLoggerLevel(Logger logger) {
 return "OFF";
   }
 
+  protected String getLogLevels() {
+return "";
--- End diff --

For a launch, the `TwillContainerService` is constructed before the run of 
`doMain` method, which means the we have not called `configureLogger()` method 
at that time so log level setting at constructor will not work. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-11-03 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r86290796
  
--- Diff: 
twill-yarn/src/test/java/org/apache/twill/yarn/LogLevelChangeTestRun.java ---
@@ -0,0 +1,249 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.twill.yarn;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.ImmutableMap;
+import org.apache.twill.api.AbstractTwillRunnable;
+import org.apache.twill.api.ResourceReport;
+import org.apache.twill.api.TwillApplication;
+import org.apache.twill.api.TwillController;
+import org.apache.twill.api.TwillRunResources;
+import org.apache.twill.api.TwillSpecification;
+import org.apache.twill.api.logging.LogEntry;
+import org.apache.twill.api.logging.PrinterLogHandler;
+import org.apache.twill.common.Threads;
+import org.junit.Assert;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.PrintWriter;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Test changing log level for a twill runnable.
+ */
+public class LogLevelChangeTestRun extends BaseYarnTest {
+  public static final Logger LOG = 
LoggerFactory.getLogger(LogLevelChangeTestRun.class);
+
+  /**
+   * Twill runnable.
+   */
+  public static final class LogLevelTestRunnable extends 
AbstractTwillRunnable {
+public static final Logger LOG = 
LoggerFactory.getLogger(LogLevelChangeTestRun.LogLevelTestRunnable.class);
+
+private volatile Thread runThread;
+
+@Override
+public void run() {
+  this.runThread = Thread.currentThread();
+
+  // check if the initial log level is DEBUG
+  Assert.assertTrue(LOG.isDebugEnabled() && !LOG.isTraceEnabled());
+
+  int i = 0;
+  while (!Thread.interrupted()) {
+if (i == 0 && !LOG.isDebugEnabled()) {
+  // check if the log level is changed to INFO
+  Assert.assertTrue(LOG.isInfoEnabled());
+  i++;
+}
+if (i == 1 && !LOG.isInfoEnabled()) {
+  // check if the log level is changed to WARN
+  Assert.assertTrue(LOG.isWarnEnabled());
+  i++;
+}
+
+try {
+  TimeUnit.MILLISECONDS.sleep(100);
+} catch (InterruptedException e) {
+  break;
+}
+  }
+}
+
+@Override
+public void stop() {
+  if (runThread != null) {
+runThread.interrupt();
+  }
+}
+  }
+
+  /**
+   * Second runnable.
+   */
+  public static final class LogLevelTestSecondRunnable extends 
AbstractTwillRunnable {
+public static final Logger LOG = 
LoggerFactory.getLogger(LogLevelChangeTestRun.LogLevelTestSecondRunnable.class);
+
+private volatile Thread runThread;
+
+@Override
+public void run() {
+  this.runThread = Thread.currentThread();
+
+  // check if the initial log level is DEBUG
+  Assert.assertTrue(LOG.isDebugEnabled() && !LOG.isTraceEnabled());
+
+  int i = 0;
+  while (!Thread.interrupted()) {
+if (i == 0 && !LOG.isDebugEnabled()) {
+  // check if the log level is changed to INFO
+  Assert.assertTrue(LOG.isInfoEnabled());
+  i++;
+}
+if (i == 1 && LOG.isDebugEnabled()) {
+  // check if the log level is changed to TRACE
+  Assert.assertTrue(LOG.isTraceEnabled());
+  i++;
+}
+
+try {
+  TimeUnit.MILLISECONDS.sleep(100);
+} catch (InterruptedException e) {
+  break;
+}
+  }
+}
+
+@Override
+public void stop() {

[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-11-02 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r86288660
  
--- Diff: 
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
 ---
@@ -475,6 +496,11 @@ private void sendMessage(final String runnableName, 
final Message message,
 Futures.addCallback(controller.sendMessage(message), new 
FutureCallback() {
   @Override
   public void onSuccess(Message result) {
+//update log level information
--- End diff --

onSuccess here means that the message has been processed by the 
TwillContainerService, and the log levels have been set up. If there is some 
error setting up the log levels, an failed future with exception will be 
returned to callback, so it will go to onFailure() method


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-11-02 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r86288445
  
--- Diff: 
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
 ---
@@ -575,18 +601,74 @@ private boolean removeContainerInfo(String 
containerId) {
 return false;
   }
 
+  private Map<String, Map<String, LogEntry.Level>> 
copyLogLevels(Map<String, Map<String, LogEntry.Level>> originalMap) {
+Map<String, Map<String, LogEntry.Level>> result = new TreeMap<>();
+for (Map.Entry<String, Map<String, LogEntry.Level>> entry : 
originalMap.entrySet()) {
+  Map<String, LogEntry.Level> value = new TreeMap<>();
+  for (Map.Entry<String, LogEntry.Level> valueEntry : 
entry.getValue().entrySet()) {
+value.put(valueEntry.getKey(), valueEntry.getValue());
+  }
+  result.put(entry.getKey(), value);
+}
+return result;
+  }
+
+  private void checkAndSetLogLevels(Message message, String runnableName) {
+if (message.getType() != Message.Type.SYSTEM
+  || 
!SystemMessages.LOG_LEVEL.equals(message.getCommand().getCommand())) {
+  return;
+}
+
+Map<String, LogEntry.Level> runnableLogLevels = 
convertLogLevelValuesToLogEntry(message.getCommand().getOptions());
+if (!logLevels.containsKey(runnableName)) {
+  logLevels.put(runnableName, runnableLogLevels);
+} else {
+  logLevels.get(runnableName).putAll(runnableLogLevels);
+}
+  }
+
+  private Location saveLogLevels() {
--- End diff --

We save the log levels to file because we have no way to know if there is 
message about log level change come before the launch of some containers(this 
can happen when we change instance count or restart the container). So when a 
message about log level change come, we will have it stored in the Map in 
`RunningContainers`. And if there is a container launch afterwards, we will 
check if there is update in the log levels, and localize it for the container 
if necessary.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-11-02 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r86282861
  
--- Diff: twill-api/src/main/java/org/apache/twill/api/TwillController.java 
---
@@ -89,4 +90,44 @@
* @return A {@link Future} that will be completed when the restart 
operation has been done.
*/
   Future restartInstances(String runnable, int instanceId, int... 
moreInstanceIds);
+
+  /**
+   * Set the root log level for Twill applications in all containers.
+   *
+   * @param logLevel The log level for the root logger to change.
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done. It will carry
+   * the log level as a result.
+   */
+  Future setRootLogLevel(LogEntry.Level logLevel);
+
+  /**
+   * Set the root log level for a particular runnable.
+   *
+   * @param runnableName The name of the runnable to set the log level.
+   * @param logLevel The log level for the root logger to change.
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done. It will carry
+   * the log level as a result.
+   */
+  Future setRootLogLevel(String runnableName, 
LogEntry.Level logLevel);
+
+  /**
+   * Set the log levels for requested logger names for Twill applications 
running in a container.
+   *
+   * @param logLevels The {@link Map} contains the requested logger names 
and log levels that need to be set.
+   * @return A {@link Future} that will be completed when the set log 
level operation has been done. It will carry the
+   * {@link Map} of log levels as the result.
+   */
+  Future<Map<String, LogEntry.Level>> setLogLevels(Map<String, 
LogEntry.Level> logLevels);
--- End diff --

Yes, you can set the root log level by calling like:
`setLogLevels(ImmutableMap.of("ROOT", TRACE))`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-10-11 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r82886998
  
--- Diff: 
twill-core/src/main/java/org/apache/twill/internal/utils/LogLevelUtil.java ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.twill.internal.utils;
+
+import com.google.common.collect.Maps;
+import org.apache.twill.api.logging.LogEntry;
+import org.apache.twill.internal.Constants;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import javax.annotation.Nullable;
+
+/**
+ * Collections of helper functions for log level change.
+ */
+public final class LogLevelUtil {
+
+  /**
+   * Convert the log level argument value type to string.
+   */
+  public static Map<String, String> 
convertLogLevelValuesToString(Map<String, LogEntry.Level> logLevelArguments) {
--- End diff --

The methods in `LogLevelUtil` are not only used in `SystemMessages`. They 
are also used in `RunningContainers` and `TwillContainerMain`, is it good to 
have all these methods in `SystemMessages`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #14: TWILL-138 Runtime log level change

2016-10-05 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/14#discussion_r82102988
  
--- Diff: 
twill-core/src/main/java/org/apache/twill/internal/AbstractTwillController.java 
---
@@ -137,7 +138,7 @@ public final synchronized ServiceDiscovered 
discoverService(String serviceName)
 
   @Override
   public final ListenableFuture<Set> restartInstances(Map<String,
-  ? extends Set> runnableToInstanceIds) {
+? extends Set> runnableToInstanceIds) {
--- End diff --

I cannot change the type to Future for this method, there is a another 
method calling it which requires the return type to be a ListenableFuture


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #13: TWILL-195 Add TwillRuntimeSpecification

2016-10-05 Thread yaojiefeng
Github user yaojiefeng commented on a diff in the pull request:

https://github.com/apache/twill/pull/13#discussion_r82036739
  
--- Diff: 
twill-common/src/main/java/org/apache/twill/internal/Constants.java ---
@@ -78,6 +78,15 @@ private Files() {
 }
   }
 
+  /**
+   * Constants for twill environment variable names.
+   */
+  public static final class Environments {
--- End diff --

They are initially used in EnvKeys to pass system env variables. But with 
the use of TwillRuntimeSpecification, they are no longer used as keys. They are 
used as constants in configuration or error messages so I move them to 
Constants class


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #13: TWILL-195 Add TwillRuntimeSpecification

2016-09-28 Thread yaojiefeng
GitHub user yaojiefeng opened a pull request:

https://github.com/apache/twill/pull/13

TWILL-195 Add TwillRuntimeSpecification

JIRA: https://issues.apache.org/jira/browse/TWILL-195

Summary: This PR is a prerequisite of TWILL-138. Add 
TwillRuntimeSpecification for twill application which includes 
TwillSpecification and other environment variables that the applications will 
need later.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yaojiefeng/twill 
feature/add-TwillRuntimeSpecification

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/twill/pull/13.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #13


commit 19a1971eaca72d1d576bd252be364c7be6aebcc4
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-22T21:35:04Z

add TwillRuntimeSpecification

commit d64abe694cda37860b6fd5ff6e4641bface3cf42
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-23T01:04:42Z

remove some env keys

commit 4c4a102074332e66f4bf7b15f6beddfe750af173
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-29T01:13:11Z

add javadoc




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] twill pull request #12: TWILL-138 TWILL-195 Runtime log level change

2016-09-27 Thread yaojiefeng
GitHub user yaojiefeng opened a pull request:

https://github.com/apache/twill/pull/12

TWILL-138 TWILL-195 Runtime log level change

JIRA: https://issues.apache.org/jira/browse/TWILL-138
https://issues.apache.org/jira/browse/TWILL-195

The first two commits are for TWILL-195, the rest are for TWILL-138.
 
Summary: Make API changes to TwillController and TwillPreparer to set the 
log level for the twill application or specific twill runnables at runtime or 
before the application starts. The TwillPreparer will serialize the log level 
change requirements and deserialize them once service starts. The 
TwillController will use messages to send out log level change request. Details 
for implementation can be found in JIRAs.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yaojiefeng/twill feature/changeLogLevel

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/twill/pull/12.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12


commit 19a1971eaca72d1d576bd252be364c7be6aebcc4
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-22T21:35:04Z

add TwillRuntimeSpecification

commit d64abe694cda37860b6fd5ff6e4641bface3cf42
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-23T01:04:42Z

remove some env keys

commit e72746cac4981c0a40edef60dedb542c8d279190
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-23T07:53:50Z

Log level API change for TwillController

commit 7d497b8c63689128e985343c37c21b766b7d85db
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-24T00:02:06Z

Log level API change for TwillPreparer

commit 3b89cba28182414c788f8fe1ede2761c2ee4778b
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-25T08:47:15Z

Remove unused imports, add javadoc, and nit

commit 2c005ec7b7cd14b620753e9c62374c05673dab41
Author: yaojiefeng <yao...@cask.co>
Date:   2016-09-25T09:25:26Z

update the unit test




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---