-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27396/#review59598
-----------------------------------------------------------



ambari-agent/src/main/python/ambari_agent/loghandler/config.py
<https://reviews.apache.org/r/27396/#comment100891>

    You should be using an AmbariConfig instance and create a section in that 
file to handle setting all these.  AmbariConfig is the "external property file" 
for things like this.



ambari-agent/src/main/python/ambari_agent/loghandler/hdfsapi.py
<https://reviews.apache.org/r/27396/#comment100896>

    We may have an HdfsExecute common library for things like this



ambari-agent/src/main/python/ambari_agent/loghandler/monitor.py
<https://reviews.apache.org/r/27396/#comment100889>

    This style is generally deprecated for the string.format() method.



ambari-agent/src/main/python/ambari_agent/loghandler/monitor.py
<https://reviews.apache.org/r/27396/#comment100899>

    Use the "with" syntax.



ambari-agent/src/main/python/ambari_agent/loghandler/monitor.py
<https://reviews.apache.org/r/27396/#comment100890>

    No error handling here can leave an open file.  Use this style and the file 
won't be left hanging.
    
    with open(self.__metadata_path, 'w') as file:
      file.write(...)



ambari-agent/src/main/python/ambari_agent/loghandler/monitor.py
<https://reviews.apache.org/r/27396/#comment100893>

    See comment about AmbariConfig



ambari-agent/src/main/python/ambari_agent/loghandler/util.py
<https://reviews.apache.org/r/27396/#comment100894>

    All you have is statics.  This doesn't need to be in a class, can just be 
loghandler/util with methods in it.


- Nate Cole


On Nov. 3, 2014, 3:30 p.m., Cabir Zounaidou wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27396/
> -----------------------------------------------------------
> 
> (Updated Nov. 3, 2014, 3:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, John 
> Speidel, Mahadev Konar, Nate Cole, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-1522
>     https://issues.apache.org/jira/browse/AMBARI-1522
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> The idea is to run a scheduler which computes diffs in a component log and 
> pushes the diff (only if available) to HDFS at fixed interval.
> 
> 1. Each component will have its own monitor
> 2. The apscheduler threadpool controls the monitor execution.
> 3. If the log directory is not available if the component (or service) is not 
> running, it will skip the processing.
> 4. It saves the last read line index and last modified time for next iteration
> 5. It uses HDFS shell utility to push log patch to HDFS.
> 6. Right now, the component log directory is configured in a json file.  In 
> the next iteration, it will try to automatically detected from the stack 
> config.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py dc3a1cf 
>   ambari-agent/src/main/python/ambari_agent/LogSchedulerHandler.py 
> PRE-CREATION 
>   ambari-agent/src/main/python/ambari_agent/loghandler/__init__.py 
> PRE-CREATION 
>   ambari-agent/src/main/python/ambari_agent/loghandler/config.py PRE-CREATION 
>   ambari-agent/src/main/python/ambari_agent/loghandler/hdfsapi.py 
> PRE-CREATION 
>   ambari-agent/src/main/python/ambari_agent/loghandler/monitor.py 
> PRE-CREATION 
>   ambari-agent/src/main/python/ambari_agent/loghandler/util.py PRE-CREATION 
>   ambari-agent/src/test/python/ambari_agent/TestLogHandler.py PRE-CREATION 
>   
> ambari-agent/src/test/python/ambari_agent/dummy_files/log_handler_config.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/27396/diff/
> 
> 
> Testing
> -------
> 
> - Ran the ambari-agent with the patch and verified
> 1. Started ambari-agent with the patch 
> 2. The scheduler started successfully
> 3. The logs files are getting pushed to HDFS successfully. Verified using 
> HDFS shell utility.
> - Ran the tests successfully
> [INFO]
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Building Ambari Agent 1.3.0-SNAPSHOT
> [INFO] 
> ------------------------------------------------------------------------
> [INFO]
> :
> ----------------------------------------------------------------------
> Ran 324 tests in 8.282s
> 
> OK
> :
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Total time: 14.759s
> [INFO] Finished at: Thu Oct 30 14:58:21 PDT 2014
> [INFO] Final Memory: 10M/4079M
> [INFO] 
> ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Cabir Zounaidou
> 
>

Reply via email to