[
https://issues.apache.org/jira/browse/SLIDER-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194747#comment-15194747
]
kyungwan nam commented on SLIDER-1055:
--------------------------------------
{quote}
I don't exactly understand why we need the new ProcessTree implementation. Can
you comment on why it was needed and what your implementation did differently?
{quote}
the ProcfsBasedProcessTree is constructing process-tree for the decedents of
the main container process.
therefore, daemon process (ppid=1) can’t be included in process-tree. as a
result it will be excepted from monitoring .
even though daemon process ppid will be 1, it's pgrpid will be the same as the
main container process.
as follows, processes belong to the same process group (29559).
{code}
PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
29556 29559 29559 29559 ? -1 Ss 500 0:00 \_ /bin/bash -c
python ./infra/agent/slider-agent/agent/main.py --label
container_e07_1451897008090_0009_01_000002___HBASE_MASTER --zk-quorum
zk01.com:2181, zk01.com:2181,zk03.com:2181 --zk-reg-path
/registry/users/yarn/services/org-apache-slider/hbase1 >
/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_000002/slider-agent.out
2>&1
29559 29564 29559 29559 ? -1 Sl 500 99:43 \_ python
./infra/agent/slider-agent/agent/main.py --label
container_e07_1451897008090_0009_01_000002___HBASE_MASTER --zk-quorum
zk01.com:2181, zk01.com:2181,zk03.com:2181 --zk-reg-path
/registry/users/yarn/services/org-apache-slider/hbase1
1 29779 29559 29559 ? -1 S 500 0:00 bash
/volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_000002/app/install/hbase-0.98.13-hadoop2/bin/hbase-daemon.sh
--config
/volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_000002/app/install/hbase-0.98.13-hadoop2/conf
foreground_start master
29779 29794 29559 29559 ? -1 Sl 500 62:56 \_
/jdk-1.7.0_45/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p
-Xmx1000m -XX:+UseConcMarkSweepGC
-XX:ErrorFile=/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_000002/hs_err_pid%p.log
-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-Xloggc:/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_000002/gc.log-201601121408
-Xmx1024m
-Dhbase.log.dir=/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_000002
-Dhbase.log.file=hbase-yarn-master-csw024.log
-Dhbase.home.dir=/volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_000002/app/install/hbase-0.98.13-hadoop2/bin/..
-Dhbase.id.str=yarn -Dhbase.root.logger=INFO,RFA
-Djava.library.path=/package/hadoop-yarn-2.7.1-c3-20151026-arch-centos6-x86_64/lib/native
-Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.master.HMaster start
{code}
in a new class, as well as the decedents, processes whose pgrpid is the same as
the pgrpid of the main container process are added to process-tree.
my concern is that daemon process would be decoupled from the agent process
lifecycle as Steve Loughran said.
but it seems that all processes in process group would be killed when the
container is killed by YARN (HADOOP-11989)
I am not familiar with the internal for the YARN. so, i don’t know this
approach is available.
but if it makes sense, there’s no limitation to execute process.
please correct me if I’m wrong.
and, I agree above 1. approach looks clear. [~Tao Jie], thanks for the patch.
thanks
> hbase-daemon executed by slider is excepted from nodemanager container
> monitoring
> ---------------------------------------------------------------------------------
>
> Key: SLIDER-1055
> URL: https://issues.apache.org/jira/browse/SLIDER-1055
> Project: Slider
> Issue Type: Bug
> Components: application/hbase
> Affects Versions: Slider 0.81
> Reporter: kyungwan nam
> Attachments: SLIDER-1055.001.patch
>
>
> here is nodemanager log of a host where a HBASE_REGIONSERVER component is
> running
> {code}
> 2016-01-12 14:11:49,237 DEBUG monitor.ContainersMonitorImpl
> (ContainersMonitorImpl.java:run(361)) - Current ProcessTree list : [ 9801 ]
> 2016-01-12 14:11:49,237 DEBUG monitor.ContainersMonitorImpl
> (ContainersMonitorImpl.java:run(436)) - Constructing ProcessTree for : PID =
> 9801 ContainerId = container_e07_1451897008090_0009_01_000003
> 2016-01-12 14:11:49,262 DEBUG util.ProcfsBasedProcessTree
> (ProcfsBasedProcessTree.java:updateProcessTree(274)) - [ 9801 9806 ]
> 2016-01-12 14:11:49,262 INFO monitor.ContainersMonitorImpl
> (ContainersMonitorImpl.java:run(458)) - Memory usage of ProcessTree 9801 for
> container-id container_e07_1451897008090_0009_01_000003: 14.2 MB of 1 GB
> physical memory used; 517.1 MB of 2.1 GB virtual memory used
> {code}
> used memory for the container is lower than i expected.
> because pids ( 9801 9806 ) are slider-agent process. regionserver process was
> excepted from monitoring.
> here is the result of "ps axjf"
> {code}
> 9798 9801 9801 9801 ? -1 Ss 500 0:00 \_ /bin/bash -c
> python ./infra/agent/slider-agent/agent/main.py --label
> container_e07_1451897008090_0009_01_000003___HBASE_REGIONSERVER --zk-quorum
> 9801 9806 9801 9801 ? -1 Sl 500 0:01 \_ python
> ./infra/agent/slider-agent/agent/main.py --label
> container_e07_1451897008090_0009_01_000003___HBASE_REGIONSERVER --zk-quorum
> 1 9979 9801 9801 ? -1 S 500 0:00 bash
> /volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_000003/app/install/hbase-0.98.13-hadoop2/bin/hbase-daemon.sh
> --config
> /volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_000003/app/install/hbase-0.98.13-hadoop2/conf
> foreground_start regionserver
> 9979 9994 9801 9801 ? -1 Sl 500 0:10 \_
> /package/jdk-1.7.0_45/bin/java -Dproc_regionserver
> -XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -XX:+UseConcMarkSweepGC
> -XX:ErrorFile=/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_000003/hs_err_pid%p.log
> -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
> -Xloggc:/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_000003/gc.log-201601121408
> -Xmn200m -XX:CMSInitiatingOccupancyFraction=70 -Xms1024m -Xmx1024m
> -Dhbase.log.dir=/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_000003
> -Dhbase.log.file=hbase-yarn-regionserver.log
> -Dhbase.home.dir=/volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_000003/app/install/hbase-0.98.13-hadoop2/bin/..
> -Dhbase.id.str=yarn -Dhbase.root.logger=INFO,RFA
> -Djava.library.path=/package/hadoop-yarn-2.7.1-arch-centos6-x86_64/lib/native
> -Dhbase.security.logger=INFO,RFAS
> org.apache.hadoop.hbase.regionserver.HRegionServer start
> {code}
> when i use the ProcfsBasedProcessTree (default)
> process-tree is determined by relationship between parent and child process.
> so, daemonized process (ppid=1) can’t be included in process-tree.
> I don't know it can be fixed in slider.
> does it need to implement another ResourceCalculatorProcessTree to replace
> the ProcfsBasedProcessTree?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
