I'm doing import params but it doesn't even reach there because it fails on `from resource_management import *` at the top of the file because the pythonpath isnt set.
On Fri, Dec 2, 2016 at 1:09 PM, Gour Saha <gs...@hortonworks.com> wrote: > Can you check your package script stop function if it is doing "import > params" like this - > https://github.com/apache/incubator-slider/blob/develop/ > app-packages/hbase/ > package/scripts/hbase_master.py#L48 > > > If yes, then you might have to share your app-package scripts (without the > app binary/tar), for us to debug further. For that you have to file a bug > and upload it to the bug. Attaching it to the email to this DL will not > work. > > -Gour > > On 12/2/16, 9:30 AM, "Ophir Etzion" <op...@foursquare.com> wrote: > > >1. you can't see the PYTHONPATH issue. you can see there is no setting of > >the PYTHONPATH that you can see in the START command. > >2. thanks for letting me know about release_timeout_secs but for my app I > >don't care if the containers die, the stop command sends an udp packet > >elsewhere. > > > >here is the output for START where you can see the PYTHONPATH being set: > >INFO 2016-11-30 17:50:32,361 AgentToggleLogger.py:40 - Running command > >['/usr/bin/python', > > '-S', > > > >u'/export/hdk3/yarn/nm/usercache/hive/appcache/ > application_1479830316320_6 > >4974/filecache/11/enable_presto_worker.zip/package/ > scripts/enable_presto_w > >orker_component.py', > > u'START', > > > >'/export/hda3/data/log/hadoop-yarn/container/ > application_1479830316320_649 > >74/container_e468_1479830316320_64974_01_000091/command-4.json', > > > >'/export/hdk3/yarn/nm/usercache/hive/appcache/ > application_1479830316320_64 > >974/filecache/11/enable_presto_worker.zip/package', > > > >'/export/hda3/data/log/hadoop-yarn/container/ > application_1479830316320_649 > >74/container_e468_1479830316320_64974_01_000091/structured-out-4.json', > > 'INFO', > > > >'/export/hdj3/yarn/nm/usercache/hive/appcache/ > application_1479830316320_64 > >974/container_e468_1479830316320_64974_01_000091'] > >INFO 2016-11-30 17:50:32,361 AgentToggleLogger.py:40 - Setting env: > >PYTHONPATH to > >/export/hdj3/yarn/nm/usercache/hive/appcache/ > application_1479830316320_649 > >74/filecache/10/slider-agent.tar.gz/slider-agent/jinja2:/ > export/hdj3/yarn/ > >nm/usercache/hive/appcache/application_1479830316320_ > 64974/filecache/10/sl > >ider-agent.tar.gz/slider-agent > >INFO 2016-11-30 17:50:32,463 AgentToggleLogger.py:40 - Queue result: > >{'componentStatus': [], > > 'reports': [{'actionId': u'4-1', > > 'clusterName': u'enable-presto-worker_cluster_a', > > 'exitcode': 777, > > 'reportResult': True, > > 'role': u'NODE', > > 'roleCommand': u'START', > > 'serviceName': u'enable-presto-worker_cluster_a', > > 'status': 'IN_PROGRESS', > > 'stderr': '', > > 'stdout': "2016-11-30 17:50:32,455 - > >Directory['/data/appdata/enable_presto_worker/data/var/run'] > {'recursive': > >True}", > > 'structuredOut': '{}', > > 'taskId': 4}]} > > > >On Fri, Dec 2, 2016 at 11:51 AM, Gour Saha <gs...@hortonworks.com> wrote: > > > >> Also keep in mind - if your application needs to run something useful > >>when > >> the stop cmd is initiated then you need to set an appropriate value to > >> site.global.app_container.release_timeout_secs. Otherwise kill signals > >>are > >> sent to the agent containers via YARN (almost immediately) and the > >> containers don¹t get time for graceful shutdown. > >> > >> -Gour > >> > >> > >> > >> On 12/2/16, 8:29 AM, "Billie Rinaldi" <billie.rina...@gmail.com> wrote: > >> > >> >It looks like the Traceback stack for the stop command output is > >>truncated > >> >in the logs you pasted. I only see the first line of the Traceback: > >> >INFO 2016-11-30 18:07:03,919 PythonExecutor.py:97 - stop command > >>output: > >> > err: Traceback (most recent call last): > >> > File > >> > >>>"/export/hdk3/yarn/nm/usercache/hive/appcache/ > application_1479830316320_ > >> >64974/filecache/11/enable_presto_worker.zip/package/ > >> >scripts/enable_presto_worker_component.py", > >> >line 23, in <module> > >> > from resource_management import * > >> > > >> >So I cannot see the PYTHONPATH error you're talking about. If you paste > >> >the > >> >entire Traceback that might tell us more. > >> > > >> >Billie > >> > > >> >On Fri, Dec 2, 2016 at 7:19 AM, Ophir Etzion <op...@foursquare.com> > >> wrote: > >> > > >> >> it does implement a STOP command that does something useful. > >> >> it fails because the PYTHONPATH isn't set like it is in different > >> >>commands. > >> >> > >> >> On Thu, Dec 1, 2016 at 10:38 PM, Gour Saha <gs...@hortonworks.com> > >> >>wrote: > >> >> > >> >> > Does enable_presto_worker_component.py support/implement a STOP > >> >>command? > >> >> > > >> >> > Does your application need to run something useful when the stop > >>cmd > >> >>is > >> >> > initiated? > >> >> > > >> >> > -Gour > >> >> > > >> >> > On 11/30/16, 10:58 AM, "Ophir Etzion" <op...@foursquare.com> > wrote: > >> >> > > >> >> > >Hi, > >> >> > > > >> >> > >I hope I'm writing to the correct mailing list. please direct me > >> >> elsewhere > >> >> > >if this is not the correct place to write to. > >> >> > > > >> >> > >I've written a simple custom slider application and the STOP > >>script > >> >> fails > >> >> > >due to what seems like a slider issue of not setting the > >>PYTHONPATH > >> >>when > >> >> > >running the stop command. > >> >> > > > >> >> > >I will probably debug to see what goes on in > >> >>CustomServiceOrchestrator > >> >> and > >> >> > >why it doesn't set the env variables there but I'll only do it in > >>a > >> >> couple > >> >> > >of weeks. > >> >> > >I wanted to ask if anyone noticed something like this before I > >>look > >> >>into > >> >> > >it > >> >> > >further. > >> >> > > > >> >> > >in the agent log it looks like this: > >> >> > > > >> >> > >INFO 2016-11-30 18:07:03,894 ActionQueue.py:173 - Running command: > >> >> > >{u'roleCommand': u'STOP', u'clusterName': > >> >> > >u'enable-presto-worker_cluster_a', u'componentName': u'NODE', > >> >> > u'hostname': > >> >> > >u'fsak20.prod.foursquare.com', u'hostLevelParams': {u'java_home': > >> >> > >u'/data/loko/infrastructure-jdk8/current/bin/', u'container_id': > >> >> > >u'container_e468_1479830316320_64974_01_000091'}, u'commandType': > >> >> > >u'EXECUTION_COMMAND', u'roleParams': {u'auto_restart': u'false'}, > >> >> > >u'serviceName': u'enable-presto-worker_cluster_a', u'role': > >>u'NODE', > >> >> > >u'commandParams': {u'record_config': u'true', > >> >>u'service_package_folder': > >> >> > >u'${AGENT_WORK_ROOT}/work/app/definition/package', u'script': > >> >> > >u'scripts/enable_presto_worker_component.py', u'schema_version': > >> >> u'2.0', > >> >> > >u'command_timeout': u'600', u'script_type': u'PYTHON'}, > >>u'taskId': 5, > >> >> > >u'yarnDockerMode': False, u'commandId': '5-1', u'containers': [], > >> >> > >u'configurations': {u'global': {u'security_enabled': u'false', > >> >> > >u'app_container_id': > >>u'container_e468_1479830316320_64974_01_000091' > >> , > >> >> > >u'data_dir': u'/data/appdata/enable_presto_worker/data', > >> u'app_name': > >> >> > >u'enable_presto_worker.py', u'app_root': > >> >> > >u'${AGENT_WORK_ROOT}/app/install', > >> >> > >u'app_log_dir': u'${AGENT_LOG_ROOT}', u'app_pid_dir': > >> >> > >u'${AGENT_WORK_ROOT}/app/run', u'app_container_tag': u'2', > >> >>u'pid_file': > >> >> > >u'${AGENT_WORK_ROOT}/app/run/component.pid', u'app_install_dir': > >> >> > >u'${AGENT_WORK_ROOT}/app/install', u'app_input_conf_dir': > >> >> > >u'${AGENT_WORK_ROOT}/propagatedconf', u'state_monitor_port': > >> >>u'9990'}}} > >> >> > >INFO 2016-11-30 18:07:03,896 CustomServiceOrchestrator.py:329 - > >> >>Storing > >> >> > >applied config: {u'global': {u'app_container_id': > >> >> > >u'container_e468_1479830316320_64974_01_000091', > >> >> > > u'app_container_tag': u'2', > >> >> > > u'app_input_conf_dir': > >> >> > >u'/export/hdj3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_6 > >> >> > >4974/container_e468_1479830316320_64974_01_000091/ > propagatedconf', > >> >> > > u'app_install_dir': > >> >> > >u'/export/hdj3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_6 > >> >> > >4974/container_e468_1479830316320_64974_01_000091/app/install', > >> >> > > u'app_log_dir': > >> >> > >u'/data/log/hadoop-yarn/container/application_ > >> >> > 1479830316320_64974/containe > >> >> > >r_e468_1479830316320_64974_01_000091', > >> >> > > u'app_name': u'enable_presto_worker.py', > >> >> > > u'app_pid_dir': > >> >> > >u'/export/hdj3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_6 > >> >> > >4974/container_e468_1479830316320_64974_01_000091/app/run', > >> >> > > u'app_root': > >> >> > >u'/export/hdj3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_6 > >> >> > >4974/container_e468_1479830316320_64974_01_000091/app/install', > >> >> > > u'data_dir': u'/data/appdata/enable_presto_ > >> worker/data', > >> >> > > u'pid_file': > >> >> > >u'/export/hdj3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_6 > >> >> > >4974/container_e468_1479830316320_64974_01_000091/ > >> >> app/run/component.pid', > >> >> > > u'security_enabled': u'false', > >> >> > > u'state_monitor_port': u'9990'}} > >> >> > >INFO 2016-11-30 18:07:03,898 PythonExecutor.py:152 - command str: > >> >> > > /usr/bin/python -S > >> >> > >/export/hdk3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_649 > >> >> > >74/filecache/11/enable_presto_worker.zip/package/ > >> >> > scripts/enable_presto_wor > >> >> > >ker_component.py > >> >> > >STOP > >> >> > >/export/hda3/data/log/hadoop-yarn/container/application_ > >> >> > 1479830316320_6497 > >> >> > >4/container_e468_1479830316320_64974_01_000091/command-5.json > >> >> > >/export/hdk3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_649 > >> >> > >74/filecache/11/enable_presto_worker.zip/package > >> >> > >/export/hda3/data/log/hadoop-yarn/container/application_ > >> >> > 1479830316320_6497 > >> >> > > >>>4/container_e468_1479830316320_64974_01_000091/structured-out-5.json > >> >> > >INFO > >> >> > >/export/hdj3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_649 > >> >> > >74/container_e468_1479830316320_64974_01_000091 > >> >> > >INFO 2016-11-30 18:07:03,919 PythonExecutor.py:97 - stop command > >> >>output: > >> >> > > err: Traceback (most recent call last): > >> >> > > File > >> >> > >"/export/hdk3/yarn/nm/usercache/hive/appcache/ > >> >> > application_1479830316320_64 > >> >> > >974/filecache/11/enable_presto_worker.zip/package/ > >> >> > scripts/enable_presto_wo > >> >> > >rker_component.py", > >> >> > >line 23, in <module> > >> >> > > from resource_management import * > >> >> > > >> >> > > >> >> > >> > >> > >
