Thanks! It works. I should look at your sample more carefully.
My start implementation is a little bit complex. I will first bootstrap
the catalog. Vertica uses spread to communicate with each other. So I
also need to update the configuration file of spread to include all the
hostnames Vertica will run on. After one node(I assign it as master) is
on, I will then do some node creation work. Then I can start all the
vertica nodes. All the work is easy through Vertica's admintool.
But it becomes difficult in the Yarn environment because I need to
coordinate every node to avoid duplicate node name, duplicate catalog
path if more than one vertica nodes run on the same machine. And I also
need to remember each node's hostname to update the configuration file.
Now I uses ZooKeeper to do all the coordination and store hostnames.
Hope Slider will support more similar functionality in future releases.
Thanks.
Rui
On 07/24/2014 05:23 PM, Sumit Mohanty wrote:
If you are using the helpers provided in the resource management library
then you can set the wait_for_finish to False.
Execute(process_cmd,
user=params.app_user,
logoutput=False,
*wait_for_finish=False*
)
See the memecached sample I shared.
How does your start() implementation look like?
-Sumit
On Thu, Jul 24, 2014 at 1:56 PM, Rui Zhang <[email protected]> wrote:
Actually Vertica is not killed by the script and run perfectly. But Slider
will think that Vertica is killed and try again.
On 07/24/2014 04:52 PM, Rui Zhang wrote:
Hi,
thanks for your reply.
I know why it hangs. I have read the code of agent/PythonExecutor.py
It blocks in the line "process.communicate()".
My start command will spawn three child processes. Maybe this line is
waiting for the three to finish? I am not very familiar with how the
communicate function works. But Vertica is a long-running process, it is
not possible for Vertica to stop. So it always timeout and get killed by
the watch-dog-thread.
Is there a way to avoid the waiting?
Thanks.
On 07/24/2014 04:15 PM, Sumit Mohanty wrote:
My guess is that start() does not complete or fails and gets retried few
times and then eventually fails. Can you share the agent logs?
We have a new release now 0.40 - can you port your package over to 0.40?
If
you can share your package I can help you do that. In any case, only
critical change is the metainfo.xml structure where its enclosed within
"<application></application>".
http://slider.incubator.apache.org/docs/slider_specs/
hello_world_slider_app.html
is a work-in-progress doc for creating an application package which will
provide you the details.
-Sumit
On Thu, Jul 24, 2014 at 12:26 PM, Rui Zhang <[email protected]> wrote:
Hi,
I can start my own created package now. But it always stay IN_PROGRESS
status according to the log and after a long time it shows Failed.
However,
my application runs perfectly without any errors.
Why is this? How to determine the start is completed or not?
Thanks.
--
Rui Zhang
Software engineer Intern
Vertica, an HP Company
[email protected]
--
Rui Zhang
Software engineer Intern
Vertica, an HP Company
[email protected]
--
Rui Zhang
Software engineer Intern
Vertica, an HP Company
[email protected]
