Hello Dear Hop-Users,
I am currently trying to figure out a simple solution to the following
problem. I kindly would like to hear your opinions or better solution
suggestions and best practice approaches from the community.
Problem: As a Data Engineer, how can I have multiple workflows that I
have created, run consecutively at certain times (during the night)?
My solution: I create a shell-script for each workflow and run the
shell-scripts via cronjobs.
Example of the shell-script:
----------
#!/bin/sh
# kill all running java processes
killall java
# get current dir where the shell-script and all workflow files are
path_to_workflow="$(dirname $(realpath $0))"
echo "files are under: $path_job"
# workflow to run
workflow="workflow_01.hwf"
# run workflow with hop-run
$HOME/hop/./hop-run.sh \
--project="hop_jobroom_candidates" \
--file="$pathtoworkflow/$workflow" \
--runconfig="local"
----------
As you can imagine, this gets somewhat tedious if I have to manage lets
say more than 50 workflows. How are you, dear hop-users, solving this
problem?
--
Best Regards from Switzerland
Sothy.
--
EIKYU GmbH
Sempacherstrasse 14
CH-4053 Basel
www.eikyu.ch
<http://www.eikyu.ch>
