Hi Antoine, Marcus and Stuart,
Thank you all for your help - setenv via login class was exactly what I
was looking for!
On 4/20/23 23:44, Stuart Henderson wrote:
On 2023-04-21, Antoine Jacoutot wrote:
Hi.
You can pas environment variables by creating a login class matching the name
of your rc.d script in login.conf and adding setenv to that class.
Or create a new file /etc/login.conf.d/(script_name). If you're writing
a port, create pkg/(script_name).login and it will be added to the plist
automatically when you "make update-list".
Thanks for that, worked perfectly!
The problem I'm facing is that it seems that Airflow looks for various
environment variables such as $HOME, $AIRFLOW_HOME, $AIRFLOW_CONFIG etc and I'm
seeing no obvious way to pass those requisite environment variables to Airflow
from my rc.d script. Without these variables set, Airflow annoyingly just looks
in /dev/null for everything and fails to function.
That seems a strange dir to use by default. Did you perhaps set the
_airflowd user's home dir to /dev/null? If so, try /var/airflow instead.
The users home folder was indeed set to /var/airflow. Turns out the
/dev/null thing was a red herring - I appear to have found a bug in
Airflow. When starting the Airflow scheduler with '-D' to daemonize, the
scheduler program just silently hangs. I got sent down a couple rabbit
holes because the scheduler continued to send heartbeats yet failed to
parse dags or run jobs. I didn't fully clue in until I compared the
process lists between the scheduler when daemonized vs running it in the
foreground. When run in the forground, all sorts of LocalExecutor and
other tasks were spawned, unlike when daemonized. It was then made super
obvious when I realized when daemonized it was leaving zombie processes
behind after stopping with rcctl/SIGTERM. Curiously, the airflow
webserver seems to daemonize just fine and I've had no such issues with it.
Long story short, after removing '-D' flag from my rc.d daemon
arguments, everything worked and I didn't even end up actually needing
to set any environment variables.
With that said, I'll put together a bug report for the Airflow folks.
Thank you muchly for the pointers.
Regards,
Jordan
