Re: [qubes-users] a few things about salt

john.david.r.smith Tue, 17 Jan 2017 07:41:14 -0800

1)
even when some states fail for some vm, the cli tool displays ok. it would
be better, if it displayed error in case of an error (some errors are
displayed).


Can you provide example error which wasn't detected? Regardless of the
result, output is logged to /var/log/qubes/mgmt-*.log in dom0.

i somehow fail to reproduce the case. (i just noticed it when playingaround with salt)there were some states failed inside domu (i think some packageinstallation)

i will try to reproduce it later.

5)
are there plans to add some functionality to the interface?


Yes, "qvm" module will be extended for new features in Qubes 4.0. Is it
what you've asked about?


yeah. the question was just about any planned additions.

I think there is currently no sane way to setup global defaults (other
than cmd.run: qubes-prefs ...). So, we'll work on that too.


nice

6)
currently i really don't like the way the configuration works.
i have a top file where i execute some states for dom0
these states create and configure my vms.
then in some top files i choose some vms and configure them again (but this
time it is some config i am doing in the domu).

so it kind of looks like this:
top.top
-----
base:
  dom0:
    - create-cfg-vm1
  vm1:
    - some-cfg-in-domu
----

now i have two layers of configuration (in top and sls).
for some config stuff i have to change a sls and for other i have to change
the top
is there a plan to change this?

e.g. some kind of virtual minions?

i would like to write something like this:
top.top
-----
base:
  dom0:
    - copy-sequence.Strg-Alt-Shift-C
  vm1:
    - create        #this affects dom0
    - color.red     #this affects dom0
    - netvm.sys-tor #this affects dom0
    - mail          #this affects domU
----
then i could see all my domU config in the top file.

i currently hacked something but this only works in a sls file and for dom0
config (but has this style of syntax)

i am currently looking whether i can do the same in a top file (but i doubt
it, since there is no templating in top files)


And the last sentence is exactly the reason why it's tricky to have it
in one place. Rendering sls files (may) require getting data (grains) from
target system and we don't want to parse that data from VM in dom0.
To limit attack surface. So, we can't render sls for VMs in dom0, we
need to decide what goes where at 'top' files level.

I think the only think that can be improved here, is some "automatic"
creation of VMs mentioned in top file - something like you've described
above. But it's tricky to do it, while keeping flexibility of salt...
Using valid salt syntax like yours, to achieve different effect looks
like asking for troubles. If going that way, IMO it would be better to
have something that isn't valid salt syntax here and have a pre-processor
script to create actual salt configuration.


i am currently working at something like this:
i have a top file activating a dom0 sls

in this sls i do dom0 config, create vms and configure them (dom0 configAND domU config).all domU config is used to generate a generated.top file activating thecorrect states for the correct minions.

then everything is in one file (not the top file, but this sls file hasthe function of a top file)the disadvantage would be that i always need to run dom0 to generate upto date files for my minions. (but in my opinion the advantages beat thedisadvantages)

how is the order of execution?
will dom0 always be executed before any domU is started?


Yes. In particular you can create VMs using states for dom0, just to
have them configured a moment later using states for VM.

when are the files for domU read?
after dom0 is configured? (then i could write state files during dom0
configuration)


Yes, those files are loaded just before configuring VM.

i noticed that, but it could have been possible you do something likethis (maybe because salt does things like this):

a) copy all files to some cache
b) run dom0 (using the files from the cache)
c) run domU (using the files from the cache)

in this case i would not be able to generate files in b to use in c

8)
is there some way to execute some dom0 scripts after configuration of domu?
(e.g. trim-template)


Currently no.


do you plan to add something like this?

9)
the fedora-24-min template can't really be configured with salt.
there is the package file missing.
after i installed the package i still got an error: "Target 'fedora-24-min'
did not return any data, probably due to an error. exit code 20"


The important thing is what is your default template - it is used for
that intermediate VM from where target VMs are configured. Is it also
fedora-24-min?
salt-ssh requirements in the target VM are really minimal - I think any
shell + python should be enough. For me it works, but it's possible that
my minimal template is no longer such minimal...
Ok, tried on fresh minimal template and found the problem: sudo
So, packages needs to be installed:
 - file
 - sudo

And also cleanup ~/QubesIncoming/disp-mgmt-* in the template, because it
isn't removed after failed run...

any idea how i could fix this?
or where i could get more data. e.g. are there some salt logs in some vm
(e.g. in the domU)?


/var/log/qubes/mgmt-*.log
That's the only thing that is preserved.


this is the file i got the error message from.

there probably are files in the management vm, but this vm gets deleted.
is there an option to stop the deletion of the management vm?


There is no option for that, but you can suspend qubesctl execution
(Ctrl-Z) to prevent that. You need to do that when you see that target
VM is being starting (at this moment dom0 have already send all required
data and all the execution is in management VM).

The above I've debugged exactly this way:
1. Ctrl-Z on qubesctl.
2. Open terminal in disp-mgmt-fedora-24-minimal.
3. Look at /etc/qubes-rpc/qubes.SaltLinuxVM - this is what is executed.
4. Get the last two lines and execute them, fix problems, repeat.

Those lines:

    export PATH="/usr/lib/qubes-vm-connector/ssh-wrapper:$PATH"
    salt-ssh "$target_vm" $salt_command

Of course adjust $target_vm (fedora-24-minimal) and $salt_command
(state.highstate).


my default template is fedora-24 (this should have all the required stuff).

i installed file and sudo in fedora-24-minimal, but it did not work.

when debugging i got an error and have no real idea what is going on.(tmp-base-f24 == fedora-24-minimal) :


# salt-ssh tmp-base-f24 state.highstate

[ERROR ] An un-handled exception from the multiprocessing process'MultiprocessingProcess-1' was caught:

Traceback (most recent call last):

File "/usr/lib/python2.7/site-packages/salt/utils/process.py", line644, in _run

    return self._original_run()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)

File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",line 418, in handle_routine

    stdout, stderr, retcode = single.run()

File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",line 834, in run

    stdout, retcode = self.run_wfunc()

File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",line 880, in run_wfunc

    opts_pkg = pre_wrapper['test.opts_pkg']()  # pylint: disable=E1102

File"/usr/lib/python2.7/site-packages/salt/client/ssh/wrapper/__init__.py",line 117, in caller

    stdout, stderr, retcode = single.cmd_block()

File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",line 1134, in cmd_blocklog.error('ERROR: Failure deploying thin, retrying:{0}\n{1}'.format(stdout, stderr), stderr, retcode)

  File "/usr/lib64/python2.7/logging/__init__.py", line 1193, in error
    self._log(ERROR, msg, args, **kwargs)

File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 310,in _log

    self, level, msg, args, exc_info=exc_info, extra=extra
  File "/usr/lib64/python2.7/logging/__init__.py", line 1285, in _log

record = self.makeRecord(self.name, level, fn, lno, msg, args,exc_info, func, extra)File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 344,in makeRecord

    exc_info, func)

File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 191,in __init__

    self.colormsg = '%s%s%s' % (cmsg, self.getMessage(), reset)
  File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Process MultiprocessingProcess-1:
Traceback (most recent call last):

File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in_bootstrap

    self.run()