I think I see what the biggest point of confusion here is. When fab executes a task but no -H parameters are set and the decorator specifies no host, the Connection object is instead an invoke Context object, which has no .local. I agree that that's not intuitive, but I'm not sure where that choice is made in the fabric codebase.
On Tue, Apr 2, 2019 at 2:50 PM Michel Albert <mic...@albert.lu> wrote: > > Creating a new Context from the connection config did indeed work. So I am > planning - for now - to go ahead with this. > > I hear you on the point of separating invoke from fabric tasks. > > To elaborate, and please don't read the following as a kind of rant, just as > an elaboration where the need to mix remote & local tasks comes from. > > My current aim is to migrate an old, rather large and complex collection of > fabric-1.x tasks to fabric-2.x And many of those tasks mix both remote and > local commands. My aim is to keep everything available under the "umbrella" > "fab" executable. Many tasks are shared among an ever growing team via a > library made available to them. I really want to avoid having two executables > where it may not be clear from the outside which task needs to be executed > using invoke and which one needs fabric. The end-goal is to have a task > runner which has the same core tasks for all of our projects. Namely: > > develop - to create a new development environment > build - To build and bundle whatever is necessary for deployment > test - To run unit-tests > deploy - To deploy to a targeted environment > > We chose fabric in the past because it is really flexible, and lets us use it > even for non-Python projects. And most of all it is extensible. So most > projects have more than these 4 core tasks. Not being able to determine > whether to use "inv" or "fab" would quickly become quite annoying. > > The library has grown over the past and it helped us a lot in our day-to-day > business. Splitting the tasks now into an invoke- and fabric-library is > definitely something I will take a closer look at, but I am afraid that some > of the more complex tasks might lose in readability when doing so. I will see. > > In any case, your feedback helped me a lot. I wonder if this would be a > useful example to put into the docs (that is,calling an invoke task from a > fabric task). It does not seem to be something very common, but I had a lot > of trouble finding a solution without falling back to this mailing list. > > On Tue, Apr 2, 2019 at 5:35 PM Brandon Whaley <redkr...@gmail.com> wrote: >> >> Have you tried instantiating a context with the config from Fabric's >> connection? I haven't mixed invoke tasks and fabric tasks in the same >> module before, I'm surprised the same task decorator will work for >> both using `inv` on the command line and `fab` as well. If you have a >> task that only does local work inside your fabfile module, I'd >> recommend using conn.local instead of making it an invoke task. If >> you have an existing invoke task codebase, I would import that module >> from my fabfile module and run tasks with a locally created context, >> since you're not calling the task from the invoke cli. >> >> ``` >> tasks.py >> ==== >> from invoke.tasks import task >> >> @task >> def build(ctx): >> ctx.run('make artifact') >> >> fabfile.py >> ==== >> from fabric.tasks import task >> from invoke.context import Context >> from tasks import build >> >> @task(hosts=PROD) >> def deploy(conn): >> ctx = Context(config=conn.config) >> build(ctx) >> conn.put('artifact', '/app') >> ``` >> >> On Tue, Apr 2, 2019 at 2:04 AM Michel Albert <mic...@albert.lu> wrote: >> > >> > I understand the differences. The only traceback I got was the one I sent >> > earlier. This traceback is correct, only maybe a bit cryptic. >> > >> > So far, I managed to get some basic tasks ported to fabric2. >> > >> > However, this morning I ran into a new problem: >> > >> > I have a remote task (deploy) which depends on a local task (build) and I >> > want to be able to call the build step separately. But now I don't know >> > how to call "build" from the "deploy" task. The docs state to simply call >> > the function and forward the connection/context. But when calling from the >> > "deploy" task, I have a "Connection" instance, but I need a "Context" >> > instance to pass into the "build" task. Is there a way to get the Invoke >> > context from the Connection instance? Example: >> > >> > @task >> > def build(ctx): >> > ctx.run('make artifact') >> > >> > @task(hosts=PROD) >> > def deploy(conn): >> > build(?) # <- what do I need to pass onto build? "conn" won't work >> > conn.put('artifact', '/app') >> > >> > On Mon, Apr 1, 2019 at 9:33 PM Brandon Whaley <redkr...@gmail.com> wrote: >> >> >> >> There is no local attribute on invoke contexts, only on fabric >> >> connections (which inherits from invoke contexts). If this isn't what >> >> you're trying to do, could you provide some code examples that give >> >> you the traceback you're seeing? >> >> >> >> On Mon, Apr 1, 2019 at 1:07 PM Michel Albert <mic...@albert.lu> wrote: >> >> > >> >> > Ok. I think I will get it to work like that. >> >> > >> >> > But after fiddling around with it I found out that the type of the >> >> > first argument depends on whether a "hosts" argument was passed into >> >> > the task decorator or not. This was quite confusing. And the error is a >> >> > bit cryptic. When calling "local" on an invoke context, the following >> >> > AttributeError is thrown: >> >> > >> >> > Traceback (most recent call last): >> >> > File "/usr/lib/python3.7/site-packages/invoke/config.py", line 113, >> >> > in __getattr__ >> >> > return self._get(key) >> >> > File "/usr/lib/python3.7/site-packages/invoke/config.py", line 178, >> >> > in _get >> >> > value = self._config[key] >> >> > File "/usr/lib/python3.7/site-packages/invoke/config.py", line 169, >> >> > in __getitem__ >> >> > return self._get(key) >> >> > File "/usr/lib/python3.7/site-packages/invoke/config.py", line 178, >> >> > in _get >> >> > value = self._config[key] >> >> > KeyError: 'local' >> >> > >> >> > During handling of the above exception, another exception occurred: >> >> > >> >> > Traceback (most recent call last): >> >> > File "/home/exhuma/.local/bin/fab", line 10, in <module> >> >> > sys.exit(program.run()) >> >> > File "/usr/lib/python3.7/site-packages/invoke/program.py", line 363, >> >> > in run >> >> > self.execute() >> >> > File "/usr/lib/python3.7/site-packages/invoke/program.py", line 532, >> >> > in execute >> >> > executor.execute(*self.tasks) >> >> > File "/usr/lib/python3.7/site-packages/invoke/executor.py", line 129, >> >> > in execute >> >> > result = call.task(*args, **call.kwargs) >> >> > File "/usr/lib/python3.7/site-packages/invoke/tasks.py", line 128, in >> >> > __call__ >> >> > result = self.body(*args, **kwargs) >> >> > File "/home/exhuma/tmp/fabfile.py", line 8, in get_version >> >> > version = ctx.local('python setup.py --version').strip() >> >> > File "/usr/lib/python3.7/site-packages/invoke/config.py", line 125, >> >> > in __getattr__ >> >> > raise AttributeError(err) >> >> > AttributeError: No attribute or config key found for 'local' >> >> > >> >> > Valid keys: ['connect_kwargs', 'forward_agent', 'gateway', >> >> > 'inline_ssh_env', 'load_ssh_configs', 'port', 'run', 'runners', >> >> > 'ssh_config_path', 'sudo', 'tasks', 'timeouts', 'user'] >> >> > >> >> > Valid real attributes: ['cd', 'clear', 'config', 'cwd', 'from_data', >> >> > 'pop', 'popitem', 'prefix', 'run', 'setdefault', 'sudo', 'update'] >> >> > >> >> > >> >> > On Mon, Apr 1, 2019 at 5:37 PM Brandon Whaley <redkr...@gmail.com> >> >> > wrote: >> >> >> >> >> >> Hi Mich, >> >> >> >> >> >> The connection object (you're using ctx in your examples) has a .local >> >> >> method that is just a pass-through to invoke.run. It's documented on >> >> >> the connection object's page: >> >> >> http://docs.fabfile.org/en/2.4/api/connection.html?highlight=local#fabric.connection.Connection.local >> >> >> >> >> >> On Mon, Apr 1, 2019 at 5:55 AM Michel Albert <mic...@albert.lu> wrote: >> >> >> > >> >> >> > Hi, >> >> >> > >> >> >> > >> >> >> > Consider the following fabric-1 task. For illustration I kept it >> >> >> > really short: >> >> >> > >> >> >> > @fab.task >> >> >> > def sample(): >> >> >> > version = fab.local('python setup.py --version') >> >> >> > fab.run('mkdir -p /snapshots/%s' % version.strip()) >> >> >> > >> >> >> > This task needs to run a local and remote command. I am now trying >> >> >> > to port this to fabric-2, and I can't figure out how I can implement >> >> >> > this. If I define the "hosts" variable in the task, then the first >> >> >> > line will be executed on the remote host as well, which I don't >> >> >> > want. A naive aproach which won't work: >> >> >> > >> >> >> > @task(hosts=PROD) >> >> >> > def sample(ctx): >> >> >> > version = ctx.run('python setup.py --version').strip() # <- >> >> >> > this won't work >> >> >> > ctx.run('mkdir -p /snapshots/%s' % version) >> >> >> > >> >> >> > At first I thought I would split the task into two, one for just >> >> >> > local commands and one for remote tasks, but then I am forced to >> >> >> > pass in the context, which will in turn cause it again to be run >> >> >> > remotely: >> >> >> > >> >> >> > @task >> >> >> > def get_version(ctx): >> >> >> > version = ctx.run('python setup.py --version').strip() >> >> >> > return version >> >> >> > >> >> >> > @task(hosts=PROD) >> >> >> > def sample(ctx): >> >> >> > version = get_version(ctx) # <- this won't work >> >> >> > ctx.run('mkdir -p /snapshots/%s' % version) >> >> >> > >> >> >> > How can I accomplish something like this? And where is it noted in >> >> >> > the docs? In the current example on the "Upgrading from 1.x" page >> >> >> > does not have a single task mixing local with remote commands in any >> >> >> > way. >> >> >> > >> >> >> > >> >> >> > Regards, >> >> >> > >> >> >> > >> >> >> > Mich >> >> >> > >> >> >> > _______________________________________________ >> >> >> > Fab-user mailing list >> >> >> > Fab-user@nongnu.org >> >> >> > https://lists.nongnu.org/mailman/listinfo/fab-user _______________________________________________ Fab-user mailing list Fab-user@nongnu.org https://lists.nongnu.org/mailman/listinfo/fab-user