Re: [openstack-dev] [TaskFlow] Cross-run persistence
> On Aug 12, 2015, at 12:49 PM, Joshua Harlow wrote: > > possibly submit some kind of oslo-spec or > blueprint for 'selective persistence' (or other better named thing); if > u want to work on said code, that's even better :) Cheers, will do.. Once I get legal approval from work to contribute. --- GPG Fingerprint: 9530 B4AF 551B F3CD A45C 476C D4E5 662D DB97 69E3 signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TaskFlow] Cross-run persistence
On Wed, 12 Aug 2015 10:31:03 -0700 Demian Brecht wrote: > > > On Aug 10, 2015, at 1:18 PM, Joshua Harlow > > wrote: > > > > Is that what u are looking for? (or possibly something else?), > > Hi Josh, > > Thanks for the reply. I haven’t had time to dig into it much further, > but I’m not sure that’s what I’m looking for (unless I’m missing > something in initial configuration. What I have tried so far is > something to the effect of: > > store = {‘foo’: ‘bar’} > class MyAuthTask(task.Task): > default_provides = ‘auth_token’ > > def execute(self): > return ‘mytoken' > > engine = engines.load(flow, …, store=store) > engine.run() > print(store) > > What I’m trying to get at is to selectively persist the token. In > this case, after engine execution, values injected to the store are > no longer there. If I understand how the engine works (which I may > not, I’ve only been looking into this /very/ lightly over the last > few days), as tasks are executed, what they provide is injected into > the store. Unless you specifically tell the system to not > purge /anything/ from the store, all will be lost once the engine is > complete. I also don’t want to /all/ data to persist, but only select > items (i.e. tokens). Hmmm, I'll be out on vacation for a couple weeks, so feel free to jump on IRC and ask there, but the summary u have is mostly correct, but what u have to do that I don't see in the above is specify which persistence backend the engine should be using, if none is selected then all data saved goes into memory (and therefore is lost when the program is done). Once u get that working things will be persisted, although selective persistence isn't currently possible, but could be done in a somewhat easy manner (as long as we are clear on what i means and/or implies), Once u get the persistence working, http://docs.openstack.org/developer/taskflow/persistence.html should help here then feel free to possibly submit some kind of oslo-spec or blueprint for 'selective persistence' (or other better named thing); if u want to work on said code, that's even better :) Spec template @ https://github.com/openstack/oslo-specs/tree/master/specs Irc folks usually @ #openstack-state-management and/or #openstack-oslo (should be folks there that can help while i'm out...) -Josh > > Am I misunderstanding something here? > > Thanks again, > Demian > > --- > GPG Fingerprint: 9530 B4AF 551B F3CD A45C 476C D4E5 662D DB97 69E3 > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TaskFlow] Cross-run persistence
> On Aug 10, 2015, at 1:18 PM, Joshua Harlow wrote: > > Is that what u are looking for? (or possibly something else?), Hi Josh, Thanks for the reply. I haven’t had time to dig into it much further, but I’m not sure that’s what I’m looking for (unless I’m missing something in initial configuration. What I have tried so far is something to the effect of: store = {‘foo’: ‘bar’} class MyAuthTask(task.Task): default_provides = ‘auth_token’ def execute(self): return ‘mytoken' engine = engines.load(flow, …, store=store) engine.run() print(store) What I’m trying to get at is to selectively persist the token. In this case, after engine execution, values injected to the store are no longer there. If I understand how the engine works (which I may not, I’ve only been looking into this /very/ lightly over the last few days), as tasks are executed, what they provide is injected into the store. Unless you specifically tell the system to not purge /anything/ from the store, all will be lost once the engine is complete. I also don’t want to /all/ data to persist, but only select items (i.e. tokens). Am I misunderstanding something here? Thanks again, Demian --- GPG Fingerprint: 9530 B4AF 551B F3CD A45C 476C D4E5 662D DB97 69E3 signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TaskFlow] Cross-run persistence
Hmmm, I'm pretty sure what u want exists, For example: https://github.com/openstack/taskflow/blob/master/taskflow/examples/persistence_example.py That uses sqlite (or a file) to store information, U just have to decide what backend is best for u, Is that what u are looking for? (or possibly something else?), -Josh Demian Brecht wrote: Hi all, As I understand it, the persistence and related backends are intended to store atom and flow state. This will only persist until the end of an entirely successful run (unless explicitly configured to not purge for auditing and other such reasons). My question is: Is there a way to configure storage to persist data across runs? My specific use case: I’ve defined an authentication task. The auth server allows for ticket-based auth, which means the user (optionally) wouldn’t have to log in on every run if storage could be configured to persist the given token across runs. Did I miss something in the docs about this use case or should I be looking at something like oslo.cache to solve this problem? Thanks, Demian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TaskFlow] Cross-run persistence
Hi all, As I understand it, the persistence and related backends are intended to store atom and flow state. This will only persist until the end of an entirely successful run (unless explicitly configured to not purge for auditing and other such reasons). My question is: Is there a way to configure storage to persist data across runs? My specific use case: I’ve defined an authentication task. The auth server allows for ticket-based auth, which means the user (optionally) wouldn’t have to log in on every run if storage could be configured to persist the given token across runs. Did I miss something in the docs about this use case or should I be looking at something like oslo.cache to solve this problem? Thanks, Demian signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev