Hey Eric, Thanks for thinking through some of those details. I don't know if this is an extension of that or an alternative to that - but the Galaxy tool shed has some support for workflows and stored these in a VCS repository (mercurial). Augmenting that to support a sort of one-click (or couple click) export from Galaxy to the tool shed was something on my radar, but going further and providing tracking, branching, etc... over time of the workflow as you have outlined here could make a lot of sense.
Echoing his comments here, I know Peter is not a fan of tool shed repositories being implemented as mercurial repositories. Continuing to make progress on hiding this fact from users while allowing leveraging of more advanced features as you described here should help that decision bear more fruit going forward. The other feature requests related to allowing multiple editors - I think this is a great idea. I think histories, pages, and workflows all need to support more sophisticated permissions. I have created Trello cards related to these requests here, feel free to comment on these to flush out the details: https://trello.com/c/3blHqhxb https://trello.com/c/YvKxFOdd Thanks again for the suggestions, -John On Thu, Oct 31, 2013 at 12:16 PM, Eric Rasche <rasche.e...@yandex.ru> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Peter, > > Please bear with me, I'm mostly tossing these ideas out for my > benefit/to spawn discussion on how one might implement this. This was > originally a short email before I figured out how much work would be > involved... > > Upon cursory inspection I see that workflows are stored as lines in the > database. Were it to be easily stored in a VCS, all database workflow > steps would probably have to be migrated to disk. In the toolsheds > workflows are stored as JSON, so it'd just be a matter of dumping > existing internal workflows to disk. > > On disk it would be stored in a "workflow" folder, with a set of > numbered sub-folders. Each subfolder would store a VCS repo. These > numbered sub-folders would be referenced in the database. > Additionally, there would be a "cache" folder in the workflow folder, to > store the current/active version of the workflow, per workflow, per user. > > Continuing just to toss ideas out, I could see implementing this in a > VCS while hiding as much of that as possible away from the user. How > about this: > > - - Sharing creates a tag in the vcs (your permanent, immutable version) > - - If it was shared to them with edit capabilities, a branch is created > to allow them to make their changes and eventually merge back. > - - Upon save, we'd ask them if they'd like to add a short message > describing their changes (by default it'd read "{UserRealName} made > modifications" or something like that so they could just click through > if they didn't want to write a message) > - - In the edit view, there'd be an additional button which reads "Merge > these changes back into {Sharer'sRealName}'s copy of this workflow" > which would execute a merge (I guess it could create a merge request but > that seems like a bit much) > - - In the workflow list we would now have an area to select previous > revisions of workflows and open those up in the edit window. > - - selecting a previous version of the workflow would set the user's > active workflow (somewhere in DB) to that revision. > > Keeping track of which workflow was "active" might be a bit difficult... > I would imagine it would work like so: > > - - workflows are separated from users. > - - the DB table of "stored_workflow" would be modified to store which > repository (in the workflow/NNN/ folder), as well as the current head > - - a table would be created, mapping users to specific revisions of a > workflow. (workflow_user_revision) [user_id,workflow_id,revision] > - maybe revision would be either revision hash or "HEAD" to say they > always want the latest version? > - - sharing a workflow (read/write) would add a new row in the > workflow_user_revision, with an updated user_id, and a copy of the that > user's current revision hash (because that user would be sharing that > specific version). > - - sharing a workflow (read only) would do pretty much the same. > > > This made sense in my mind at least...does this seem like a reasonable > way to implement this? Do you (or anyone else) have comments to make on > this proposal? > > (Mind you, this is just me thinking out loud for the time being...I have > a lot of other galaxy modifications to get through first for my > organisation (native LDAP integration, script ACLs)) > > > > > > On 10/30/2013 05:11 PM, Peter Cock wrote: >> Hi Eric, >> >> I've being pondering this too - it can easily get very complicated >> with multiple copies of a workflow with minor variations. >> >> I agree that "read only" sharing makes sense as a short term >> goal, but I see two variants being useful here - an read only >> snapshot copy from the instant it was shared (where the shared >> version is locked forever but may of course optionally be >> copied by the receiver, and the copy edited), and as an >> always up to date live pointer (in case the original owner >> revises it - then all the users get the changes immediately). >> >> Some of your wish list ideas also sound very good, but may >> not be practical - not everyone will understand a VCS >> approach to managing workflows :( >> >> Regards, >> >> Peter >> >> >> On Wed, Oct 30, 2013 at 8:02 PM, Eric Rasche <rasche.e...@yandex.ru> wrote: >> I'm working with a number of people in my department to develop a single >> workflow that will be used for a course we teach. So far I've found that >> "sharing" a workflow with individual users/roles isn't very optimal. >> They email me with changes they'd like to happen, or they create their >> own copies which immediately branch off and don't help everyone else who >> might need them. >> >> As such, I'd like to propose changes and would like community feedback >> on my proposed changes so when I create the trello card/maybe try my >> hand at implementing them, they aren't completely specific to my use case. >> >> In my mind, the 'optimal' version of managing workflows would be >> something along the following lines: >> >> Top priorities: >> >> - workflows can be shared as "write" or "read only" with multiple >> users/roles. >> - This would allow there to be a single copy of a workflow in cases >> where multiple people might make changes to a single workflow >> - For "read only", this allows the previous/traditional model of >> sharing a workflow and the receiver being able to run it as is, or to >> copy and modify it. >> >> Note: >> >> For my case at least, realtime multiple person editing is not needed. If >> a banner shows up which reads "someone started editing before you, >> either ask them to save + leave or save as a copy" would work in my case. >> >> Wishlist features: >> >> - workflows are treated as VCS repositories >> - Note that full merge/branching probably not really necessary for >> this >> case (that would be really nice, but probably too much of a nightmare to >> code...) >> - You can see previous iterations of workflows. >> - Workflows can be "owned" by either a user or a role >> - users could create a workflow and transfer ownership to a role >> - role owned workflows are delete-able (or maybe require some X% of >> users in that role to confirmation deletion?) >> - This would sort of work like a group owned VCS repository, where >> multiple users can create workflows of interest to a group of people >> (role) and have them, by default, be available. >> >> >> >> Anyone have input on this proposal? >> >> Cheers, >> Eric >> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ > > Cheers, > Eric > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.19 (GNU/Linux) > > iQIcBAEBAgAGBQJScpCLAAoJEMqDXdrsMcpV8usQAKxwj7m6zpsiI5Yj54Yzr5dU > LsPKUriNcvzCxNWXvjzvXjywzulNQtWCBIpiecHhK8kuXkc+QlLkJ/bHNvaCyS7U > 99haARombM2+3vQ4UF3kn7q4OgoFOe7OM+N6g9cxvsyrf+jC2m7k/HD5rfN+WPtG > xIMwGfROg6eSWrB3jJJtFEprRDntTT9H94RYEi0XfRtsv4KEAMsQv4gXtk7AdjDg > 4HfdLZPkQ7GV/SxLopwj/halMIiKk7E3Kb762UI3Rf9IFPBUEftXKDbfIzrriBb/ > k/Ya3VhQEK9GRwnzcZbVY7Z/yTHFq8FvaBkFOniY+bnHPqcoPknFKWkvnTGry3B5 > 4y18KvTnLYc/2EEk4MeHq9ErQr7XMH0YAc027vKrEFBIzyhs6xwLyY4Ud2skrw0J > H7bKY2RpXICkumMmeVEn6n2cOJYT1rexHlRld/OJYEc2hHfSNEi3t9lZ8NK+Ljhx > U4eAt/uMVl437gWRV/0NJlKdT/+lNLB4plvmswXdSvG2yQsaFbfCFYbbwcaKFUMA > Hd9j9lJwVj076j0UzM3NITSgoxRB2oOIoAVccGjRTEShBvNCL9KuggoFosWQf28i > CTOYB8YNw8ud9rBt9HrTVDWmqowC+azEYBgavyY6ZoMpF1IvFQnA8Lqx0PHFFRdd > Zq/KWDhf37aGw5pF/QoH > =OAgc > -----END PGP SIGNATURE----- > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
