Excellent. On Apr 5, 2013, at 12:33 PM, fatmcgav <fatmc...@gmail.com> wrote:
> I was looking at doing all the work in the 'flush' method, however have just > been doing some more testing and think I've come up with an alternative... > > Can run the user creation in the 'create' def, and then clear @property_hash > so that it doesn't get invoked in 'flush'. > 'Flush' then calls the user modification if required... > > Seems to work quite well from my testing :) > > Latest code is: > https://github.com/fatmcgav/fatmcgav-netapp/blob/master/lib/puppet/provider/netapp_user/netapp_user.rb > > Cheers > Gavin > > > On 5 April 2013 11:15, Luke Kanies <l...@puppetlabs.com> wrote: > Ah. You should copy the model in the user/group providers: The providers > have a 'create' method that uses useradd, for instance, and then a 'uid=' > method that uses usermod. > > Does that help? Are you doing all of the actual work in the 'flush' method, > and that's where you need to differentiate? > > On Apr 5, 2013, at 11:06 AM, fatmcgav <fatmc...@gmail.com> wrote: > >> Luke >> >> A user create requires a different API call to a modify... >> (useradmin-user-add vs useradmin-user-modify). >> >> So I could leave the creation to 'create', however how do I then trigger a >> modify in flush when required? >> Or should I treat any call into flush with a :present ensure as a modify? >> Though I vaguely remember that causing a double-call on a resource create - >> Once from create, and then another from flush... >> >> Cheers >> Gavin >> >> >> On 5 April 2013 10:01, Luke Kanies <l...@puppetlabs.com> wrote: >> Can you explain why you need to know the difference? >> >> The framework should handle that for you. >> >> On Apr 4, 2013, at 2:48 PM, Gavin Williams <fatmc...@gmail.com> wrote: >> >>> Indeed... :) >>> >>> Ok, next question for you/all... >>> >>> How could I differentiate between what is a resource creation and a >>> resource modification? >>> >>> Have updated the netapp_user provider to prefetch/flush, however am >>> struggling to differentiate between what is a resource creation, and what >>> is a modification... >>> Code is here: >>> https://github.com/fatmcgav/fatmcgav-netapp/commit/66092978f4182c5474a60011db99ee2e3e12e689 >>> >>> Thoughts? >>> >>> Cheers >>> Gavin >>> >>> On Tuesday, 2 April 2013 16:17:33 UTC+1, Luke Kanies wrote: >>> That is awesome. It's amazing what a bit of optimization will do for you. >>> >>> On Apr 2, 2013, at 3:12 PM, fatmcgav <fatm...@gmail.com> wrote: >>> >>>> Yep, have been testing it today on my dev and production Puppet masters, >>>> and seems to be working as expected... >>>> >>>> Have been very impressed with the performance improvements - Has dropped >>>> the netapp_volume resource time from 40+ seconds to <0.1 seconds. So >>>> MASSIVE improvements to be made from switching to the prefetch/flush >>>> style... >>>> >>>> Cheers >>>> Gav >>>> >>>> >>>> On 2 April 2013 13:24, Luke Kanies <lu...@puppetlabs.com> wrote: >>>> Looks good from a cursory glance. Hopefully it's all working for you. >>>> >>>> On Apr 2, 2013, at 9:52 AM, Gavin Williams <fatm...@gmail.com> wrote: >>>> >>>>> As a quick update, I've pushed my latest set of code tweaks up - >>>>> https://github.com/fatmcgav/fatmcgav-netapp/commit/7a7d18bf39cdbb04a3b0b5192929ec6a857c0a5e >>>>> >>>>> Need to tidy the code up a bit, but should be able to get the gist :) >>>>> Have got a couple of defs which pull back the bulk of the information, >>>>> and then a couple that need to be called on a volume by volume basis. >>>>> All gets put together in instances, and provider'd up in prefetch. >>>>> >>>>> Comments welcome. >>>>> >>>>> Cheers >>>>> Gavin >>>>> >>>>> On Friday, 29 March 2013 13:15:12 UTC, Gavin Williams wrote: >>>>> Luke >>>>> >>>>> Cheers for that info, should prive useful... >>>>> >>>>> Guess I could implement a method for each property that pulls back all >>>>> that property values for all the volumes, and then match them up in >>>>> prefetch. >>>>> >>>>> Cheers >>>>> Gav >>>>> >>>>> On 29 Mar 2013 12:49, "Luke Kanies" <lu...@puppetlabs.com> wrote: >>>>> Hi Gavin, >>>>> >>>>> Yeah, there isn't as much consistency in API as I would like. >>>>> >>>>> The way it should work, I believe, is something like: >>>>> >>>>> * The Type should support an 'instances' method that returns a complete >>>>> list of all instances of that type that exist on the host itself. This >>>>> method should not require a catalog. >>>>> >>>>> * The provider should support a 'prefetch' method, which accepts a list >>>>> of resources from the catalog and updates them with the correct provider >>>>> instance, with appropriate data already filled in to reflect the system >>>>> state. >>>>> >>>>> * Optionally, many providers implement (I think) their own 'instances' >>>>> method, which returns all instances and doesn't require a catalog, to >>>>> implement both of the above methods. >>>>> >>>>> I just looked at the Provider base class, though, and it's not at all >>>>> clear from this. Part of the problem is that the system is smart enough >>>>> to skip providers that don't support prefetch, and the way we denote >>>>> 'doesn't support prefetch' is that it doesn't have the method. Given >>>>> that, I left the method off of the base class. Because 'instances' isn't >>>>> used for this kind of test, the base class ships with a stub method that >>>>> just throws an error. >>>>> >>>>> So yeah, all that's confusing, and could really use some additional work >>>>> to make it easier to understand, and to bring some consistency. >>>>> >>>>> On Mar 29, 2013, at 12:37 PM, fatmcgav <fatm...@gmail.com> wrote: >>>>> >>>>>> Luke >>>>>> >>>>>> No worries. >>>>>> >>>>>> For the 2 I've converted soo far, it's made quite a performance >>>>>> difference. Thankfully can pull back all the resources with one NetApp >>>>>> api call, so it's drop the resource time from 5+ seconds to sub 1 >>>>>> second. >>>>>> However was a bit confused initially, as different providers seem to use >>>>>> different models... Some with just prefetch, some with just instances, >>>>>> some with both... >>>>>> >>>>>> However I'm just about to start the more complex netapp_volume provider, >>>>>> which has 4+ properties that have different api calls on top of the base >>>>>> resource list call. This is by far the slowest provider out of them with >>>>>> an average of 20+ seconds per run. >>>>>> So not sure of the best way to handle as yet. >>>>>> >>>>>> Cheers >>>>>> Gav >>>>>> >>>>>> On 29 Mar 2013 12:11, "Luke Kanies" <lu...@puppetlabs.com> wrote: >>>>>> Hi Gavin, >>>>>> >>>>>> I'm glad to see you've sorted it out, sorry I didn't jump in before it >>>>>> was all resolved. >>>>>> >>>>>> How has prefetching worked out for you, in terms of performance? >>>>>> >>>>>> On Mar 28, 2013, at 3:28 PM, Gavin Williams <fatm...@gmail.com> wrote: >>>>>> >>>>>>> Afternoon >>>>>>> >>>>>>> Managed to find my issue.. Variable name re-use :( >>>>>>> >>>>>>> Defined 'qtrees' as an empty array on line 3, and then populated it >>>>>>> with a whole load of device output on line 18 :( >>>>>>> >>>>>>> So have defined a 'qtree_instances' array on the outside to contain my >>>>>>> output, with qtrees being used on the inside to hold the NetApp filer >>>>>>> response... Latest code in Github. >>>>>>> >>>>>>> Cheers >>>>>>> Gavin >>>>>>> >>>>>>> On Thursday, 28 March 2013 11:44:39 UTC, Gavin Williams wrote: >>>>>>> Morning all >>>>>>> >>>>>>> Quick update... Looks like I managed to hack around the issue by adding >>>>>>> the following: >>>>>>> >>>>>>> ... >>>>>>> ap qtree_info >>>>>>> >>>>>>> # Check if it is a NaElement >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> next unless qtree_info.respond_to?(:child_get_string) >>>>>>> >>>>>>> # Pull out the qtree name. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> name = qtree_info.child_get_string("qtree") >>>>>>> ... >>>>>>> >>>>>>> However this shows that I'm getting a total of 78 items processed, >>>>>>> whereas the original array only contains 53 items... >>>>>>> The additional items being processed are all like: >>>>>>> '#<Puppet::Type::Netapp_qtree::ProviderNetapp_qtree:'. >>>>>>> Have updated the gist with latest code and log file. >>>>>>> >>>>>>> Would like to understand where these are coming from, and if it's >>>>>>> something I'm doing incorrectly? >>>>>>> >>>>>>> In the mean-time, following fixing that bug, the provider now seems to >>>>>>> work as expected :) >>>>>>> >>>>>>> Cheers >>>>>>> Gavin >>>>>>> >>>>>>> >>>>>>> On Wednesday, 27 March 2013 17:32:23 UTC, Gavin Williams wrote: >>>>>>> Afternoon all >>>>>>> >>>>>>> I've started working on converting a couple more of my NetApp network >>>>>>> device providers to use a prefetch/flush model, as can see performance >>>>>>> gains available, etc... >>>>>>> >>>>>>> Anyways, I'm having issues with my netapp_qtree provider. It would >>>>>>> appear that somehow, an additional Puppet::Type... row is getting into >>>>>>> an array and breaking things... >>>>>>> >>>>>>> Have created a gist here with the details, as the log file is quite >>>>>>> long. >>>>>>> Also includes the instances and prefetch def's for my netapp_qtree >>>>>>> provider... >>>>>>> >>>>>>> As you can see on Line 337 of the log, the array contains 40 items, >>>>>>> however on line 734 self.instances is trying to process item 41?!?! >>>>>>> What's also strange is that the item contents look like a Puppet Type >>>>>>> (#<Puppet::Type::Netapp_qtree), whereas all the others in the array are >>>>>>> NetApp specific items (#<NaElement:). >>>>>>> >>>>>>> So, any ideas??? >>>>>>> >>>>>>> Cheers >>>>>>> Gavin >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Puppet Developers" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to puppet-dev+...@googlegroups.com. >>>>>>> To post to this group, send email to puppe...@googlegroups.com. >>>>>>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | >>>>>> +1-615-594-8199 >>>>>> Join us at PuppetConf 2013, August 22-23 in San Francisco - >>>>>> http://bit.ly/pupconf13 >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in the >>>>>> Google Groups "Puppet Developers" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en. >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> puppet-dev+...@googlegroups.com. >>>>>> To post to this group, send email to puppe...@googlegroups.com. >>>>>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Puppet Developers" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to puppet-dev+...@googlegroups.com. >>>>>> To post to this group, send email to puppe...@googlegroups.com. >>>>>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | >>>>> +1-615-594-8199 >>>>> Join us at PuppetConf 2013, August 22-23 in San Francisco - >>>>> http://bit.ly/pupconf13 >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "Puppet Developers" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> puppet-dev+...@googlegroups.com. >>>>> To post to this group, send email to puppe...@googlegroups.com. >>>>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "Puppet Developers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to puppet-dev+...@googlegroups.com. >>>>> To post to this group, send email to puppe...@googlegroups.com. >>>>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>>> >>>> >>>> >>>> -- >>>> Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | >>>> +1-615-594-8199 >>>> Join us at PuppetConf 2013, August 22-23 in San Francisco - >>>> http://bit.ly/pupconf13 >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Puppet Developers" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en. >>>> To unsubscribe from this group and all its topics, send an email to >>>> puppet-dev+...@googlegroups.com. >>>> To post to this group, send email to puppe...@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Puppet Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to puppet-dev+...@googlegroups.com. >>>> To post to this group, send email to puppe...@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> >>> >>> >>> -- >>> Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | +1-615-594-8199 >>> Join us at PuppetConf 2013, August 22-23 in San Francisco - >>> http://bit.ly/pupconf13 >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Puppet Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to puppet-dev+unsubscr...@googlegroups.com. >>> To post to this group, send email to puppet-dev@googlegroups.com. >>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >> >> >> -- >> Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | +1-615-594-8199 >> Join us at PuppetConf 2013, August 22-23 in San Francisco - >> http://bit.ly/pupconf13 >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Puppet Developers" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en. >> To unsubscribe from this group and all its topics, send an email to >> puppet-dev+unsubscr...@googlegroups.com. >> To post to this group, send email to puppet-dev@googlegroups.com. >> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-dev+unsubscr...@googlegroups.com. >> To post to this group, send email to puppet-dev@googlegroups.com. >> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > > -- > Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | +1-615-594-8199 > Join us at PuppetConf 2013, August 22-23 in San Francisco - > http://bit.ly/pupconf13 > > > -- > You received this message because you are subscribed to a topic in the Google > Groups "Puppet Developers" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en. > To unsubscribe from this group and all its topics, send an email to > puppet-dev+unsubscr...@googlegroups.com. > To post to this group, send email to puppet-dev@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-dev?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-dev+unsubscr...@googlegroups.com. > To post to this group, send email to puppet-dev@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-dev?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > -- Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | +1-615-594-8199 Join us at PuppetConf 2013, August 22-23 in San Francisco - http://bit.ly/pupconf13 -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To post to this group, send email to puppet-dev@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-dev?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
