Hi Gavin,
Yeah, there isn't as much consistency in API as I would like.
The way it should work, I believe, is something like:
* The Type should support an 'instances' method that returns a complete list of
all instances of that type that exist on the host itself. This method should
not require a catalog.
* The provider should support a 'prefetch' method, which accepts a list of
resources from the catalog and updates them with the correct provider instance,
with appropriate data already filled in to reflect the system state.
* Optionally, many providers implement (I think) their own 'instances' method,
which returns all instances and doesn't require a catalog, to implement both of
the above methods.
I just looked at the Provider base class, though, and it's not at all clear
from this. Part of the problem is that the system is smart enough to skip
providers that don't support prefetch, and the way we denote 'doesn't support
prefetch' is that it doesn't have the method. Given that, I left the method
off of the base class. Because 'instances' isn't used for this kind of test,
the base class ships with a stub method that just throws an error.
So yeah, all that's confusing, and could really use some additional work to
make it easier to understand, and to bring some consistency.
On Mar 29, 2013, at 12:37 PM, fatmcgav <fatmc...@gmail.com> wrote:
> Luke
>
> No worries.
>
> For the 2 I've converted soo far, it's made quite a performance difference.
> Thankfully can pull back all the resources with one NetApp api call, so it's
> drop the resource time from 5+ seconds to sub 1 second.
> However was a bit confused initially, as different providers seem to use
> different models... Some with just prefetch, some with just instances, some
> with both...
>
> However I'm just about to start the more complex netapp_volume provider,
> which has 4+ properties that have different api calls on top of the base
> resource list call. This is by far the slowest provider out of them with an
> average of 20+ seconds per run.
> So not sure of the best way to handle as yet.
>
> Cheers
> Gav
>
> On 29 Mar 2013 12:11, "Luke Kanies" <l...@puppetlabs.com> wrote:
> Hi Gavin,
>
> I'm glad to see you've sorted it out, sorry I didn't jump in before it was
> all resolved.
>
> How has prefetching worked out for you, in terms of performance?
>
> On Mar 28, 2013, at 3:28 PM, Gavin Williams <fatmc...@gmail.com> wrote:
>
>> Afternoon
>>
>> Managed to find my issue.. Variable name re-use :(
>>
>> Defined 'qtrees' as an empty array on line 3, and then populated it with a
>> whole load of device output on line 18 :(
>>
>> So have defined a 'qtree_instances' array on the outside to contain my
>> output, with qtrees being used on the inside to hold the NetApp filer
>> response... Latest code in Github.
>>
>> Cheers
>> Gavin
>>
>> On Thursday, 28 March 2013 11:44:39 UTC, Gavin Williams wrote:
>> Morning all
>>
>> Quick update... Looks like I managed to hack around the issue by adding the
>> following:
>>
>> ...
>> ap qtree_info
>>
>> # Check if it is a NaElement
>>
>> next unless qtree_info.respond_to?(:child_get_string)
>>
>> # Pull out the qtree name.
>>
>> name = qtree_info.child_get_string("qtree")
>> ...
>>
>> However this shows that I'm getting a total of 78 items processed, whereas
>> the original array only contains 53 items...
>> The additional items being processed are all like:
>> '#<Puppet::Type::Netapp_qtree::ProviderNetapp_qtree:'.
>> Have updated the gist with latest code and log file.
>>
>> Would like to understand where these are coming from, and if it's something
>> I'm doing incorrectly?
>>
>> In the mean-time, following fixing that bug, the provider now seems to work
>> as expected :)
>>
>> Cheers
>> Gavin
>>
>>
>> On Wednesday, 27 March 2013 17:32:23 UTC, Gavin Williams wrote:
>> Afternoon all
>>
>> I've started working on converting a couple more of my NetApp network device
>> providers to use a prefetch/flush model, as can see performance gains
>> available, etc...
>>
>> Anyways, I'm having issues with my netapp_qtree provider. It would appear
>> that somehow, an additional Puppet::Type... row is getting into an array and
>> breaking things...
>>
>> Have created a gist here with the details, as the log file is quite long.
>> Also includes the instances and prefetch def's for my netapp_qtree
>> provider...
>>
>> As you can see on Line 337 of the log, the array contains 40 items, however
>> on line 734 self.instances is trying to process item 41?!?!
>> What's also strange is that the item contents look like a Puppet Type
>> (#<Puppet::Type::Netapp_qtree), whereas all the others in the array are
>> NetApp specific items (#<NaElement:).
>>
>> So, any ideas???
>>
>> Cheers
>> Gavin
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to puppet-dev+unsubscr...@googlegroups.com.
>> To post to this group, send email to puppet-dev@googlegroups.com.
>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
>
> --
> Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | +1-615-594-8199
> Join us at PuppetConf 2013, August 22-23 in San Francisco -
> http://bit.ly/pupconf13
>
>
> --
> You received this message because you are subscribed to a topic in the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> puppet-dev+unsubscr...@googlegroups.com.
> To post to this group, send email to puppet-dev@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-dev+unsubscr...@googlegroups.com.
> To post to this group, send email to puppet-dev@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
--
Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | +1-615-594-8199
Join us at PuppetConf 2013, August 22-23 in San Francisco -
http://bit.ly/pupconf13
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-dev@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
