Alternative (sketch) proposal: Track this via resource quota's, since we have an API that's supposed to deal with this already.
I'd imagine one new API:
void grpc_resource_quota_notify_on_memory_change(
grpc_resource_quota* resource_quota,
size_t min_memory_usage,
size_t max_memory_usage,
size_t* current_memory_usage,
void* tag,
grpc_completion_queue* cq);
With the semantics that 'tag' would be emitted on 'cq' whenever memory
usage falls outside of the min/max. On entry current_memory_usage would be
what the application last observed, prior to queuing the tag it would be
updated to be the instantaneous current value, and so have the
thrash-prevention properties of channel state.
This will (of course) need the resource quota code to be informed of (most)
of the allocations within core, and that hookup will be time consuming.
On Thu, Jan 18, 2018 at 10:56 PM apolcyn via grpc.io <
grpc-io@googlegroups.com> wrote:
> Personally, for the API I am thinking it would be flexible if core exposed
> a single global callback, e.g., "notify_of_change_in_memory(amount)", where
> amount is positive or negative for alloc/free.
>
>
> On Thursday, January 18, 2018 at 10:03:34 PM UTC-8, apo...@google.com
> wrote:
>>
>> Good point, I think that really anything consuming large/unbounded memory
>> should potentially be tracked.
>>
>> As for the API, I am thinking:
>>
>> - If the API is synchronous, we could provide a rough estimate of an
>> object upon it's creation that doesn't change throughout it's lifetime.
>> Perhaps the estimate could change change throughout the lifetime of the
>> process though. For example, after a channel allocates a large buffer once,
>> future channels can use that to increase their estimate.
>>
>> - An callback or queue based API might be more accurate though.
>>
>> One example of something to account for is the cert file (what originally
>> motivated this). In ruby, if the user uses the default channel SSL creds,
>> then we end up loading a ~300KB cert file (
>> https://github.com/grpc/grpc/blob/master/etc/roots.pem), which is
>> apparently copied (at least partially and at least once) roughly per
>> channel.
>>
>>
>>
>> On Thursday, January 18, 2018 at 9:31:32 PM UTC-8, Craig Tiller wrote:
>>>
>>> Let's enumerate the things that need sizes in this grfc... At first cut
>>> I'd suggest:
>>>
>>> - channels
>>> - servers
>>> - calls
>>> - completion queues
>>>
>>> Do we also need slices, slice buffers?
>>> What about metadata?
>>>
>>> Server memory can change drastically over time since it also maintains a
>>> list of channels and incoming but not yet delivered calls.
>>>
>>> Should we additionally account for this? If yes, how? Is it polled? Is
>>> there a callback model?
>>>
>>> On Thu, Jan 18, 2018, 8:56 PM apolcyn via grpc.io <
>>> grp...@googlegroups.com> wrote:
>>>
>>>> Discussion of https://github.com/grpc/proposal/pull/55
>>>>
>>>> this thread supersedes
>>>> https://groups.google.com/forum/#!topic/grpc-io/FPoXprcT0d4
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "grpc.io" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to grpc-io+u...@googlegroups.com.
>>>> To post to this group, send email to grp...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/grpc-io.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/grpc-io/ee85c861-e4a7-4cbb-a0eb-444b3ea05ce8%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/grpc-io/ee85c861-e4a7-4cbb-a0eb-444b3ea05ce8%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
> You received this message because you are subscribed to the Google Groups "
> grpc.io" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to grpc-io+unsubscr...@googlegroups.com.
> To post to this group, send email to grpc-io@googlegroups.com.
> Visit this group at https://groups.google.com/group/grpc-io.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/grpc-io/1c77c9c9-e310-4e61-a520-b14a9c94099a%40googlegroups.com
> <https://groups.google.com/d/msgid/grpc-io/1c77c9c9-e310-4e61-a520-b14a9c94099a%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit
https://groups.google.com/d/msgid/grpc-io/CAAvp3oOWgii_ibwgArJxaNLrCEh5MUoNMu%2BHs1vfcOMyU%3DJ3EA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
smime.p7s
Description: S/MIME Cryptographic Signature
