That's very interesting. Can you run with the following environment
variables and send me the log?
GRPC_VERBOSITY=DEBUG GRPC_TRACE=xds_client,xds_resolver
On Wed, Mar 16, 2022 at 10:53 PM František Bořánek <fbora...@gmail.com>
wrote:
> C++, the latest 1.44.0 release.
>
> 16. 3. 2022 v 21:32, Mark D. Roth <r...@google.com>:
>
>
> It is valid behavior, but I agree that it's a little sub-optimal for the
> client to send two messages when it reestablishes the stream. What
> language are you using gRPC in?
>
> On Wed, Mar 16, 2022 at 1:19 PM František Bořánek <fbora...@gmail.com>
> wrote:
>
>> It makes sense.
>>
>> However, in this case, the initial connection was already made and this
>> tcpdump samples recovering of ADS stream. And as you write:
>>
>> > Note that once your application receives all 7 resources from the xDS
>> server, if the ADS stream is broken and the client reestablishes it at that
>> point, it should immediately subscribe to all 7 resources in the initial
>> message on the new stream.
>>
>> It behaves as I described. First 1 resource and then 7 resources. Note
>> that the messages have version_info. Brand new channels don't have
>> version_info.
>>
>> But as you pointed, the channels can be created anytime, thus it's valid
>> bahaviur.
>>
>> Thanks for replying.
>>
>> F.
>>
>> 16. 3. 2022 v 19:49, Mark D. Roth <r...@google.com>:
>>
>> The gRPC client does not know a priori what set of resource names are
>> available on the xDS server, and even if it did, it would not request all
>> of them proactively, because it may not actually need all of them.
>> Instead, each time a gRPC channel is created with an "xds:" target URI,
>> that tells gRPC which LDS resource to request, and it requests the
>> resources as they are needed. Because the application can create new
>> channels at any time, the set of resources requested on the ADS stream can
>> change at any time.
>>
>> If your application immediately creates 7 channels when it starts up,
>> then the pattern you're seeing is expected. When the first channel
>> (presumably for "xds:server-1") starts connecting, it will immediately
>> subscribe to the corresponding LDS resource ("server-1"). While it's
>> waiting for that message to be sent, it notices that the other 6 channels
>> have started up, so it realizes that it has to subscribe to the other 6
>> resources. By the time the initial message is sent and it's able to send a
>> second message, it knows about all 7 subscriptions, so the second message
>> contains all 7 of them.
>>
>> Note that once your application receives all 7 resources from the xDS
>> server, if the ADS stream is broken and the client reestablishes it at that
>> point, it should immediately subscribe to all 7 resources in the initial
>> message on the new stream. That is because it already knows about all of
>> the subscriptions at the point at which it sends the initial message on the
>> new stream.
>>
>> This communication is fully legal and should work fine with any
>> xDS-compliant server. As described in this section of the xDS spec
>> <https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol#ack-nack-and-resource-type-instance-version>,
>> the intent of the version field in the DiscoveryRequest is to provide the
>> server with a way to know when it sends a version that the client considers
>> invalid. But if the client sends a second DiscoveryRequest with the same
>> version and a different list of resource names, that indicates that it is
>> changing what resources it is subscribing to, and the server cannot ignore
>> that just because the version is the same as one it has previously seen.
>>
>> I hope this information is helpful.
>>
>> On Mon, Mar 14, 2022 at 5:41 AM František Bořánek <fbora...@gmail.com>
>> wrote:
>>
>>> Hi,
>>> I don't know if such behaviour is a bug.
>>>
>>> I have server communicates with 7 services configured via XDS and after
>>> the xds-client got GO AWAY packet from XDS server 6 channels cannot connect
>>> due to error and only one is ok. I did a tcpdump to see a chat between our
>>> implementation of XDS and C++gRPC and cannot deside if C++gRPC behave
>>> correct.
>>>
>>> It relates to
>>> https://github.com/grpc/proposal/blob/master/A27-xds-global-load-balancing.md#initial-request-on-the-ads-stream
>>> > The first request on the ADS stream will include a Node message
>>> that identifies the client. As mentioned above, most of the fields for the
>>> Node message will be read from the bootstrap file.
>>>
>>> I would expect that after reconnect the xds client sends to ADS stream 4
>>> requests (LDS, CDS, EDS, RDS) however it sends 5 requests (LDS, LDS, CDS,
>>> EDS, RDS). On our side of xds server we are confused by these 2 LDS
>>> requests.
>>>
>>> 1.
>>> (PROTOBUF) envoy.api.v2.DiscoveryRequest
>>> {
>>> version_info: "ba0255a2-07b6-45ed-83b4-dad0cbf96b1f"
>>> node: {
>>> cluster: "default-cluster"
>>> user_agent_name: "gRPC C-core linux"
>>> user_agent_version: "C-core 22.0.0"
>>> client_features: "envoy.lb.does_not_support_overprovisioning"
>>> build_version: "gRPC C-core linux 22.0.0"
>>> }
>>> resource_names: "server-1"
>>> type_url: "type.googleapis.com/envoy.api.v2.Listener"
>>> }
>>>
>>> 2.
>>> (PROTOBUF) envoy.api.v2.DiscoveryRequest
>>> {
>>> version_info: "ba0255a2-07b6-45ed-83b4-dad0cbf96b1f"
>>> resource_names: "server-1"
>>> resource_names: "server-2"
>>> resource_names: "server-3"
>>> resource_names: "server-4"
>>> resource_names: "server-5"
>>> resource_names: "server-6"
>>> resource_names: "server-7"
>>> type_url: "type.googleapis.com/envoy.api.v2.Listener"
>>> }
>>>
>>> - Shouldn't it be only one LDS request?
>>> - Shouldn't it the initial request contains all configured resources?
>>>
>>> Root of our problem is that we responds only on first request since it
>>> have the same version. I know how to fix it on our end, but it is behaviour
>>> of gRPC xds-client correct?
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "grpc.io" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to grpc-io+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/grpc-io/e2a5befd-5943-4b40-afc4-4864938b9b88n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/grpc-io/e2a5befd-5943-4b40-afc4-4864938b9b88n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
>>
>> --
>> Mark D. Roth <r...@google.com>
>> Software Engineer
>> Google, Inc.
>>
>>
>>
>
> --
> Mark D. Roth <r...@google.com>
> Software Engineer
> Google, Inc.
>
>
--
Mark D. Roth <r...@google.com>
Software Engineer
Google, Inc.
--
You received this message because you are subscribed to the Google Groups
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to grpc-io+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/grpc-io/CAJgPXp7o2_Gev822v4R8VQA5GJLDQJkSAbNuftun-4QN_6PjPg%40mail.gmail.com.
