Let's move this to review on GitHub PR, which should be more friendly.

Sheng Wu 吴晟
Twitter, wusheng1108


wei jin <k...@apache.org> 于2021年4月9日周五 下午7:05写道:

> hello, community,
>
> After some discussion, I made a summary and listed some function points
> that need to be collaborated. The content is as follows:
>
> - Protocol
> Firstly, use http protocol to send access log to skywalking, and in the
> future use GRPC protocol.
>
> - Structure
> AccessLogProto (ALP) [1]
>
> - Community Cooperation
>
> Apache APISIX
> 1. Support the collection of fields required by access log
> 2. Extend plugin, use http protocol to send to the target address
>
> Apache Skywalking
> 1. Implement http receiver, support AccessLogProto (ALP) format analysis,
> and write to OAP
>
> Welcome to add and discuss, thanks.
>
> [1] Data structure definition for reference
> ```go
> message AccessLogEntry {
> AccessLogCommon common = 1;
> HTTPRequestProperties request = 2;
>   HTTPResponseProperties response = 3;
> }
> ```
>
> ```go
> message AccessLogCommon {
> double sample_rate = 1;
> string downstream_remote_address = 2;
> string downstream_local_address = 3;
> string upstream_remote_address = 4;
> string upstream_local_address = 5;
> google.protobuf.Timestamp start_time = 6;
> // Interval between the first downstream byte received and the first
> upstream byte sent
> google.protobuf.Duration time_to_first_upstream_tx_byte = 7;
> string route_name = 8;
> HTTPRequestProperties request = 9;
>   HTTPResponseProperties response = 10;
> }
> ```
>
>
> ```go
> message HTTPRequestProperties {
> string method = 1;
> string scheme = 2;
> google.protobuf.UInt32Value port = 3;
> string path = 4;
> string user_agent = 5;
> string forwarded_for = 6;
> tring request_id = 7;
> uint64 request_headers_bytes = 8;
> uint64 request_body_bytes = 9;
> map<string, string> request_headers = 10;
> }
> ```
>
> ```go
> message HTTPResponseProperties {
> uint64 response_code = 1;
> uint64 response_headers_bytes = 2;
> uint64 response_body_bytes = 3;
> map<string, string> response_headers = 4;
> }
> ```
>
> Sheng Wu <wu.sheng.841...@gmail.com> 于2021年4月6日周二 下午4:45写道:
>
> > This seems a very abstract question.
> > What kind of access log do you have? Is there a design or format document
> > to describe that?
> >
> > Sheng Wu 吴晟
> > Twitter, wusheng1108
> >
> >
> > Ming Wen <wenm...@apache.org> 于2021年4月6日周二 下午4:17写道:
> >
> > > Hi, sheng wu,
> > > How can Apache APISIX send ALS to skywalking?
> > > Are we implementing it in the existing skywalking plugin? Or need to
> add
> > a
> > > new plug-in?
> > >
> > > Thanks,
> > > Ming Wen, Apache APISIX PMC Chair
> > > Twitter: _WenMing
> > >
> > >
> > > Sheng Wu <wu.sheng.841...@gmail.com> 于2021年3月23日周二 下午12:07写道:
> > >
> > > > Ming Wen <wenm...@apache.org> 于2021年3月23日周二 上午11:25写道:
> > > >
> > > > > > The biggest advantage of ALS, is replacing the physical local
> > access
> > > > log
> > > > > > file. So you have metrics, topology(dependency), and logs in one
> > > > package
> > > > > of
> > > > > > data.
> > > > >
> > > > > If we also record the metrics and tracing data with the physical
> > local
> > > > > access log, can we achieve the same effect?
> > > > >
> > > >
> > > > Technically, it is practicable. Once we add file scanning capability
> in
> > > the
> > > > satellite, or use 3rd party tool, and forward to the OAP.
> > > > The question is, do we really need the physical log file, as
> basically,
> > > it
> > > > costs unnecessary resources.
> > > >
> > > >
> > > > Sheng Wu 吴晟
> > > > Twitter, wusheng1108
> > > >
> > > >
> > > > >
> > > > > > If you want to discuss most ALS in APISIX, I am glad to help with
> > the
> > > > > > design and integration solution.
> > > > >
> > > > > yes, I want to discuss more details.
> > > > >
> > > > > Thanks,
> > > > > Ming Wen, Apache APISIX PMC Chair
> > > > > Twitter: _WenMing
> > > > >
> > > > >
> > > > > wei jin <k...@apache.org> 于2021年3月22日周一 下午3:58写道:
> > > > >
> > > > > > Zexuan Luo <spacewan...@apache.org> 于2021年3月21日周日 下午2:18写道:
> > > > > >
> > > > > > > I am not familiar with
> > > > https://github.com/apache/skywalking-satellite.
> > > > > > > Can we use this project to convert plain HTTP to GRPC?
> > > > > >
> > > > > >
> > > > > > Skywalking satellite is usually deployed as a sidecar. It can
> > convert
> > > > > > APISIX gateway metrics to other data formats, do some
> > preprocessing,
> > > > and
> > > > > > even change the data collection method from pull to push.
> > > > > >
> > > > > > However, the original prometheus metrics are still generated by
> > > APISIX,
> > > > > > which is not suitable for URI-level statistics.
> > > > > >
> > > > > > wei jin <k...@apache.org> 于2021年3月22日周一 下午3:57写道:
> > > > > >
> > > > > > > Zexuan Luo <spacewan...@apache.org> 于2021年3月21日周日 下午2:18写道:
> > > > > > >
> > > > > > > > I am not familiar with
> > > > > https://github.com/apache/skywalking-satellite.
> > > > > > > > Can we use this project to convert plain HTTP to GRPC?
> > > > > > >
> > > > > > >
> > > > > > > Skywalking satellite is usually deployed as a sidecar. It can
> > > convert
> > > > > > > APISIX gateway metrics to other data formats, do some
> > > preprocessing,
> > > > > and
> > > > > > > even change the data collection method from pull to push.
> > > > > > >
> > > > > > > However, the original prometheus metrics are still generated by
> > > > APISIX,
> > > > > > > which is not suitable for URI-level statistics.
> > > > > > >
> > > > > > >
> > > > > > > YuanSheng Wang <membp...@apache.org> 于2021年3月21日周日 下午10:08写道:
> > > > > > >
> > > > > > >> For APISIX, gRPC calls need to be supported.
> > > > > > >>
> > > > > > >> Either subrequest or pure Lua mode is ok, this feature is also
> > > very
> > > > > > useful
> > > > > > >> in other cases.
> > > > > > >>
> > > > > > >> We can use `subrequest` to implement the first version, this
> way
> > > > seems
> > > > > > >> simpler.
> > > > > > >>
> > > > > > >>
> > > > > > >> On Sun, Mar 21, 2021 at 2:34 PM Zexuan Luo <
> > > spacewan...@apache.org>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Another solution is writing a Lua GRPC library with
> > > > > > >> > https://github.com/tokers/lua-resty-http2
> > > > > > >> > https://github.com/starwing/lua-protobuf
> > > > > > >> >
> > > > > > >> > Zexuan Luo <spacewan...@apache.org> 于2021年3月21日周日 下午2:18写道:
> > > > > > >> > >
> > > > > > >> > > I am not familiar with
> > > > > > https://github.com/apache/skywalking-satellite
> > > > > > >> .
> > > > > > >> > > Can we use this project to convert plain HTTP to GRPC?
> > > > > > >> > >
> > > > > > >> > > Sheng Wu <wu.sheng.841...@gmail.com> 于2021年3月21日周日
> > 下午1:59写道:
> > > > > > >> > > >
> > > > > > >> > > > Oh, you mean that.
> > > > > > >> > > > SkyWalking natively provides HTTP APIs for nearly all
> > > > protocols,
> > > > > > >> even
> > > > > > >> > if
> > > > > > >> > > > not, we are happy to add. There is no block between
> APISIX
> > > and
> > > > > > >> > SkyWalking.
> > > > > > >> > > > The point to discuss the gRPC service here is about the
> > > > > > performance,
> > > > > > >> > > > including proto's no field name in transmission, enum as
> > > int,
> > > > > > small
> > > > > > >> > int as
> > > > > > >> > > > small, and especially streaming.
> > > > > > >> > > > If you want to implement ALS, then this is super
> important
> > > to
> > > > > > >> provide
> > > > > > >> > > > better performance.
> > > > > > >> > > >
> > > > > > >> > > > Sheng Wu 吴晟
> > > > > > >> > > > Twitter, wusheng1108
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >> > > > Chao Zhang <zchao1...@gmail.com> 于2021年3月21日周日
> 下午1:55写道:
> > > > > > >> > > >
> > > > > > >> > > > > It’s not so easy to use gRPC in APISIX as the lack of
> > > > > ecosystem,
> > > > > > >> but
> > > > > > >> > with
> > > > > > >> > > > > the help of gRPC gateway (
> > > > > > >> > https://github.com/grpc-ecosystem/grpc-gateway).
> > > > > > >> > > > > We can use the restful APIs to talk with gRPC
> endpoints.
> > > > Just
> > > > > > like
> > > > > > >> > the way
> > > > > > >> > > > > APISIX communicates with ETCD V3 APIs.
> > > > > > >> > > > >
> > > > > > >> > > > > Chao Zhang
> > > > > > >> > > > > https://github.com/tokers
> > > > > > >> > > > >
> > > > > > >> > > > > On March 21, 2021 at 1:53:41 PM, Sheng Wu (
> > > > > > >> wu.sheng.841...@gmail.com
> > > > > > >> > )
> > > > > > >> > > > > wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > Chao Zhang <zchao1...@gmail.com> 于2021年3月21日周日
> > 下午1:46写道:
> > > > > > >> > > > >
> > > > > > >> > > > > > Does the gRPC service enables grpc gateway? If so,
> > it’s
> > > > easy
> > > > > > for
> > > > > > >> > APISIX
> > > > > > >> > > > > to
> > > > > > >> > > > > > integrate it?
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > > > What is the relationship with grpc gateway?
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > Zexuan Luo <spacewan...@apache.org>于2021年3月21日
> > > 周日13:35写道:
> > > > > > >> > > > > >
> > > > > > >> > > > > > > Another solution is wrapping the C++
> implementation
> > of
> > > > the
> > > > > > >> GPRC
> > > > > > >> > > > > > > client. But it requires you to prebuild the proto
> > file
> > > > > into
> > > > > > >> *.c
> > > > > > >> > source
> > > > > > >> > > > > > > and can only be distributed as part of
> > > apisix-openresty.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Zexuan Luo <spacewan...@apache.org> 于2021年3月21日周日
> > > > > 下午1:25写道:
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > AFAIK, there is not library support making GPRC
> > > > > directly.
> > > > > > >> > > > > > > > One solution is subrequest + http2grpc, but this
> > way
> > > > > > doesn't
> > > > > > >> > support
> > > > > > >> > > > > > > > stream mode. So it is not a good solution.
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > YuanSheng Wang <membp...@apache.org>
> > 于2021年3月20日周六
> > > > > > >> 下午10:53写道:
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > LGTM
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > > The biggest advantage of ALS, is replacing
> the
> > > > > > physical
> > > > > > >> > local
> > > > > > >> > > > > > access
> > > > > > >> > > > > > > log
> > > > > > >> > > > > > > > > file.
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > The APISIX has a plugin `http-logger`, I think
> > > > APISIX
> > > > > is
> > > > > > >> > easy to
> > > > > > >> > > > > > report
> > > > > > >> > > > > > > > > those data to SkyWalking.
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > And I think we can try to use `gRPC` to report
> > > data.
> > > > > > What
> > > > > > >> do
> > > > > > >> > > > > > > > > you think? @ming @spacewander
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > On Fri, Mar 19, 2021 at 11:43 PM Ming Wen <
> > > > > > >> > wenm...@apache.org>
> > > > > > >> > > > > > wrote:
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > > Cool, I think we can try ALS
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > > > > > Sheng Wu <wu.sheng.841...@gmail.com
> > >于2021年3月19日
> > > > > > >> > 周五下午11:21写道:
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > > > > > > We have run the Envoy ALS performance
> test a
> > > > long
> > > > > > time
> > > > > > >> > ago(2
> > > > > > >> > > > > > > years, I
> > > > > > >> > > > > > > > > > > think). Sadly, no public blog.
> > > > > > >> > > > > > > > > > > It could easily support 10k-20k/s traffic
> > > with a
> > > > > > very
> > > > > > >> > low load
> > > > > > >> > > > > to
> > > > > > >> > > > > > > the
> > > > > > >> > > > > > > > > > > proxy. But, it is based on gRPC, so
> > basically,
> > > > it
> > > > > > >> > doesn't send
> > > > > > >> > > > > > many
> > > > > > >> > > > > > > > > > things
> > > > > > >> > > > > > > > > > > out, mostly only flags and latency value.
> > > > > > >> > > > > > > > > > > APISIX is using HTTP1.1 mostly, so, this
> > could
> > > > be
> > > > > > >> > different.
> > > > > > >> > > > > > > > > > > Also, at the same time, besides the ALS,
> we
> > > > have a
> > > > > > >> > chance to
> > > > > > >> > > > > > > provide a
> > > > > > >> > > > > > > > > > push
> > > > > > >> > > > > > > > > > > mode LUA SDK in skywalking meter format.
> It
> > > > could
> > > > > > >> provide
> > > > > > >> > > > > > URI-level
> > > > > > >> > > > > > > > > > metrics
> > > > > > >> > > > > > > > > > > at reasonable resource cost, especially it
> > > works
> > > > > > with
> > > > > > >> > > > > SkyWalking
> > > > > > >> > > > > > > > > > satellite,
> > > > > > >> > > > > > > > > > > which could leverage local disk to cache
> the
> > > > data.
> > > > > > >> > > > > > > > > > > The biggest advantage of ALS, is replacing
> > the
> > > > > > >> physical
> > > > > > >> > local
> > > > > > >> > > > > > > access log
> > > > > > >> > > > > > > > > > > file. So you have metrics,
> > > topology(dependency),
> > > > > and
> > > > > > >> > logs in
> > > > > > >> > > > > one
> > > > > > >> > > > > > > package
> > > > > > >> > > > > > > > > > of
> > > > > > >> > > > > > > > > > > data.
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > > If you want to discuss most ALS in
> APISIX, I
> > > am
> > > > > glad
> > > > > > >> to
> > > > > > >> > help
> > > > > > >> > > > > with
> > > > > > >> > > > > > > the
> > > > > > >> > > > > > > > > > > design and integration solution.
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > > Sheng Wu 吴晟
> > > > > > >> > > > > > > > > > > Twitter, wusheng1108
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > > Ming Wen <wenm...@apache.org>
> 于2021年3月19日周五
> > > > > > >> 下午11:08写道:
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > yes, Prometheus is not designed for
> > > URI-level
> > > > > > >> metrics.
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > Does envoy ALS will send too much data
> and
> > > > cause
> > > > > > >> > performance
> > > > > > >> > > > > > > problems?
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > And I know skywalking will supoort envoy
> > > WASM,
> > > > > is
> > > > > > >> there
> > > > > > >> > > > > > anything
> > > > > > >> > > > > > > apisix
> > > > > > >> > > > > > > > > > > can
> > > > > > >> > > > > > > > > > > > learn from this?
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > Sheng Wu <wu.sheng.841...@gmail.com
> > > > >于2021年3月19日
> > > > > > >> > 周五下午8:11写道:
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > > Hi APISIX & SkyWalking community
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > > I was discussing with Wei Jin about
> > using
> > > > > > >> Prometheus
> > > > > > >> > SDK
> > > > > > >> > > > > to
> > > > > > >> > > > > > > monitor
> > > > > > >> > > > > > > > > > > > > URI-level metrics of APISIX/OpenResty.
> > > > > > >> > > > > > > > > > > > > Wei received feedback that way of
> using
> > > > > > Prometheus
> > > > > > >> > SDK
> > > > > > >> > > > > > causing
> > > > > > >> > > > > > > > > > > > > a significant performance impact. From
> > the
> > > > > > >> > observability
> > > > > > >> > > > > > > perspective,
> > > > > > >> > > > > > > > > > > > Prom
> > > > > > >> > > > > > > > > > > > > is not good at gathering a large scale
> > of
> > > > > > >> entities'
> > > > > > >> > > > > metrics
> > > > > > >> > > > > > > due to
> > > > > > >> > > > > > > > > > its
> > > > > > >> > > > > > > > > > > > > local cache and pull mode.
> > > > > > >> > > > > > > > > > > > > Instead, we should consider providing
> a
> > > push
> > > > > way
> > > > > > >> like
> > > > > > >> > > > > Envoy
> > > > > > >> > > > > > > ALS[1]
> > > > > > >> > > > > > > > > > and
> > > > > > >> > > > > > > > > > > > > Metrics Service[2] did(it does through
> > > gRPC,
> > > > > and
> > > > > > >> > > > > SkyWalking
> > > > > > >> > > > > > > > > > recommended
> > > > > > >> > > > > > > > > > > > in
> > > > > > >> > > > > > > > > > > > > the prod environment.
> > > > > > >> > > > > > > > > > > > > If this is accepted and agreed by the
> > > APISIX
> > > > > > >> > community, we
> > > > > > >> > > > > > > could look
> > > > > > >> > > > > > > > > > > for
> > > > > > >> > > > > > > > > > > > > implementation in the
> > > > skywalking-nginx-lua[3]
> > > > > > >> repo to
> > > > > > >> > > > > collect
> > > > > > >> > > > > > > > > > > > metrics(meter
> > > > > > >> > > > > > > > > > > > > in SkyWalking) and use HTTP(in
> > SkyWalking
> > > > > native
> > > > > > >> > meter
> > > > > > >> > > > > > format)
> > > > > > >> > > > > > > to
> > > > > > >> > > > > > > > > > > report
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > > This mail has been sent to
> > > > > > dev@apisix.apache.org
> > > > > > >> and
> > > > > > >> > > > > > > > > > > > > d...@skywalking.apache.org. Please
> reply
> > > all
> > > > > > when
> > > > > > >> you
> > > > > > >> > > > > discuss
> > > > > > >> > > > > > > in the
> > > > > > >> > > > > > > > > > > > > thread.
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > > [1]
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/accesslog/v2/als.proto
> > > > > > >> > > > > > > > > > > > > [2]
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/metrics/v2/metrics_service.proto
> > > > > > >> > > > > > > > > > > > > [3]
> > > > > > >> https://github.com/apache/skywalking-nginx-lua
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > > Sheng Wu 吴晟
> > > > > > >> > > > > > > > > > > > > Twitter, wusheng1108
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > --
> > > > > > >> > > > > > > > > > > > Thanks,
> > > > > > >> > > > > > > > > > > > Ming Wen, Apache APISIX PMC Chair
> > > > > > >> > > > > > > > > > > > Twitter: _WenMing
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > --
> > > > > > >> > > > > > > > > > Thanks,
> > > > > > >> > > > > > > > > > Ming Wen, Apache APISIX PMC Chair
> > > > > > >> > > > > > > > > > Twitter: _WenMing
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > --
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > *MembPhis*
> > > > > > >> > > > > > > > > My GitHub: https://github.com/membphis
> > > > > > >> > > > > > > > > Apache APISIX:
> https://github.com/apache/apisix
> > > > > > >> > > > > > >
> > > > > > >> > > > > > --
> > > > > > >> > > > > > Best regards
> > > > > > >> > > > > > Alex Zhang
> > > > > > >> > > > > >
> > > > > > >> > > > > > https://github.com/tokers
> > > > > > >> > > > > > https://github.com/upyun/upyun-resty
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >> *MembPhis*
> > > > > > >> My GitHub: https://github.com/membphis
> > > > > > >> Apache APISIX: https://github.com/apache/apisix
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Reply via email to