You could learn from the Satellite project design,
1. Discuss here, https://github.com/apache/skywalking/issues/5871
2. Final design posted as a blog,
http://skywalking.apache.org/blog/2020-11-25-skywalking-satellite-0.1.0-design/

People could learn from these things when you join the community in the
future.

Sheng Wu 吴晟
Twitter, wusheng1108


Hoshea Jiang <[email protected]> 于2020年11月26日周四 下午1:38写道:

> I agree with the conclusion, and I hope to have some diagrams such as
> flowcharts or data flow diagrams, which help provide an overview of the E2E
> test before we start to do this.
>
> Ke Zhang <[email protected]> 于2020年11月26日周四 下午12:34写道:
>
> > I think supporting both docker-compose and KIND in the E2E test is
> > reasonable and  necessary, and I also agree with other conclusions.😀
> >
> > kezhenxu94@apache <[email protected]> 于2020年11月26日周四 上午11:58写道:
> >
> > >
> > > > 1. Docker-compose and kind are 2 options to set up the environments.
> > > > 2. Totally replace the Java and Maven based driving system.
> > > > 3. Enhance CLI to validate the GraphQL
> > > > 4. Keep the agent test tool unchanged for now as it is already a
> > separate
> > > > system from the e2e.
> > >
> > > This conclusion is good for me.
> > >
> > > What are others opinions?
> > >
> > > —————————
> > > Zhenxu Ke (柯振旭)
> > > GitHub @kezhenxu94
> > >
> > > > On Nov 17, 2020, at 2:42 PM, Sheng Wu <[email protected]>
> > wrote:
> > > >
> > > > Using k8s and docker-compose as 2 options in the test process is
> > > > reasonable, and should be supported.
> > > >
> > > > Let's keep the conclusion as simple as possible. In the next
> generation
> > > e2e
> > > > framework
> > > > 1. Docker-compose and kind are 2 options to set up the environments.
> > > > 2. Totally replace the Java and Maven based driving system.
> > > > 3. Enhance CLI to validate the GraphQL
> > > > 4. Keep the agent test tool unchanged for now as it is already a
> > separate
> > > > system from the e2e.
> > > >
> > > > Are these good enough for everyone?
> > > >
> > > > Sheng Wu 吴晟
> > > > Twitter, wusheng1108
> > > >
> > > >
> > > > Hongtao Gao <[email protected]> 于2020年11月17日周二 下午12:41写道:
> > > >
> > > >>>
> > > >>> We can see from above, your discussion actually is only related to
> 3.
> > > >>
> > > >>
> > > >> That's the only I intend to discuss here if you notice my first
> > > >> response('May SWCK help about provision and demo") in this thread.
> For
> > > >> other parts, I don't have any tips to share, though.
> > > >> Let me explain it more clearly. I want kubernetes to be seen as an
> > > >> alternative to docker-compose to play a running-engine role in the
> > test
> > > >> framework. If kubernetes can't replace
> > > >> the latter one Is it able to participate in it to solve the
> dedicated
> > or
> > > >> special issues.
> > > >>
> > > >> Our e2e test and plugin test frameworks are already top-level
> > complicity
> > > >>> thing in the worldwide open source field. Please don't over-design
> > it,
> > > >> and
> > > >>> too aggressive, the world will stay in the hybrid env for a very
> long
> > > >> time,
> > > >>> same as the developer.
> > > >>
> > > >>
> > > >> Reusing current elaborate output might be reasonable. But in this
> > > thread,
> > > >> we're talking about the potential solution to build a
> > > >> next-generation framework, which might mean that the current
> framework
> > > >> is too complicated to maintain; we need a more tiny way to support
> > more
> > > and
> > > >> more cases.
> > > >>
> > > >> And finally, SWCK is gonna and has to build a similar test
> framework.
> > If
> > > >> the test framework opts for the kubernetes solution, we could share
> > test
> > > >> cases and the underlying framework. It's a more efficient
> > > >> and consistent path for the entire system.
> > > >>
> > > >> regards, Hongtao.
> > > >>
> > > >> Sheng Wu <[email protected]> 于2020年11月17日周二 上午10:33写道:
> > > >>
> > > >>> Hi
> > > >>>
> > > >>> I don't think we have to make the decision on k8s or out-of-k8s.
> > > >>> e2e and all integration tests have their scenarios like SkyWalking
> > have
> > > >>> multiple languages based and mesh-based, even Prometheus adoption.
> > > >>> The same thing happens on the test field, k8s, and out of k8s.
> > > >>>
> > > >>> I would like to agree with both of you, k8s is needed, no matter in
> > > mesh
> > > >>> solution(ALS, telemetry v2), but also not anyone needs that, such
> as
> > > >> agent
> > > >>> developer. We wouldn't deny the fact, the docker-compose is more
> > > >>> lightweight and easier to learn.
> > > >>> So, let me get this straight, for the test you need
> > > >>> 1. Build the source from different repos
> > > >>> 2. Build the images
> > > >>> 3. Use some platform(docker-compose or k8s) to run the images in
> one
> > > >> piece.
> > > >>> 4. Give some inputs to the env
> > > >>> 5. Use some tools/ways to check what should happen according to
> (4)'s
> > > >> input
> > > >>> 6. Checked or timeout failure.
> > > >>>
> > > >>> We can see from above, your discussion actually is only related to
> 3.
> > > >> Let's
> > > >>> not see this as a battle, we need both. Many developers/users use
> OAP
> > > and
> > > >>> SkyWalking out of k8s, that is a simple fact.
> > > >>> Our e2e test and plugin test frameworks are already top-level
> > > complicity
> > > >>> thing in the worldwide open source field. Please don't over-design
> > it,
> > > >> and
> > > >>> too aggressive, the world will stay in the hybrid env for a very
> long
> > > >> time,
> > > >>> same as the developer.
> > > >>>
> > > >>> Also, let's keep in mind, k8s is popular becomes it doesn't require
> > the
> > > >>> developers to understand as the VM did. We as an open-source
> > community,
> > > >> are
> > > >>> focusing on the developer's capabilities.
> > > >>>
> > > >>> Sheng Wu 吴晟
> > > >>> Twitter, wusheng1108
> > > >>>
> > > >>>
> > > >>> Hongtao Gao <[email protected]> 于2020年11月17日周二 上午12:07写道:
> > > >>>
> > > >>>>>
> > > >>>>> but I have been thinking whether or not it is overkill for
> testing
> > > >>>>> purposes to introduce Kubernetes things.
> > > >>>>
> > > >>>>
> > > >>>> we all know the fact docker-compose is more portable than
> > Kubernetes,
> > > >>>> friendly to local running. But there're also some benefits to
> > replace
> > > >> it
> > > >>>> with kubernetes system:
> > > >>>>
> > > >>>> 1. SWCK has to test based on a real Kubernetes environment if the
> > main
> > > >>>> repo test framework doesn't support it(following the
> docker-compose
> > > >>> stack),
> > > >>>> That means skywalking ecosystem MUST introduce kubernetes which is
> > not
> > > >> an
> > > >>>> optional component.
> > > >>>>
> > > >>>> 2. Kubernetes support to scale applications at runtime, that help
> > > >>> improve
> > > >>>> the process of e2e. For instance, we could merge a single instance
> > and
> > > >>>> cluster test, by scaling up replicas. This strategy will reduce
> the
> > > >> total
> > > >>>> time sharply.
> > > >>>>
> > > >>>> If we are going to use KIND, I hope we can give a script or
> > something
> > > >>>>> similar to install all the needed components and create cluster
> for
> > > >>>>> convenient testing locally, Ke Zhang and Huaxi.
> > > >>>>
> > > >>>>
> > > >>>> That's SWCK's capacity to provide a standard and simple way to
> > create
> > > a
> > > >>>> cluster in every environment,
> > > >>>>
> > > >>>> kezhenxu94@apache <[email protected]> 于2020年11月16日周一
> 下午7:00写道:
> > > >>>>
> > > >>>>> Thanks Hongtao’s suggestion, I’ve thought about adopting
> > > >>> Kubernetes-based
> > > >>>>> tech stack (e.g. minikube or KIND), I personally prefer KIND too,
> > > >> but I
> > > >>>>> have been thinking whether or not it is overkill for testing
> > purposes
> > > >>> to
> > > >>>>> introduce Kubernetes things.
> > > >>>>>
> > > >>>>> Since you mentioned this, we are open to discuss this much more
> > > >> deeply:
> > > >>>>>
> > > >>>>> It would be definitely a benefit to involve more “end”s in the
> > > >> so-call
> > > >>>>> “end to end” tests, I’m +1 to leverage SWCK as well.
> > > >>>>>
> > > >>>>> If we are going to use KIND, I hope we can give a script or
> > something
> > > >>>>> similar to install all the needed components and create cluster
> for
> > > >>>>> convenient testing locally, Ke Zhang and Huaxi.
> > > >>>>>
> > > >>>>>
> > > >>>>> —————————
> > > >>>>> Zhenxu Ke (柯振旭)
> > > >>>>> GitHub @kezhenxu94
> > > >>>>>
> > > >>>>>> On Nov 16, 2020, at 4:03 PM, Hongtao Gao <[email protected]>
> > > >>> wrote:
> > > >>>>>>
> > > >>>>>> May SWCK help about provision and demo
> > > >>>>>>
> > > >>>>>> 1. SWCK is able to provide a stable and healthy OAP standalone
> > > >>> instance
> > > >>>>> or
> > > >>>>>> cluster for e2e. We could compose indicated CR yamls for
> different
> > > >>>>>> scenarios.
> > > >>>>>> 2. Different demo for different cases is mandatory for e2e and
> > > >> swck,
> > > >>> we
> > > >>>>>> could build a group of demo projects for them. With Kubernetes's
> > > >>>> support,
> > > >>>>>> we could
> > > >>>>>>   get free of verbose scripts that are written for installation,
> > > >>>>> checking
> > > >>>>>> the state of running applications, collecting logs.
> > > >>>>>>
> > > >>>>>> Ppl might have concerts about the minkube we currently have,
> > > >> which's
> > > >>>> hard
> > > >>>>>> to debug and runs slowly.
> > > >>>>>>
> > > >>>>>> I prefer to kind[1] instead of minkube. From my experience, kind
> > > >>> works
> > > >>>>> fine
> > > >>>>>> in an e2e scenario(limited cpu, memory resources), and it is run
> > > >>>> million
> > > >>>>>> times
> > > >>>>>> on my team's e2e environment, which proves the stability and
> > > >>>>>> maintainability of it.
> > > >>>>>>
> > > >>>>>> * 1.https://kind.sigs.k8s.io/docs/user/quick-start/
> > > >>>>>>
> > > >>>>>> regards, Hongtao
> > > >>>>>>
> > > >>>>>> Sheng Wu <[email protected]> 于2020年11月16日周一 上午8:30写道:
> > > >>>>>>
> > > >>>>>>> Inline
> > > >>>>>>>
> > > >>>>>>> kezhenxu94@apache <[email protected]> 于2020年11月15日周日
> > > >> 下午11:16写道:
> > > >>>>>>>
> > > >>>>>>>> Hi Jiajing,
> > > >>>>>>>>
> > > >>>>>>>> Thanks for sharing your context and joining the discussion.
> It’s
> > > >>> true
> > > >>>>>>> that
> > > >>>>>>>> we’re actually tackling the same problems in this domain. Some
> > of
> > > >>> the
> > > >>>>>>>> issues you mentioned are solved in the current E2E framework
> > > >> (like
> > > >>>>>>> `retry`,
> > > >>>>>>>> `timeout`, etc.), while some others are still there without
> > ideal
> > > >>>>>>> solution.
> > > >>>>>>>> Other discussions, please see my comments inline.
> > > >>>>>>>>
> > > >>>>>>>>> - the first issue is useful in health check steps, just like
> > the
> > > >>>> ones
> > > >>>>>>>>> defined in readiness probe and liveness probe in kubelet.
> > > >>>>>>>>
> > > >>>>>>>> Yes, the reason why we decided to pick docker-compose is that,
> > it
> > > >>> has
> > > >>>>> the
> > > >>>>>>>> same functionality with kubelet in terms of dependent services
> > > >>>> startup
> > > >>>>>>>> order and healthiness checking, while it’s more lite-weight
> and
> > > >>> easy
> > > >>>>> for
> > > >>>>>>>> both developers and GitHub Actions environment. Right?
> > > >>>>>>>>
> > > >>>>>>>>> - the second issue is critical for highly-customized
> > assertions,
> > > >>> we
> > > >>>>>>>>> can leverage the `text/template` module to provide highly
> > > >>> extensive
> > > >>>>>>>>> assertion functions
> > > >>>>>>>>
> > > >>>>>>>> `text/template` is also the first thought that came to my mind
> > > >> when
> > > >>>>>>>> considering customized assertors, customized assertors are for
> > > >> sure
> > > >>>>>>>> critical in SkyWalking E2E tests because we have many kinds of
> > > >>>>>>> assertions,
> > > >>>>>>>> some of which are not foreseen until we actually need them,
> > > >>>>>>>> `AtLeastOneGreaterThanZero` is an example.
> > > >>>>>>>>
> > > >>>>>>>>> - the last but not the least, the lifecycle hooks can be used
> > > >> for
> > > >>>>>>>>> accurate control of test setup and teardown, such as
> > > >>> `docker-compose
> > > >>>>>>>>> build, up -d, down`
> > > >>>>>>>>
> > > >>>>>>>> The lifecycle hooks are what I didn’t take into consideration
> > > >>>> because I
> > > >>>>>>>> thought it is quite natural for test framework to provide this
> > > >>>>> features,
> > > >>>>>>>> maybe Ke Zhang and Huaxi can do some research on this part.
> > > >>>>>>>>
> > > >>>>>>>>> Together with the aforementioned ideas, we will have a
> > > >> go-written,
> > > >>>>>>>>> yaml-data driven e2e testing framework which can overcome the
> > > >>>> current
> > > >>>>>>>>> problems that occur,
> > > >>>>>>>>
> > > >>>>>>>> I agree.
> > > >>>>>>>>
> > > >>>>>>>>> in which the most annoying issue is the fragmentation of
> > > >> different
> > > >>>>>>>>> scripts in different submodules, IMO.
> > > >>>>>>>>
> > > >>>>>>>> That’s why I list the last item in the previous email, (Nice
> to
> > > >>> have,
> > > >>>>>>> wrap
> > > >>>>>>>> them into a GitHub Action), with that GitHub Action, we can
> > > >> simply
> > > >>>>>>>> configure a workflow/job in every submodule, and the tests are
> > > >>>> executed
> > > >>>>>>>> with the shared scripts, no copy-pasting.
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>> Note, GitHub Action will require an Apache release every time
> > > >>> because
> > > >>>> we
> > > >>>>>>> are distributing the binary.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Sheng Wu 吴晟
> > > >>>>>>> Twitter, wusheng1108
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> Regards,
> > > >>>>>>>>> Jiajing
> > > >>>>>>>>>
> > > >>>>>>>>> On Sun, Nov 15, 2020 at 9:43 PM kezhenxu94@apache <
> > > >>>>>>> [email protected]>
> > > >>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> So, we would provide a script set to drive this?
> > > >>>>>>>>>>
> > > >>>>>>>>>> Yes, a script or a GoLang-written CLI, it depends.
> > > >>>>>>>>>>
> > > >>>>>>>>>>> My question is more about,
> > > >>>>>>>>>>> how to work with docker-compose
> > > >>>>>>>>>>
> > > >>>>>>>>>> It’s the same as what we’re doing now in the current version
> > of
> > > >>> E2E
> > > >>>>>>>> tests, now we have many `docker-compose.yaml` files, and all
> the
> > > >>>>>>>> `docker-compose.yaml` files are reusable in the new framework,
> > we
> > > >>>> will
> > > >>>>>>> keep
> > > >>>>>>>> this mechanism as it is proved to be convenient and easy to
> use
> > > >>>>> (remember
> > > >>>>>>>> when we added an ElasticSearch 7.9 case, and the TiDB storage
> > E2E
> > > >>>> test,
> > > >>>>>>> not
> > > >>>>>>>> yet merged though, it’s fast/easy to add a new case).
> > > >>>>>>>>>>
> > > >>>>>>>>>> —————————
> > > >>>>>>>>>> Zhenxu Ke (柯振旭)
> > > >>>>>>>>>> GitHub @kezhenxu94
> > > >>>>>>>>>>
> > > >>>>>>>>>>> On Nov 15, 2020, at 9:29 PM, Sheng Wu <
> > > >>> [email protected]>
> > > >>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> kezhenxu94@apache <[email protected]> 于2020年11月15日周日
> > > >>>> 下午9:16写道:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> I want to ask, how the docker-compose lands in your
> mind? I
> > > >>>> think
> > > >>>>>>> CLI
> > > >>>>>>>>>>>> can't
> > > >>>>>>>>>>>>> control this part.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> docker-compose just fits the requirements perfectly that
> we
> > > >>> need,
> > > >>>>> we
> > > >>>>>>>> have
> > > >>>>>>>>>>>> been using it in the current E2E and turns out it has
> > > >>> advantages
> > > >>>> in
> > > >>>>>>>>>>>> starting many services correctly with health check and
> > > >>> dependency
> > > >>>>>>>>>>>> declaration. (One may say Kubernetes can do the same
> thing,
> > > >> but
> > > >>>>> it’s
> > > >>>>>>>> kind
> > > >>>>>>>>>>>> of heavy in E2E and GitHub Actions and complex).
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> The SkyWalking-CLI won’t get involved with docker-compose
> > > >>> related
> > > >>>>>>>> things,
> > > >>>>>>>>>>>> it just provides query ability (the same as what it is
> doing
> > > >>> now,
> > > >>>>>>>> just need
> > > >>>>>>>>>>>> to check whether it covers all the needed query interfaces
> > or
> > > >>>> not),
> > > >>>>>>>> we will
> > > >>>>>>>>>>>> have a dedicated control program to
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> (1) start docker-compose (existing standard mechanism, no
> > new
> > > >>>>>>>> knowledge);
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> So, we would provide a script set to drive this? My
> question
> > > >> is
> > > >>>> more
> > > >>>>>>>> about,
> > > >>>>>>>>>>> how to work with docker-compose, rather than challanging
> the
> > > >>>> choice.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Sheng Wu 吴晟
> > > >>>>>>>>>>> Twitter, wusheng1108
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> (2) query actual data (by SkyWalking-CLI, existing
> > repository
> > > >>> and
> > > >>>>>>>> codes);
> > > >>>>>>>>>>>> (3) verify the actual data (by verification tool, a new
> > tool,
> > > >>>>> TODO);
> > > >>>>>>>>>>>> (4) and determine the test result (success or failed);
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> —————————
> > > >>>>>>>>>>>> Zhenxu Ke (柯振旭)
> > > >>>>>>>>>>>> GitHub @kezhenxu94
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> Sheng Wu 吴晟
> > > >>>>>>>>>>>>> Twitter, wusheng1108
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> kezhenxu94@apache <[email protected]> 于2020年11月15日周日
> > > >>>>> 下午5:27写道:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Hi the community, I want to raise a discussion to build
> > the
> > > >>>> next
> > > >>>>>>>>>>>>>> generation of E2E test framework for Apache SkyWalking,
> > > >> which
> > > >>>> may
> > > >>>>>>>> not
> > > >>>>>>>>>>>> be a
> > > >>>>>>>>>>>>>> high priority but a necessity.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> As we already know, tests in SkyWalking play a very
> > > >> important
> > > >>>>> role
> > > >>>>>>>> in
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>> contribution process, and the current E2E tests just
> work
> > > >>> well
> > > >>>> to
> > > >>>>>>>> verify
> > > >>>>>>>>>>>>>> every single pull request before merging, so why bother
> to
> > > >>>> build
> > > >>>>>>> the
> > > >>>>>>>>>>>> next
> > > >>>>>>>>>>>>>> generation of E2E test framework?
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> 1. In the SkyWalking's ecosystem, there are many
> > > >> sub-projects
> > > >>>>> that
> > > >>>>>>>> vary
> > > >>>>>>>>>>>> in
> > > >>>>>>>>>>>>>> terms of language, almost all of them need E2E tests to
> > > >> help
> > > >>> us
> > > >>>>>>>> verify
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>> pull requests, but we can see that they reimplement the
> > E2E
> > > >>>> test
> > > >>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>> in their languages, (or even worse, some of them don’t
> > have
> > > >>> E2E
> > > >>>>>>>> tests at
> > > >>>>>>>>>>>>>> all). Reimplementing the E2E test framework is
> unnecesarry
> > > >>> and
> > > >>>>>>>>>>>> introduces
> > > >>>>>>>>>>>>>> more duplicated work when adding a new test case. The
> > > >>> ultimate
> > > >>>>>>> goal
> > > >>>>>>>> is
> > > >>>>>>>>>>>> to
> > > >>>>>>>>>>>>>> reuse all the test cases when needed.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> 2. The current E2E framework is built with Java, running
> > > >>>>>>> Java-based
> > > >>>>>>>>>>>>>> program is not an easy thing for non-Java developers
> > > >>>> (maintainers
> > > >>>>>>> of
> > > >>>>>>>>>>>> other
> > > >>>>>>>>>>>>>> sub-projects such as PHP, C++, LUA, etc.), they need to
> > set
> > > >>> up
> > > >>>>> the
> > > >>>>>>>>>>>>>> environment correctly and then run the tests, neither is
> > it
> > > >>> an
> > > >>>>>>> easy
> > > >>>>>>>>>>>> skill
> > > >>>>>>>>>>>>>> to debug the tests.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> 3. It’s a good opportunity to optimize the E2E tests
> > > >> because
> > > >>> we
> > > >>>>>>>> added
> > > >>>>>>>>>>>> many
> > > >>>>>>>>>>>>>> cases and many duplicated codes that need to be removed.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Therefore we're planning to build an unified,
> easy-to-use,
> > > >>> and
> > > >>>>>>> fast
> > > >>>>>>>> E2E
> > > >>>>>>>>>>>>>> test framework to benifit all the sub-projects.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Here are some rough ideas about this framework:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> 0. (Unchanged) We will continue to use
> docker-compose.yaml
> > > >> to
> > > >>>>>>>>>>>> orchestrate
> > > >>>>>>>>>>>>>> the services that are to be tested, it’s proved to be
> > > >>> friendly
> > > >>>>> and
> > > >>>>>>>>>>>>>> debuggable because we can start it directly even if we
> are
> > > >>> not
> > > >>>>>>>> writing
> > > >>>>>>>>>>>> E2E
> > > >>>>>>>>>>>>>> tests.
> > > >>>>>>>>>>>>>> 1. This framework will take full advantages of the CLI
> > > >>> project
> > > >>>> to
> > > >>>>>>>> query
> > > >>>>>>>>>>>>>> the traces/metrics/logs, we don’t want to maintain two
> > > >>> versions
> > > >>>>> of
> > > >>>>>>>> query
> > > >>>>>>>>>>>>>> codes anymore, (now we have a version in the test codes
> > and
> > > >>> one
> > > >>>>> in
> > > >>>>>>>> the
> > > >>>>>>>>>>>> CLI).
> > > >>>>>>>>>>>>>> 2. We will build a CLI tool to compare the expected data
> > > >> and
> > > >>>> the
> > > >>>>>>>> actual
> > > >>>>>>>>>>>>>> data, we now have a custom schema of the expected data
> > yaml
> > > >>>> file,
> > > >>>>>>>> the
> > > >>>>>>>>>>>>>> verification codes should be packaged into an executable
> > > >>>> command
> > > >>>>>>> so
> > > >>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>> can be executed standalone without Java and maven
> > > >> knowledge.
> > > >>> I
> > > >>>>>>> hope
> > > >>>>>>>> we
> > > >>>>>>>>>>>> can
> > > >>>>>>>>>>>>>> leverage existing standards to write the YAML files and
> do
> > > >>>>>>>>>>>> verifications.
> > > >>>>>>>>>>>>>> 3. Build an orchestrating tool to start the
> > > >>>>> `docker-compose.yaml`,
> > > >>>>>>>>>>>> health
> > > >>>>>>>>>>>>>> check, query actual data, and verify the result.
> > > >>>>>>>>>>>>>> 4. (Nice to have) wrap all the tools as a GitHub Actions
> > to
> > > >>>> share
> > > >>>>>>>>>>>> between
> > > >>>>>>>>>>>>>> sub-projects.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> If you have any other better idea or want to complement
> to
> > > >>> this
> > > >>>>>>>>>>>> proposal,
> > > >>>>>>>>>>>>>> please reply here. I will create an issue to track these
> > > >>> tasks
> > > >>>>>>>> later.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> We have two students (who just finished the Summer Code
> > > >> 2020
> > > >>>>>>>> Projects)
> > > >>>>>>>>>>>> to
> > > >>>>>>>>>>>>>> lead this part, Ke Zhang [1] and Huaxi Jiang [2], as I
> > said
> > > >>>> this
> > > >>>>>>> is
> > > >>>>>>>> a
> > > >>>>>>>>>>>> not
> > > >>>>>>>>>>>>>> high priority, you have adequate time to get yourselves
> > > >>>> familiar
> > > >>>>>>>> with
> > > >>>>>>>>>>>> how
> > > >>>>>>>>>>>>>> the components (OAP, WebUI, CLI, storages, etc.) of
> > > >>> SkyWalking
> > > >>>>>>> work
> > > >>>>>>>> as
> > > >>>>>>>>>>>> well
> > > >>>>>>>>>>>>>> as the ecosystem. Thanks for your interests and
> > willingness
> > > >>> to
> > > >>>>>>> help.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> [1] https://github.com/Humbertzhang
> > > >>>>>>>>>>>>>> [2] https://github.com/fgksgf/
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> —————————
> > > >>>>>>>>>>>>>> Zhenxu Ke (柯振旭)
> > > >>>>>>>>>>>>>> GitHub @kezhenxu94
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> —————————
> > > >>>>>>>>>>>> Zhenxu Ke (柯振旭)
> > > >>>>>>>>>>>> GitHub @kezhenxu94
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> —————————
> > > >>>>>>>> Zhenxu Ke (柯振旭)
> > > >>>>>>>> GitHub @kezhenxu94
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>

Reply via email to