Re: [DISCUSS] Add Apache ShardingSphere as the server storage for Apache SkyWalking

[Juan Pan]

Hi Sheng and Daming,


Today, i guess i can propose one sharding strategy for us to discuss, which 
means sub-task 2 
(Analyse those tables and query SQLs to explore appropriate sharding) will be 
done.
Once the sharding strategy is settled down by us, 
i am going to work on sub-task 3(Provide class implementation 
ShardingAlgorithm), meanwhile, Daming can prepare configuration of Sharding 
Proxy.
After that, e2e tests will come behind.


It is my plan draft, looking forward to your suggestions.


 Juan Pan (Trista) 
                         
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panj...@apache.org




On 02/17/2020 14:01，Sheng Wu<wu.sheng.841...@gmail.com> wrote：
Daming <zt...@foxmail.com> 于2020年2月17日周一 上午10:33写道：


Could I pick up this two sub-tasks?

4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]


Hi Juan

I think you have a draft about this already?

Sheng Wu 吴晟
Twitter, wusheng1108



5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]


This would be helpful. You are more familiar with the e2e than her.





—————————
Daming(@dmsolr)
Apache SkyWalking


在 2020年2月17日，上午9:58，Juan Pan <panj...@apache.org> 写道：

That exactly what i concerned, moreover thanks for your joining.
Could you tell me which part you are interested in, so that we can share
the task list and work together. :-)


Juan Pan (Trista)

Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panj...@apache.org




On 02/17/2020 00:34，吴晟 Sheng Wu<wu.sh...@foxmail.com> wrote：
Hi


Which part or both?




Sheng&nbsp;Wu&nbsp;吴晟

From&nbsp;Wu&nbsp;Sheng&nbsp;'s&nbsp;phone.



------------------ Original ------------------
From: apmail-wusheng-owner <apmail-wusheng-ow...@apache.org&gt;
Date: Mon,Feb 17,2020 0:26 AM
To: dev <dev@skywalking.apache.org&gt;
Subject: Re: [DISCUSS] Add Apache ShardingSphere as the server storage
for Apache SkyWalking



hi&amp;nbsp;
I am glad to take part in.


发自我的iPhone

------------------ Original ------------------
From: Sheng Wu <wu.sheng.841...@gmail.com&amp;gt;
Date: Sun,Feb 16,2020 10:20 PM
To: dev <dev@skywalking.apache.org&amp;gt;
Subject: Re: [DISCUSS] Add Apache ShardingSphere as the server storage
for Apache SkyWalking



Juan Pan <panj...@apache.org&amp;gt; 于2020年2月16日周日 下午6:36写道：

&amp;gt; Hi fellows,
&amp;gt;
&amp;gt;
&amp;gt; I am working on providing ShardingSphere[1] as one of the
server storages
&amp;gt; for Skywalking.
&amp;gt; ShardingSphere will help shard the data collected by Skywalking
into
&amp;gt; different databases(Popular RDBMS, like MySQL, PostgreSQL and
so on) and
&amp;gt; tables to seek better query performance and do scale-out of
database.
&amp;gt;
&amp;gt;
&amp;gt; Since it is not an easy job, and needs some helps from
community, i put
&amp;gt; task schedule and discussion items here to welcome your
comments and
&amp;gt; suggestions.
&amp;gt; If you are interested in this one, welcome to be my partner! :-)
&amp;gt;
&amp;gt;
&amp;gt; ------------
&amp;gt; Here is the schedule,
&amp;gt; 1. Acquire all required tables and query SQLs from db logs,
like general
&amp;gt; log of MySQL. [DONE]
&amp;gt; 2. Analyse those tables and query SQLs to explore appropriate
sharding
&amp;gt; strategy.[DOING]
&amp;gt; 3. Provide class implementation of `PreciseShardingAlgorithm`
and
&amp;gt; `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
&amp;gt; 4. Provide configuration of Sharding Proxy[2] for
Skywalking.[TODO]

5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO


Yes. We call it e2e. You could find many storage implementation tests
there. Zhenxu Ke and Haochao Zhuang are the two leading this part.

BTW, your plan looks good to me.


Sheng Wu 吴晟
Twitter, wusheng1108


&amp;gt; ]
&amp;gt; 6. Update readme[3].[TODO]
&amp;gt; 7. Update storage doc[4].[TODO]
&amp;gt; -------------
&amp;gt;
&amp;gt;
&amp;gt; It seems a long-term work, and i will do it step by step. Any
progress
&amp;gt; will be updated here, also ISSUE [5] will give you thread of
PRs and
&amp;gt; discussion detail.
&amp;gt;
&amp;gt;
&amp;gt; [1] https://shardingsphere.apache.org/
&amp;gt; [2]
&amp;gt;
https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
&amp;gt; [3] https://github.com/apache/skywalking/tree/master/install
&amp;gt; [4 ]
&amp;gt;
https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
&amp;gt; [5] https://github.com/apache/skywalking/issues/4131
&amp;gt;
&amp;gt;
&amp;gt;
&amp;gt;
&amp;gt;
&amp;gt;&amp;nbsp; Juan Pan (Trista)
&amp;gt;
&amp;gt; Senior DBA &amp;amp; PPMC of Apache ShardingSphere(Incubating)
&amp;gt; E-mail: panj...@apache.org
&amp;gt;
&amp;gt;
&amp;gt;
&amp;gt;