Re: [DISCUSS] about integration test
we can try to move CI to huawei cloud, that must faster than Travis if fast enough, then no need to make integration test independent 2018-07-11 6:59 GMT+08:00 Willem Jiang : > I think we may change the travis script a little bit, to find out if we can > run the integration test once per day. > > > > Willem Jiang > > Twitter: willemjiang > Weibo: 姜宁willem > > On Wed, Jul 11, 2018 at 12:45 AM, wjm wjm wrote: > > > multiple SC is no problem, infact, not all test case should be run for > > multiple SC > > > > the problem is Travis is so slow > > yes, we can not ask too much about that, so my suggestion is to make > > integration test independent > > that will not block our PR > > and integration test run once a day at least, can make sure there are not > > too many mistakes. > > > > for contributors, still must provide unit test case, and better to > provide > > integration test, that not too much different than current > > > > 2018-07-10 21:17 GMT+08:00 Willem Jiang : > > > > > As Travis is a free service for Open Source projects, we cannot ask too > > > much about that. > > > Current ASF provides Jenkins Service here[1], there are some > shortcoming > > > if we use it, such as lots of projects use it, and it doesn't > integrate > > > with github. > > > > > > For running the test case with different SC, I think we could leverage > > the > > > env properties[2] to run the test separately. > > > Current we run the saga build with Spring Boot 1.x and Spring Boot 2.x > > [3], > > > I think it quite easy to list all the SC versions as the env properties > > > those we need to support. > > > > > > [1]https://builds.apache.org/ > > > [2] > > > https://github.com/apache/incubator-servicecomb-saga/ > > > blob/master/.travis.yml#L28 > > > [3]https://travis-ci.org/apache/incubator-servicecomb-saga > > > > > > > > > Willem Jiang > > > > > > Twitter: willemjiang > > > Weibo: 姜宁willem > > > > > > On Tue, Jul 10, 2018 at 8:56 PM, wjm wjm wrote: > > > > > > > yes > > > > everything no problem. but: > > > > > > > > 1.Travis is so slow, not our integration test too slow > > > > when run in our development environment, it's very fast. > > > > 2.when run all test case multi times with different SC, will produce > > too > > > > many log, that will treat Travis log size limit. > > > > > > > > 2018-07-10 19:23 GMT+08:00 Sure : > > > > > > > > > Agree! > > > > > > > > > > > > > > > 1. Just like willem.jiang said, we can push multiple SC versions to > > > > docker > > > > > hub > > > > > 2. Chassis integration test can run with different SC version in > > every > > > > time > > > > > > > > > > > > > > > > > > > > > > > > > -- 原始邮件 -- > > > > > 发件人: "willem.jiang"; > > > > > 发送时间: 2018年7月10日(星期二) 下午5:16 > > > > > 收件人: "dev"; > > > > > > > > > > 主题: Re: [DISCUSS] about integration test > > > > > > > > > > > > > > > > > > > > Current ServiceComb CI just leverage service of Travis[1] which > has a > > > > very > > > > > good integration with Github. > > > > > > > > > > Current we don't test the multiple SC at same time, but I think > with > > > the > > > > > help of docker, it could easy for us to run the examples against > > > multiple > > > > > version of SC. > > > > > From the proposal, we need multiple node to handle the test, it > could > > > be > > > > a > > > > > challenge for us if we still use Travis. > > > > > > > > > > If we put the integration test in a separated project we may need > to > > > > find a > > > > > way to let the integration test looks up the latest snap short > > artifact > > > > for > > > > > running the test. > > > > > > > > > > Any thought? > > > > > > > > > > > > > > > > > > > > > > > > > Willem Jiang > > > > > > > > > > Twitter: willemjiang > > > > > Weibo: 姜宁willem > > > > > > > > > > On Tue, Jul 10, 2018 at 9:41 AM, wjm wjm wrote: > > > > > > > > > > > currently we embed integration test with PR CI > > > > > > > > > > > > but: > > > > > > 1.we will add more and more integration test case > > > > > > 2.test time will be more and more longer > > > > > > 3.integration test should include work with multiple SC versions > > > > > > 4... > > > > > > > > > > > > so, maybe it's better that make integration test to be a > > independent > > > > > > project, and run it once a day at least > > > > > > > > > > > > some other idea of integration test: > > > > > > https://issues.apache.org/jira/browse/SCB-719 > > > > > > > > > > > > what's your suggestion? > > > > > > > > > > > > > > > > > > > > >
Re: [DISCUSS] about integration test
I think we may change the travis script a little bit, to find out if we can run the integration test once per day. Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Wed, Jul 11, 2018 at 12:45 AM, wjm wjm wrote: > multiple SC is no problem, infact, not all test case should be run for > multiple SC > > the problem is Travis is so slow > yes, we can not ask too much about that, so my suggestion is to make > integration test independent > that will not block our PR > and integration test run once a day at least, can make sure there are not > too many mistakes. > > for contributors, still must provide unit test case, and better to provide > integration test, that not too much different than current > > 2018-07-10 21:17 GMT+08:00 Willem Jiang : > > > As Travis is a free service for Open Source projects, we cannot ask too > > much about that. > > Current ASF provides Jenkins Service here[1], there are some shortcoming > > if we use it, such as lots of projects use it, and it doesn't integrate > > with github. > > > > For running the test case with different SC, I think we could leverage > the > > env properties[2] to run the test separately. > > Current we run the saga build with Spring Boot 1.x and Spring Boot 2.x > [3], > > I think it quite easy to list all the SC versions as the env properties > > those we need to support. > > > > [1]https://builds.apache.org/ > > [2] > > https://github.com/apache/incubator-servicecomb-saga/ > > blob/master/.travis.yml#L28 > > [3]https://travis-ci.org/apache/incubator-servicecomb-saga > > > > > > Willem Jiang > > > > Twitter: willemjiang > > Weibo: 姜宁willem > > > > On Tue, Jul 10, 2018 at 8:56 PM, wjm wjm wrote: > > > > > yes > > > everything no problem. but: > > > > > > 1.Travis is so slow, not our integration test too slow > > > when run in our development environment, it's very fast. > > > 2.when run all test case multi times with different SC, will produce > too > > > many log, that will treat Travis log size limit. > > > > > > 2018-07-10 19:23 GMT+08:00 Sure : > > > > > > > Agree! > > > > > > > > > > > > 1. Just like willem.jiang said, we can push multiple SC versions to > > > docker > > > > hub > > > > 2. Chassis integration test can run with different SC version in > every > > > time > > > > > > > > > > > > > > > > > > > > -- 原始邮件 -- > > > > 发件人: "willem.jiang"; > > > > 发送时间: 2018年7月10日(星期二) 下午5:16 > > > > 收件人: "dev"; > > > > > > > > 主题: Re: [DISCUSS] about integration test > > > > > > > > > > > > > > > > Current ServiceComb CI just leverage service of Travis[1] which has a > > > very > > > > good integration with Github. > > > > > > > > Current we don't test the multiple SC at same time, but I think with > > the > > > > help of docker, it could easy for us to run the examples against > > multiple > > > > version of SC. > > > > From the proposal, we need multiple node to handle the test, it could > > be > > > a > > > > challenge for us if we still use Travis. > > > > > > > > If we put the integration test in a separated project we may need to > > > find a > > > > way to let the integration test looks up the latest snap short > artifact > > > for > > > > running the test. > > > > > > > > Any thought? > > > > > > > > > > > > > > > > > > > > Willem Jiang > > > > > > > > Twitter: willemjiang > > > > Weibo: 姜宁willem > > > > > > > > On Tue, Jul 10, 2018 at 9:41 AM, wjm wjm wrote: > > > > > > > > > currently we embed integration test with PR CI > > > > > > > > > > but: > > > > > 1.we will add more and more integration test case > > > > > 2.test time will be more and more longer > > > > > 3.integration test should include work with multiple SC versions > > > > > 4... > > > > > > > > > > so, maybe it's better that make integration test to be a > independent > > > > > project, and run it once a day at least > > > > > > > > > > some other idea of integration test: > > > > > https://issues.apache.org/jira/browse/SCB-719 > > > > > > > > > > what's your suggestion? > > > > > > > > > > > > > > >
Re: [DISCUSS] about integration test
multiple SC is no problem, infact, not all test case should be run for multiple SC the problem is Travis is so slow yes, we can not ask too much about that, so my suggestion is to make integration test independent that will not block our PR and integration test run once a day at least, can make sure there are not too many mistakes. for contributors, still must provide unit test case, and better to provide integration test, that not too much different than current 2018-07-10 21:17 GMT+08:00 Willem Jiang : > As Travis is a free service for Open Source projects, we cannot ask too > much about that. > Current ASF provides Jenkins Service here[1], there are some shortcoming > if we use it, such as lots of projects use it, and it doesn't integrate > with github. > > For running the test case with different SC, I think we could leverage the > env properties[2] to run the test separately. > Current we run the saga build with Spring Boot 1.x and Spring Boot 2.x [3], > I think it quite easy to list all the SC versions as the env properties > those we need to support. > > [1]https://builds.apache.org/ > [2] > https://github.com/apache/incubator-servicecomb-saga/ > blob/master/.travis.yml#L28 > [3]https://travis-ci.org/apache/incubator-servicecomb-saga > > > Willem Jiang > > Twitter: willemjiang > Weibo: 姜宁willem > > On Tue, Jul 10, 2018 at 8:56 PM, wjm wjm wrote: > > > yes > > everything no problem. but: > > > > 1.Travis is so slow, not our integration test too slow > > when run in our development environment, it's very fast. > > 2.when run all test case multi times with different SC, will produce too > > many log, that will treat Travis log size limit. > > > > 2018-07-10 19:23 GMT+08:00 Sure : > > > > > Agree! > > > > > > > > > 1. Just like willem.jiang said, we can push multiple SC versions to > > docker > > > hub > > > 2. Chassis integration test can run with different SC version in every > > time > > > > > > > > > > > > > > > -- 原始邮件 -- > > > 发件人: "willem.jiang"; > > > 发送时间: 2018年7月10日(星期二) 下午5:16 > > > 收件人: "dev"; > > > > > > 主题: Re: [DISCUSS] about integration test > > > > > > > > > > > > Current ServiceComb CI just leverage service of Travis[1] which has a > > very > > > good integration with Github. > > > > > > Current we don't test the multiple SC at same time, but I think with > the > > > help of docker, it could easy for us to run the examples against > multiple > > > version of SC. > > > From the proposal, we need multiple node to handle the test, it could > be > > a > > > challenge for us if we still use Travis. > > > > > > If we put the integration test in a separated project we may need to > > find a > > > way to let the integration test looks up the latest snap short artifact > > for > > > running the test. > > > > > > Any thought? > > > > > > > > > > > > > > > Willem Jiang > > > > > > Twitter: willemjiang > > > Weibo: 姜宁willem > > > > > > On Tue, Jul 10, 2018 at 9:41 AM, wjm wjm wrote: > > > > > > > currently we embed integration test with PR CI > > > > > > > > but: > > > > 1.we will add more and more integration test case > > > > 2.test time will be more and more longer > > > > 3.integration test should include work with multiple SC versions > > > > 4... > > > > > > > > so, maybe it's better that make integration test to be a independent > > > > project, and run it once a day at least > > > > > > > > some other idea of integration test: > > > > https://issues.apache.org/jira/browse/SCB-719 > > > > > > > > what's your suggestion? > > > > > > > > > >
Re: [DISCUSS] about integration test
As Travis is a free service for Open Source projects, we cannot ask too much about that. Current ASF provides Jenkins Service here[1], there are some shortcoming if we use it, such as lots of projects use it, and it doesn't integrate with github. For running the test case with different SC, I think we could leverage the env properties[2] to run the test separately. Current we run the saga build with Spring Boot 1.x and Spring Boot 2.x [3], I think it quite easy to list all the SC versions as the env properties those we need to support. [1]https://builds.apache.org/ [2] https://github.com/apache/incubator-servicecomb-saga/blob/master/.travis.yml#L28 [3]https://travis-ci.org/apache/incubator-servicecomb-saga Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Tue, Jul 10, 2018 at 8:56 PM, wjm wjm wrote: > yes > everything no problem. but: > > 1.Travis is so slow, not our integration test too slow > when run in our development environment, it's very fast. > 2.when run all test case multi times with different SC, will produce too > many log, that will treat Travis log size limit. > > 2018-07-10 19:23 GMT+08:00 Sure : > > > Agree! > > > > > > 1. Just like willem.jiang said, we can push multiple SC versions to > docker > > hub > > 2. Chassis integration test can run with different SC version in every > time > > > > > > > > > > -- 原始邮件 ------ > > 发件人: "willem.jiang"; > > 发送时间: 2018年7月10日(星期二) 下午5:16 > > 收件人: "dev"; > > > > 主题: Re: [DISCUSS] about integration test > > > > > > > > Current ServiceComb CI just leverage service of Travis[1] which has a > very > > good integration with Github. > > > > Current we don't test the multiple SC at same time, but I think with the > > help of docker, it could easy for us to run the examples against multiple > > version of SC. > > From the proposal, we need multiple node to handle the test, it could be > a > > challenge for us if we still use Travis. > > > > If we put the integration test in a separated project we may need to > find a > > way to let the integration test looks up the latest snap short artifact > for > > running the test. > > > > Any thought? > > > > > > > > > > Willem Jiang > > > > Twitter: willemjiang > > Weibo: 姜宁willem > > > > On Tue, Jul 10, 2018 at 9:41 AM, wjm wjm wrote: > > > > > currently we embed integration test with PR CI > > > > > > but: > > > 1.we will add more and more integration test case > > > 2.test time will be more and more longer > > > 3.integration test should include work with multiple SC versions > > > 4... > > > > > > so, maybe it's better that make integration test to be a independent > > > project, and run it once a day at least > > > > > > some other idea of integration test: > > > https://issues.apache.org/jira/browse/SCB-719 > > > > > > what's your suggestion? > > > > > >
Re: [DISCUSS] about integration test
yes everything no problem. but: 1.Travis is so slow, not our integration test too slow when run in our development environment, it's very fast. 2.when run all test case multi times with different SC, will produce too many log, that will treat Travis log size limit. 2018-07-10 19:23 GMT+08:00 Sure : > Agree! > > > 1. Just like willem.jiang said, we can push multiple SC versions to docker > hub > 2. Chassis integration test can run with different SC version in every time > > > > > -- 原始邮件 -- > 发件人: "willem.jiang"; > 发送时间: 2018年7月10日(星期二) 下午5:16 > 收件人: "dev"; > > 主题: Re: [DISCUSS] about integration test > > > > Current ServiceComb CI just leverage service of Travis[1] which has a very > good integration with Github. > > Current we don't test the multiple SC at same time, but I think with the > help of docker, it could easy for us to run the examples against multiple > version of SC. > From the proposal, we need multiple node to handle the test, it could be a > challenge for us if we still use Travis. > > If we put the integration test in a separated project we may need to find a > way to let the integration test looks up the latest snap short artifact for > running the test. > > Any thought? > > > > > Willem Jiang > > Twitter: willemjiang > Weibo: 姜宁willem > > On Tue, Jul 10, 2018 at 9:41 AM, wjm wjm wrote: > > > currently we embed integration test with PR CI > > > > but: > > 1.we will add more and more integration test case > > 2.test time will be more and more longer > > 3.integration test should include work with multiple SC versions > > 4... > > > > so, maybe it's better that make integration test to be a independent > > project, and run it once a day at least > > > > some other idea of integration test: > > https://issues.apache.org/jira/browse/SCB-719 > > > > what's your suggestion? > > >
Re: [DISCUSS] about integration test
Current ServiceComb CI just leverage service of Travis[1] which has a very good integration with Github. Current we don't test the multiple SC at same time, but I think with the help of docker, it could easy for us to run the examples against multiple version of SC. >From the proposal, we need multiple node to handle the test, it could be a challenge for us if we still use Travis. If we put the integration test in a separated project we may need to find a way to let the integration test looks up the latest snap short artifact for running the test. Any thought? Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Tue, Jul 10, 2018 at 9:41 AM, wjm wjm wrote: > currently we embed integration test with PR CI > > but: > 1.we will add more and more integration test case > 2.test time will be more and more longer > 3.integration test should include work with multiple SC versions > 4... > > so, maybe it's better that make integration test to be a independent > project, and run it once a day at least > > some other idea of integration test: > https://issues.apache.org/jira/browse/SCB-719 > > what's your suggestion? >
Re: [DISCUSS] about integration test
ServiceComb is an open source project that has been adopted for a variety of commercial scenarios. As wrote in the report, at least more than 10 users have used ServiceComb for commercial use. Therefore, our integration testing will be more and more comprehensive and huge, it is a good idea to resolve this problem, but directly separating the inheritance test to the independent project will cause some problems: 1. This is contrary to the ordinary practice of open source projects, causing the unconsistent experience for our developers. 2. When the developers commmit, they will need to commit to both projects, which will affect the uniform traceability of the test code and function code. 3. The results of the integration test will lag, which will have an impact on the community's timely feedback when submitting the function code. So my suggestion is whether it can be compromised: Select out the time-consuming parts of the integration test and then reduce the run cycle, but don't strip it into separate branches. Any other thoughts? Best Regards, --- Zen Lin zenlintechnofr...@gmail.com Focused on Micro Service and Apache ServiceComb wjm wjm 于2018年7月10日周二 上午9:41写道: > currently we embed integration test with PR CI > > but: > 1.we will add more and more integration test case > 2.test time will be more and more longer > 3.integration test should include work with multiple SC versions > 4... > > so, maybe it's better that make integration test to be a independent > project, and run it once a day at least > > some other idea of integration test: > https://issues.apache.org/jira/browse/SCB-719 > > what's your suggestion? >