Re: hbase build sizes
As I commented in my previous email, there was no other change made. On Wed, Mar 8, 2017 at 1:37 PM, Sean Busbey wrote: > Change is in place, I'll take a look at some build results tomorrow to > see if it's going well. > > In the mean time, just a heads up to the committers that contributors > will have questions if things fail and the logs didn't get properly > zipped up. > > @Ted Yu I also undid your change of keeping builds for only 3 days. > Please don't change the retention period; 3 days is not enough time > for us reasonably expect volunteer contributors to circle back on > getting their build results. If you updated the retention on any of > the other jobs please restore them. > > On Wed, Mar 8, 2017 at 3:26 PM, Sean Busbey wrote: > > Okay, I'm reasonably sure this is just test logs. I'm updating our > > builds to zip up the test logs as a post-build action, which should > > bring down our space consumption by a couple of orders of magnitude. > > > > > > On Wed, Mar 8, 2017 at 3:03 PM, Sean Busbey > wrote: > >> I grabbed one of the items Chris pointed me towards: > >> > >> https://builds.apache.org/job/PreCommit-HBASE-Build/5991/ > >> > >> If you download the archived artifacts as a zip, it's ~61MiB. If you > >> uncompressed that zip file, you get ~1.3GiB of logs. > >> > >> I'm going to try to grab a couple more to make sure it's consistently > test logs. > >> > >> On Wed, Mar 8, 2017 at 2:31 PM, Chris Lambertus wrote: > >>> Hi HBase folks, > >>> > >>> Just wanted to advise you of some build size issues we noticed on the > jenkins master. It looks like over a span of about 2 weeks, the hbase > builds are using a very large amount of storage in the retained builds > directory: > >>> > >>> 79G PreCommit-HBASE-Build > >>> 72G HBase-1.3-JDK8 > >>> 68G HBase-1.3-JDK7 > >>> 60G HBase-1.2-JDK7t > >>> 59G HBase-1.2-JDK8 > >>> 51G HBase-1.2-IT > >>> 36G HBase Website Link Ckecker > >>> > >>> Each stored build appears to use on the average of 600-1.1GB of disk > space, and there are over 200 builds in two weeks for PreCommit-HBASE-Build > alone. Some are much smaller (1-8MB.) > >>> > >>> We are running a bit low on storage on the jenkins master, so anything > you could help us do to reduce the footprint here would be appreciated. > >>> > >>> I’ve already discussed this a bit with Sean Busbey on Hipchat, he > asked me to follow up here. > >>> > >>> Thanks, > >>> > >>> -Chris > >>> ASF Infra > >>> > >> > >> > >> > >> -- > >> Sean > > > > > > > > -- > > Sean > > > > -- > Sean >
Re: hbase build sizes
Change is in place, I'll take a look at some build results tomorrow to see if it's going well. In the mean time, just a heads up to the committers that contributors will have questions if things fail and the logs didn't get properly zipped up. @Ted Yu I also undid your change of keeping builds for only 3 days. Please don't change the retention period; 3 days is not enough time for us reasonably expect volunteer contributors to circle back on getting their build results. If you updated the retention on any of the other jobs please restore them. On Wed, Mar 8, 2017 at 3:26 PM, Sean Busbey wrote: > Okay, I'm reasonably sure this is just test logs. I'm updating our > builds to zip up the test logs as a post-build action, which should > bring down our space consumption by a couple of orders of magnitude. > > > On Wed, Mar 8, 2017 at 3:03 PM, Sean Busbey wrote: >> I grabbed one of the items Chris pointed me towards: >> >> https://builds.apache.org/job/PreCommit-HBASE-Build/5991/ >> >> If you download the archived artifacts as a zip, it's ~61MiB. If you >> uncompressed that zip file, you get ~1.3GiB of logs. >> >> I'm going to try to grab a couple more to make sure it's consistently test >> logs. >> >> On Wed, Mar 8, 2017 at 2:31 PM, Chris Lambertus wrote: >>> Hi HBase folks, >>> >>> Just wanted to advise you of some build size issues we noticed on the >>> jenkins master. It looks like over a span of about 2 weeks, the hbase >>> builds are using a very large amount of storage in the retained builds >>> directory: >>> >>> 79GPreCommit-HBASE-Build >>> 72GHBase-1.3-JDK8 >>> 68GHBase-1.3-JDK7 >>> 60GHBase-1.2-JDK7t >>> 59GHBase-1.2-JDK8 >>> 51GHBase-1.2-IT >>> 36GHBase Website Link Ckecker >>> >>> Each stored build appears to use on the average of 600-1.1GB of disk space, >>> and there are over 200 builds in two weeks for PreCommit-HBASE-Build alone. >>> Some are much smaller (1-8MB.) >>> >>> We are running a bit low on storage on the jenkins master, so anything you >>> could help us do to reduce the footprint here would be appreciated. >>> >>> I’ve already discussed this a bit with Sean Busbey on Hipchat, he asked me >>> to follow up here. >>> >>> Thanks, >>> >>> -Chris >>> ASF Infra >>> >> >> >> >> -- >> Sean > > > > -- > Sean -- Sean
Re: hbase build sizes
For PreCommit-HBASE-Build, I have cut down days to keep builds from 14 to 3. For HBase-1.3-JDK8, the fields is empty. Not sure what the default is. "/configure" loaded vey slowly. I can try other builds later. Cheers On Wed, Mar 8, 2017 at 12:31 PM, Chris Lambertus wrote: > Hi HBase folks, > > Just wanted to advise you of some build size issues we noticed on the > jenkins master. It looks like over a span of about 2 weeks, the hbase > builds are using a very large amount of storage in the retained builds > directory: > > 79G PreCommit-HBASE-Build > 72G HBase-1.3-JDK8 > 68G HBase-1.3-JDK7 > 60G HBase-1.2-JDK7t > 59G HBase-1.2-JDK8 > 51G HBase-1.2-IT > 36G HBase Website Link Ckecker > > Each stored build appears to use on the average of 600-1.1GB of disk > space, and there are over 200 builds in two weeks for PreCommit-HBASE-Build > alone. Some are much smaller (1-8MB.) > > We are running a bit low on storage on the jenkins master, so anything you > could help us do to reduce the footprint here would be appreciated. > > I’ve already discussed this a bit with Sean Busbey on Hipchat, he asked me > to follow up here. > > Thanks, > > -Chris > ASF Infra > >
Re: hbase build sizes
Okay, I'm reasonably sure this is just test logs. I'm updating our builds to zip up the test logs as a post-build action, which should bring down our space consumption by a couple of orders of magnitude. On Wed, Mar 8, 2017 at 3:03 PM, Sean Busbey wrote: > I grabbed one of the items Chris pointed me towards: > > https://builds.apache.org/job/PreCommit-HBASE-Build/5991/ > > If you download the archived artifacts as a zip, it's ~61MiB. If you > uncompressed that zip file, you get ~1.3GiB of logs. > > I'm going to try to grab a couple more to make sure it's consistently test > logs. > > On Wed, Mar 8, 2017 at 2:31 PM, Chris Lambertus wrote: >> Hi HBase folks, >> >> Just wanted to advise you of some build size issues we noticed on the >> jenkins master. It looks like over a span of about 2 weeks, the hbase builds >> are using a very large amount of storage in the retained builds directory: >> >> 79GPreCommit-HBASE-Build >> 72GHBase-1.3-JDK8 >> 68GHBase-1.3-JDK7 >> 60GHBase-1.2-JDK7t >> 59GHBase-1.2-JDK8 >> 51GHBase-1.2-IT >> 36GHBase Website Link Ckecker >> >> Each stored build appears to use on the average of 600-1.1GB of disk space, >> and there are over 200 builds in two weeks for PreCommit-HBASE-Build alone. >> Some are much smaller (1-8MB.) >> >> We are running a bit low on storage on the jenkins master, so anything you >> could help us do to reduce the footprint here would be appreciated. >> >> I’ve already discussed this a bit with Sean Busbey on Hipchat, he asked me >> to follow up here. >> >> Thanks, >> >> -Chris >> ASF Infra >> > > > > -- > Sean -- Sean
Re: hbase build sizes
I grabbed one of the items Chris pointed me towards: https://builds.apache.org/job/PreCommit-HBASE-Build/5991/ If you download the archived artifacts as a zip, it's ~61MiB. If you uncompressed that zip file, you get ~1.3GiB of logs. I'm going to try to grab a couple more to make sure it's consistently test logs. On Wed, Mar 8, 2017 at 2:31 PM, Chris Lambertus wrote: > Hi HBase folks, > > Just wanted to advise you of some build size issues we noticed on the jenkins > master. It looks like over a span of about 2 weeks, the hbase builds are > using a very large amount of storage in the retained builds directory: > > 79GPreCommit-HBASE-Build > 72GHBase-1.3-JDK8 > 68GHBase-1.3-JDK7 > 60GHBase-1.2-JDK7t > 59GHBase-1.2-JDK8 > 51GHBase-1.2-IT > 36GHBase Website Link Ckecker > > Each stored build appears to use on the average of 600-1.1GB of disk space, > and there are over 200 builds in two weeks for PreCommit-HBASE-Build alone. > Some are much smaller (1-8MB.) > > We are running a bit low on storage on the jenkins master, so anything you > could help us do to reduce the footprint here would be appreciated. > > I’ve already discussed this a bit with Sean Busbey on Hipchat, he asked me to > follow up here. > > Thanks, > > -Chris > ASF Infra > -- Sean
hbase build sizes
Hi HBase folks, Just wanted to advise you of some build size issues we noticed on the jenkins master. It looks like over a span of about 2 weeks, the hbase builds are using a very large amount of storage in the retained builds directory: 79GPreCommit-HBASE-Build 72GHBase-1.3-JDK8 68GHBase-1.3-JDK7 60GHBase-1.2-JDK7t 59GHBase-1.2-JDK8 51GHBase-1.2-IT 36GHBase Website Link Ckecker Each stored build appears to use on the average of 600-1.1GB of disk space, and there are over 200 builds in two weeks for PreCommit-HBASE-Build alone. Some are much smaller (1-8MB.) We are running a bit low on storage on the jenkins master, so anything you could help us do to reduce the footprint here would be appreciated. I’ve already discussed this a bit with Sean Busbey on Hipchat, he asked me to follow up here. Thanks, -Chris ASF Infra signature.asc Description: Message signed with OpenPGP
