I can help set this up on Azure. Give me some time to work out the details?
Thanks, Om On Sun, Apr 12, 2020, 11:49 AM Harbs <harbs.li...@gmail.com> wrote: > OK. Good to know. > > > On Apr 12, 2020, at 9:45 PM, Alex Harui <aha...@adobe.com.INVALID> > wrote: > > > > FWIW, I just looked and the longest job in the release steps so far is 8 > minutes. Most are under 2 minutes. There might be jobs later that take > longer that we haven't run yet. IMO, the issue isn't speed of the machine, > it is just that we are sharing the machine with longer jobs (1 hour for > TourDeFlexMigration). And again, the machine will be idle for stretches of > time while the RM verifies artifacts after each step. > > > > -Alex > > > > On 4/12/20, 11:32 AM, "Harbs" <harbs.li...@gmail.com <mailto: > harbs.li...@gmail.com>> wrote: > > > > Fair enough. > > > > I expect builds to be somewhere between 10 and 20 times faster on a > powerful machine. > > > > Yeah. It’s probably going to be a bit of work changing the server, > but probably worth it in the long run. > > > > I think I’ll try this when I do the next release unless Yishay wants > to work with me on this for this release — but I’m not going to be able to > help until after Passover (i.e. next week). > > > > Thanks, > > Harbs > > > >> On Apr 12, 2020, at 9:22 PM, Alex Harui <aha...@adobe.com.INVALID> > wrote: > >> > >> The Azure portal says: Standard F2s_v2 (2 vcpus, 4 GiB memory) > >> > >> I think I am reading changes to the build process in your suggestions. > I do not really want to spend more of my time on this process. But if you > want to do the work, that's fine with me. > >> > >> -Alex > >> > >> On 4/12/20, 10:57 AM, "Harbs" <harbs.li...@gmail.com <mailto: > harbs.li...@gmail.com> <mailto:harbs.li...@gmail.com <mailto: > harbs.li...@gmail.com>>> wrote: > >> > >> What kind of horsepower is included in the free Azure account? > >> > >> The server I mentioned builds (considerably) faster than my own local > machine. The ci server seems to build many times slower. > >> > >> One thing we can do to minimize running server time would be to > transfer the artifacts to storage instead of keeping them on the server. On > AWS, I’d probably use S3. Not sure what the similar service on Azure is > called. > >> > >>> On Apr 12, 2020, at 8:26 PM, Alex Harui <aha...@adobe.com.INVALID > <mailto:aha...@adobe.com.INVALID>> wrote: > >>> > >>> OK, that's pretty much how I understand Azure as well. The key thing > is that "running" includes time where the CI server is not running any > Jenkins jobs. The CI Server steps might take only a few hours of actual > server time, but there is time where the RM is verifying artifacts locally > so you'd be paying for that or the RM would have to keep shutting down and > restarting. > >>> > >>> Seems like it would be cheaper/simpler to get the free MSDN account > and leave it running. > >>> > >>> -Alex > >>> > >>> On 4/12/20, 10:15 AM, "Harbs" <harbs.li...@gmail.com <mailto: > harbs.li...@gmail.com>> wrote: > >>> > >>> My experience is with AWS. > >>> > >>> I assume Microsoft has similar offerings, but I don’t have experience > with Azure. > >>> > >>> AWS has on-demand EC2 instances which you pay for only the actual > time that they are running.[1] > >>> > >>> Instances can be started and stopped via command line (or via the web > interface) as long as you have valid credentials to do so. > >>> > >>> For example: an m5.4xlarge instance has 16 cores and costs about $1.5 > per hour. On a machine like that, a full build would probably take less > than 10 minutes. It’s probably possible to do a full release with only a > few hours of server time. > >>> > >>> Leaving a server like that running all the time would get expensive, > but if it’s just spun up for releases, you’d get very fast builds at a > reasonable price. > >>> > >>> I’d be happy to pay $10-$50 (and possibly more) per release to make > the release process painless for the RM. > >>> > >>> [1] > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > < > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > >< > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > < > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > >>< > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > < > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > >< > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > < > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7Cd99d1960963240ce500c08d7df0fe417%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637223131651425591&sdata=HPLHT0r9qTmJhR4f4j52wtslRJaCCnqf2lj8CM3x0LE%3D&reserved=0 > >>> > >>> > >>>> On Apr 12, 2020, at 7:45 PM, Alex Harui <aha...@adobe.com.INVALID> > wrote: > >>>> > >>>> I'm not very experienced with spinning up servers. The CI server we > are using is effectively free, based on a generous donation from Microsoft > of MSDN accounts to ASF committers. So I leave it up 24/7, and share the > RDP access on private@. I think any other ASF committer could do the > same. IIRC, if that server actually is stopped, I have to use my personal > (unshared) MSDN credentials to start it again. AIUI, if I actually paid > for the server, it would cost me to leave it running even if it didn't run > jobs between releases. > >>>> > >>>> Is that what you are basically saying? I think it might be best if > another committer got a CI server going via the MS donation and could leave > it up 24/7. > >>>> > >>>> -Alex > >>>> > >>>> On 4/12/20, 9:28 AM, "Harbs" <harbs.li...@gmail.com> wrote: > >>>> > >>>> I’m willing to do this. > >>>> > >>>> Considering that the release will be run infrequently, it should be > doable to have a relatively powerful server that could be spun up on > demand. This is something I have setup for my own releases. > >>>> > >>>> The only complication would be that each RM would need valid > credentials to spin up the server. > >>>> > >>>> Harbs > >>>> > >>>>> On Apr 12, 2020, at 7:10 PM, Alex Harui <aha...@adobe.com.INVALID> > wrote: > >>>>> > >>>>> A better solution, IMO, is for someone else to offer up a CI server > only for release jobs. > >
