[OpenStack-Infra] Work toward a translations checksite and call for help

[Elizabeth K. Joseph]

Hi everyone,

I brought this up a few meetings ago, but I wanted to collect the
thoughts in one place to more easily get infra team input on the
status of work toward a translations checksite for the i18n team. As
some history, the i18n team wrote a specification a while back which
we approved, which folks can read for background:
http://specs.openstack.org/openstack-infra/infra-specs/specs/translation_check_site.html

The original assignees were mostly i18n people, and have been pulled
off to other things. As one of the primary infra liaisons with the
i18n team I've been pulled into helping, but my ability to help is
limited due to time and need for collaboration with some other infra
folks on some decisions. So here I am emailing the rest of the team
for help. Plus we also wanted to bring the conversations happening
privately about roadblocks to happen publicly so I don't continue to
be a blocker here.

Over the past several months Frank Kloeker worked to write a
preliminary Puppet module for us in puppet-translation_checksite (now
merged) and he has an outstanding corresponding system-config patch:
https://review.openstack.org/#/c/276466/

As the spec outlines, the assumption was that we'd run this on a
long-lived server in some way, updating the translation strings
directly from Zanata daily, and re-installing DevStack once a week.
We've run into a few issues with this, which I'd appreciate some
thoughts about so I have some help evaluating how to move forward.

1. The Puppet module is really fragile. In theory it works, Frank did
a good job with it. But almost every time I run it I run into another
problem. Sometimes it has to do with a DevStack error (there was a
known problem a couple times when I tried to run it), or trouble with
my environment (DevStack doesn't fail gracefully if a dependency is
not satisfied due to network timeout or whatnot) and sometimes it's
just a change in our infra that breaks things (yesterday it was an
unexpected problem with the puppet apt module).

The module itself doesn't yet have any recovery for any of this. If we
had DevStack running along well for a week, and it gets to the next
week and it fails to build, we're stuck with a broken system and no
notification that it's broken. We could spend time building fault
tolerance and build failure alerts into it, but I want to make sure
we're on the right track first.

2. We don't actually have a solution to run "new" DevStack once a
week. Some options:

 - The once a week rebuild is just known downtime for the checksite,
have a cron job to ./unstack and delete /opt/devstack?
 - Get to a place we're we're auto-building new servers, and just
build a new one and swap DNS once a week once we know the new server
also is running properly with something like a health script that must
pass
 - Something else?

3. It takes a long time to run DevStack's stack.sh, which this module
does. Current timeout is 3600 (1 hour), but I have to bump it up to
run it locally in my tests. Even at an hour, this will really gum up
the works if it's part of system-config and running alongside all our
other ansible+puppet runs, even if the building of DevStack is only
once a week. Is this acceptable to us?

4. While we will have i18n team members logging into the Horizon
interface to see the progress of their translations work (that's the
whole point), the translations checksite is essentially read-only and
we have a pretty good mechanism in place for spinning up daily
DevStack instances for all our tests. Maybe we should back-peddle and
somehow leverage this tooling instead?

Thanks everyone.

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] JJB V2.0 planning

[Paul Belanger]

On Mon, Jul 04, 2016 at 05:43:14PM +0100, Darragh Bailey wrote:
> Hi,
> 
> 
> To try and minimise trashing of both core reviews and V2.0 patch author(s),
> I'd like to propose that we pick a time/day every second week for 3-4
> iterations where those interested set aside a set block of time to
> collaborate in getting the main rework patches landed. Consider it a set of
> mini bug days focused on JJB 2.0 API changes.
> 
> To get the ball rolling, I'm going to suggest one of the following 2
> timezones (obviously these suit me best, but I'm available the other days
> as well):
> 14:00-1800 UTC Thurs (Staring 7th July - not available the 14th hence
> suggesting this Thurs)
> 14:00-1800 UTC Tues (Staring 12th July)
> 
> I'm assuming that later in the day for me aligns better with others, but I
> could be very wrong.
> 
> Also thinking that spinning up a temporary public dedicated IRC chat room
> would be helpful here, probably look to avoid using one of the existing
> meeting rooms because I'm assuming that would conflict with other teams,
> unless someone tells me there is a simple solution to this. Only negative I
> can see is that the chats wouldn't be logged.
> 
You may want to check if the #openstack-sprint channel is available on freenode.
We have logging enabled on it and seems like a natural fit for your agenda.

> 
> 
> More info below on why suggest this:
> 
> 
> Having gone through a few cycles where patches get reviewed, reworked and
> then broken by other changes landing, reworked again, reviewed and broken
> again, it can be quite onerous on both author and reviewer getting a change
> that touches a number of places to land as the risk of another patch
> landing causing a merge failure increases dramatically the more places the
> patch touches.
> 
> The set of V2 patches has to bring the existing code through some amount of
> interim steps to make it easy to review, unfortunately given the amount of
> rework to do, the odds of anything else triggering a conflict is pretty
> high and basically faced with the following choices:
> 
>- Take a long time complete the cycle of rework -> review -> rework ->
>break -> rework ->. ...
>- Block landing any changes that touch any of the code impacted by V2
>work until most V2 patches are landed.
> 
> 
> 
> However we can get enough cores on around the same time and try for some
> synchronized collaboration, I think it's probably far easier to land a
> series of patches over a few meetings and get everything far enough along
> with much less workload placed on everyone involved that we can then revert
> back to the more async approach without the same issues around the
> remaining changes.
> 
> Expect that this would only take 3-4 of these to get the major part of the
> rewrite in place.
> 
> Thoughts? Does this work for enough other JJB reviewers?
> 
> 
> -- 
> Darragh Bailey
> "Nothing is foolproof to a sufficiently talented fool"

> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] JJB V2.0 planning

[Kien Ha]

I am fine with starting this Friday. I am also okay with starting any time
afterwards too.

On Wed, Jul 6, 2016 at 3:14 PM, Darragh Bailey 
wrote:

>
> Sounds like Friday is the clear winner. Do we want to stay this Friday? Or
> would everyone prefer a bit more notice to plan things with their other
> work?
>
> I'm not around Friday 15th, but it could start then anyway if others are
> happy (might need one more core to help land stuff that day)? Otherwise
> could wait until the 22nd, naturally can still do normal reviews in the
> mean time.
>
> --
> Darragh Bailey
> "Nothing is foolproof to a sufficiently talented fool" - unknown
> On 6 Jul 2016 08:31, "Dong Ma"  wrote:
>
>> Hey Darragh,
>>
>> For Mon/Fri, the current time slots both works for me.
>>
>> Thanks,
>> Dong
>>
>>
>>
 On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma 
 wrote:

> Hey Darragh,
>
>
>
> I agree with your thought and would like to participate to the
> reviews, although the time slots is a little late for me but it works.
>
>
>

>>> Would moving the time slot an hour earlier on either Mon/Fri suit you
>>> better?
>>>
>>> --
>>> Darragh Bailey
>>> "Nothing is foolproof to a sufficiently talented fool"
>>>
>>
>>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] JJB V2.0 planning

[Darragh Bailey]

Sounds like Friday is the clear winner. Do we want to stay this Friday? Or
would everyone prefer a bit more notice to plan things with their other
work?

I'm not around Friday 15th, but it could start then anyway if others are
happy (might need one more core to help land stuff that day)? Otherwise
could wait until the 22nd, naturally can still do normal reviews in the
mean time.

--
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool" - unknown
On 6 Jul 2016 08:31, "Dong Ma"  wrote:

> Hey Darragh,
>
> For Mon/Fri, the current time slots both works for me.
>
> Thanks,
> Dong
>
>
>
>>> On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma  wrote:
>>>
 Hey Darragh,



 I agree with your thought and would like to participate to the reviews,
 although the time slots is a little late for me but it works.



>>>
>> Would moving the time slot an hour earlier on either Mon/Fri suit you
>> better?
>>
>> --
>> Darragh Bailey
>> "Nothing is foolproof to a sufficiently talented fool"
>>
>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] JJB V2.0 planning

[Dong Ma]

Hey Darragh,

For Mon/Fri, the current time slots both works for me.

Thanks,
Dong



>> On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma  wrote:
>>
>>> Hey Darragh,
>>>
>>>
>>>
>>> I agree with your thought and would like to participate to the reviews,
>>> although the time slots is a little late for me but it works.
>>>
>>>
>>>
>>
> Would moving the time slot an hour earlier on either Mon/Fri suit you
> better?
>
> --
> Darragh Bailey
> "Nothing is foolproof to a sufficiently talented fool"
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Jenkins job builder status report

[Dong Ma]

Hi Kien,

What I thought is the changes made to convert to xml function have been
updated to a lot of plugins, also some plugins have not been updated. So if
we have a spec to keep why and how to use the convert to xml function and
which plugins have been updated, which not, it will be great. But it's not
must needed, just my opinion.

Thanks,
Dong

2016-07-06 14:20 GMT+08:00 Kien Ha :

> Hi Dong,
>
> Looks like I misinterpreted what you mean the first time around, sorry
> about that. To clarify, did you mean to keep track of changes made to
> convert to xml function or to keep track of the plugins that have been
> updated to use the function? If it is wanted, I can go ahead and start
> adding it to the commit messages.
>
> Thanks,
> Kien
>
> On Sun, Jul 3, 2016 at 5:13 AM, Dong Ma  wrote:
>
>> Hey Darragh and Kien,
>>
>> What I proposed is to create a etherpad/spec/blueprint to cover these
>> changes of using the convert to xml function in JJB, and include it in the
>> commit message as reference to keep tracking the process, so if someone
>> else would like to contribute, it will be easy to find it.
>>
>> Yes, include the document link into the commit message is not a good
>> commit message, I agree to keep the commits describing the functional
>> reason.
>>
>> Thanks,
>> Vincent
>>
>> 2016-07-01 22:25 GMT+08:00 Darragh Bailey :
>>
>>> Hi Kien,
>>>
>>>
>>> I missed this email, but spotted this appearing in recent commits.
>>>
>>>
>>> On 13 June 2016 at 03:28, Kien Ha  wrote:
>>>

 On Sat, Jun 11, 2016 at 11:12 PM, Dong Ma 
 wrote:

> Hello Kien Ha,
>
> Thanks for the contribution to the Jenkins job builder projects, have
> one comment here, how about add your proposal document link or create a 
> new
> etherpad into the commit message of each patches as reference to keep
> tracking the process.
>
> Vincent
>

>>> This kind goes outside of what is generally described as
>>> https://wiki.openstack.org/wiki/GitCommitMessages#Information_in_commit_messages
>>>
>>>
>>> The update to the mailing list is more than enough, let's keep the
>>> commits describing the functional reason for the change, or if there is a
>>> separate proposal spec/blueprint that covers changing to use the convert to
>>> xml function in JJB, changes relevant can include a reference, otherwise
>>> lets keep the commit messages focused on the change itself.
>>>
>>>
>>> --
>>> Darragh Bailey
>>> "Nothing is foolproof to a sufficiently talented fool"
>>>
>>
>>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Jenkins job builder status report

[Kien Ha]

Hi Dong,

Looks like I misinterpreted what you mean the first time around, sorry
about that. To clarify, did you mean to keep track of changes made to
convert to xml function or to keep track of the plugins that have been
updated to use the function? If it is wanted, I can go ahead and start
adding it to the commit messages.

Thanks,
Kien

On Sun, Jul 3, 2016 at 5:13 AM, Dong Ma  wrote:

> Hey Darragh and Kien,
>
> What I proposed is to create a etherpad/spec/blueprint to cover these
> changes of using the convert to xml function in JJB, and include it in the
> commit message as reference to keep tracking the process, so if someone
> else would like to contribute, it will be easy to find it.
>
> Yes, include the document link into the commit message is not a good
> commit message, I agree to keep the commits describing the functional
> reason.
>
> Thanks,
> Vincent
>
> 2016-07-01 22:25 GMT+08:00 Darragh Bailey :
>
>> Hi Kien,
>>
>>
>> I missed this email, but spotted this appearing in recent commits.
>>
>>
>> On 13 June 2016 at 03:28, Kien Ha  wrote:
>>
>>>
>>> On Sat, Jun 11, 2016 at 11:12 PM, Dong Ma 
>>> wrote:
>>>
 Hello Kien Ha,

 Thanks for the contribution to the Jenkins job builder projects, have
 one comment here, how about add your proposal document link or create a new
 etherpad into the commit message of each patches as reference to keep
 tracking the process.

 Vincent

>>>
>> This kind goes outside of what is generally described as
>> https://wiki.openstack.org/wiki/GitCommitMessages#Information_in_commit_messages
>>
>>
>> The update to the mailing list is more than enough, let's keep the
>> commits describing the functional reason for the change, or if there is a
>> separate proposal spec/blueprint that covers changing to use the convert to
>> xml function in JJB, changes relevant can include a reference, otherwise
>> lets keep the commit messages focused on the change itself.
>>
>>
>> --
>> Darragh Bailey
>> "Nothing is foolproof to a sufficiently talented fool"
>>
>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra