Re: Recommended strategy for running 1.5 in production

[Thomas Heil]

Hi,

On 18.04.2014 08:24, Willy Tarreau wrote:
> On Thu, Apr 17, 2014 at 10:30:45AM -0500, Ryan O'Hara wrote:
>> On Wed, Apr 16, 2014 at 11:12:07PM +0100, Kobus Bensch wrote:
>>> I use haproxy on centos. So I build a RPM i then use in spacewalk to
>>> first roll out to test, then post testing to production.
>> I can add el6 to my copr build if you need an rpm build. Currently I'm
>> only building 1.5-dev22 in copr for F20, but it should be too much
>> trouble add el6.
> Just to save you one package, I managed to fix the chunked-encoded
> compression so I'm planning on issuing dev23 this week-end :-)
Having compression also for chunked-encoded would help a lot. This is a
long awaited feature.
Iam curious about it.

> Willy
>
>
>
cheers,
thomas

Re: Recommended strategy for running 1.5 in production

[Willy Tarreau]

On Thu, Apr 17, 2014 at 10:30:45AM -0500, Ryan O'Hara wrote:
> On Wed, Apr 16, 2014 at 11:12:07PM +0100, Kobus Bensch wrote:
> > I use haproxy on centos. So I build a RPM i then use in spacewalk to
> > first roll out to test, then post testing to production.
> 
> I can add el6 to my copr build if you need an rpm build. Currently I'm
> only building 1.5-dev22 in copr for F20, but it should be too much
> trouble add el6.

Just to save you one package, I managed to fix the chunked-encoded
compression so I'm planning on issuing dev23 this week-end :-)

Willy

Re: Recommended strategy for running 1.5 in production

[Ryan O'Hara]

On Wed, Apr 16, 2014 at 11:12:07PM +0100, Kobus Bensch wrote:
> I use haproxy on centos. So I build a RPM i then use in spacewalk to
> first roll out to test, then post testing to production.

I can add el6 to my copr build if you need an rpm build. Currently I'm
only building 1.5-dev22 in copr for F20, but it should be too much
trouble add el6.

Ryan

> On 16/04/2014 17:14, pablo platt wrote:
> >An official Ubuntu dev repo will also make testing easier.
> >It's much easier to use a apt-get than building from source and
> >figuring out command line options.
> >
> >
> >On Wed, Apr 16, 2014 at 7:05 PM, Philipp
> > >> wrote:
> >
> >Am 16.04.2014 17:40 schrieb Willy Tarreau:
> >
> >I think you summarized very well how to carefully use a
> >development
> >version in prod. That requires a bit of care, but with that
> >you can
> >get both nice features and quick fixes.
> >
> >
> >Indeed :)
> >
> >After 1.5 is released, I'd like to switch to a faster and more
> >regular
> >release cycle with less constraints on the features.
> >
> >
> >And with above said: I, personally, give a rats a** if a version
> >is called
> >alpha, rc123, -dev or whatever fancy version string it has.
> >
> >Test the thing and find out the hairy bits after it hits
> >production :-)
> >
> >I was sooo often burned by "oh, finally release" and then it was worse
> >then the "RC" before the actual release whatsoever.
> >
> >My kudos to Willy and the other developers of haproxy, awesome work
> >overall AND in the nitbits :-).
> >
> >
> 
> -- 
> Kobus Bensch Trustpay Global LTD email signature Kobus Bensch
> Senior Systems Administrator
> Address:  22 & 24 | Frederick Sanger Road | Guildford | Surrey | GU2 7YD
> DDI:  0207 871 3958
> Tel:  0207 871 3890
> Email: kobus.ben...@trustpayglobal.com
> 
> 
> -- 
> 
> 
> Trustpay Global Limited is an authorised Electronic Money
> Institution regulated by the Financial Conduct Authority
> registration number 900043. Company No 07427913 Registered in
> England and Wales with registered address 130 Wood Street, London,
> EC2V 6DL, United Kingdom.
> 
> For further details please visit our website at www.trustpayglobal.com.
> 
> The information in this email and any attachments are confidential
> and remain the property of Trustpay Global Ltd unless agreed by
> contract. It is intended solely for the person to whom or the entity
> to which it is addressed. If you are not the intended recipient you
> may not use, disclose, copy, distribute, print or rely on the
> content of this email or its attachments. If this email has been
> received by you in error please advise the sender and delete the
> email from your system. Trustpay Global Ltd does not accept any
> liability for any personal view expressed in this message.

Re: Recommended strategy for running 1.5 in production

[Kobus Bensch]

Hi List

I use haproxy on centos. I regularly build a RPM I then use in spacewalk 
to first roll out to test, then post testing to production.


This process works great as if we need to roll back we can do that very 
easily.


Kobus

On 17/04/2014 08:10, Vincent Bernat wrote:

  ❦ 16 avril 2014 21:07 CEST, pablo platt  :


The Ubuntu PPA is great but it is not 'official' and I couldn't find
Ubuntu 14.04 package.
https://launchpad.net/~vbernat/+archive/haproxy-1.5

Ubuntu 14.04 LTS will be out tomorrow which means that haproxy-1.5
will be included only in the next LTS release 2 years from now.
That's why an official Ubuntu repo will be very useful.
Nginx and MongoDB for example has one.

What is an "official" repository? The packages from the PPA are clean
backports from "official" Debian (and hence Ubuntu) packages. We could
copy the packages in "haproxy.debian.net" to make them look more
official, but using a PPA is convenient for users to add (one command).

As for 14.04, I will push it the PPA shortly.



--


Trustpay Global Limited is an authorised Electronic Money Institution 
regulated by the Financial Conduct Authority registration number 900043. 
Company No 07427913 Registered in England and Wales with registered address 
130 Wood Street, London, EC2V 6DL, United Kingdom.


For further details please visit our website at www.trustpayglobal.com.

The information in this email and any attachments are confidential and 
remain the property of Trustpay Global Ltd unless agreed by contract. It is 
intended solely for the person to whom or the entity to which it is 
addressed. If you are not the intended recipient you may not use, disclose, 
copy, distribute, print or rely on the content of this email or its 
attachments. If this email has been received by you in error please advise 
the sender and delete the email from your system. Trustpay Global Ltd does 
not accept any liability for any personal view expressed in this message.

Re: Recommended strategy for running 1.5 in production

[Vincent Bernat]

 ❦ 16 avril 2014 21:07 CEST, pablo platt  :

> The Ubuntu PPA is great but it is not 'official' and I couldn't find
> Ubuntu 14.04 package.
> https://launchpad.net/~vbernat/+archive/haproxy-1.5
>
> Ubuntu 14.04 LTS will be out tomorrow which means that haproxy-1.5
> will be included only in the next LTS release 2 years from now.
> That's why an official Ubuntu repo will be very useful.
> Nginx and MongoDB for example has one.

What is an "official" repository? The packages from the PPA are clean
backports from "official" Debian (and hence Ubuntu) packages. We could
copy the packages in "haproxy.debian.net" to make them look more
official, but using a PPA is convenient for users to add (one command).

As for 14.04, I will push it the PPA shortly.
-- 
Don't sacrifice clarity for small gains in "efficiency".
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: Recommended strategy for running 1.5 in production

[Kobus Bensch]

I use haproxy on centos. So I build a RPM i then use in spacewalk to 
first roll out to test, then post testing to production.

On 16/04/2014 17:14, pablo platt wrote:

An official Ubuntu dev repo will also make testing easier.
It's much easier to use a apt-get than building from source and 
figuring out command line options.



On Wed, Apr 16, 2014 at 7:05 PM, Philipp 
> wrote:


Am 16.04.2014 17:40 schrieb Willy Tarreau:

I think you summarized very well how to carefully use a
development
version in prod. That requires a bit of care, but with that
you can
get both nice features and quick fixes.


Indeed :)

After 1.5 is released, I'd like to switch to a faster and more
regular
release cycle with less constraints on the features.


And with above said: I, personally, give a rats a** if a version
is called
alpha, rc123, -dev or whatever fancy version string it has.

Test the thing and find out the hairy bits after it hits
production :-)

I was sooo often burned by "oh, finally release" and then it was worse
then the "RC" before the actual release whatsoever.

My kudos to Willy and the other developers of haproxy, awesome work
overall AND in the nitbits :-).




--
Kobus Bensch Trustpay Global LTD email signature Kobus Bensch
Senior Systems Administrator
Address:  22 & 24 | Frederick Sanger Road | Guildford | Surrey | GU2 7YD
DDI:  0207 871 3958
Tel:  0207 871 3890
Email: kobus.ben...@trustpayglobal.com 



--


Trustpay Global Limited is an authorised Electronic Money Institution 
regulated by the Financial Conduct Authority registration number 900043. 
Company No 07427913 Registered in England and Wales with registered address 
130 Wood Street, London, EC2V 6DL, United Kingdom.


For further details please visit our website at www.trustpayglobal.com.

The information in this email and any attachments are confidential and 
remain the property of Trustpay Global Ltd unless agreed by contract. It is 
intended solely for the person to whom or the entity to which it is 
addressed. If you are not the intended recipient you may not use, disclose, 
copy, distribute, print or rely on the content of this email or its 
attachments. If this email has been received by you in error please advise 
the sender and delete the email from your system. Trustpay Global Ltd does 
not accept any liability for any personal view expressed in this message.
<>

Re: Recommended strategy for running 1.5 in production

[Ramin K]

Nothing stopping you from using the PPA as originally suggested. :-)

	In regards to easy there are hundreds of how-to webpages for building 
deb packages. I'd grade it slightly more complicated than building RPMs 
correctly. When you're done you can use the same infrastructure to build 
other packages instead of waiting for them to appear in the next LTS. 
Also haproxy is likely to have at least another -dev release if not 
more. May as well do it right so it is easy to repackage as needed 
later. And you'll get to add something that will be forever useful to 
your sysadmin skill set.


Ramin

On 4/16/2014 1:25 PM, pablo platt wrote:

So I "only" need to setup a VM, download a file from a personal PPA, add
the source code and learn how to build a deb package...
And every user need to do the same...
Easy :)


On Wed, Apr 16, 2014 at 11:12 PM, Ramin K mailto:ramin-l...@badapple.net>> wrote:

 It's easy to build your own packages with the files from
the PPA which is what we do. BTW, thanks Vincent for doing the
groundwork.

Simplest method might be:
-  Decide which archs you need to target. Follow a how-to for
setting up a environment to build deb packages. I recommend
dedicating a VM for each arch though we only build for x86_64.
- Use the debian.tar.gz file from the PPA to populate the ./debian/ dir
- make a orig.tar.gz from dev22 or preferred revision
- probably take you 2-3 tries to get everything in the right place
- make the deb packages
- copy deb packages to your internal repo.

Ramin


On 4/16/2014 12:07 PM, pablo platt wrote:

The Ubuntu PPA is great but it is not 'official' and I couldn't find
Ubuntu 14.04 package.
https://launchpad.net/~__vbernat/+archive/haproxy-1.5

__>


Ubuntu 14.04 LTS will be out tomorrow which means that
haproxy-1.5 will
be included only in the next LTS release 2 years from now.
That's why an official Ubuntu repo will be very useful.
Nginx and MongoDB for example has one.

Is there a script that we can use to build a deb package?


On Wed, Apr 16, 2014 at 9:55 PM, Willy Tarreau mailto:w...@1wt.eu>
>> wrote:

 Hi Apollon,

 On Wed, Apr 16, 2014 at 09:22:56PM +0300, Apollon
Oikonomopoulos wrote:
  > (Cc'ing the Debian maintainers as well)
  >
  > Hi all,
  >
  > On 19:28 Wed 16 Apr , Willy Tarreau wrote:
  > > On Wed, Apr 16, 2014 at 07:14:31PM +0300, pablo platt
wrote:
  > > > An official Ubuntu dev repo will also make testing
easier.
  > > > It's much easier to use a apt-get than building from
source
 and figuring
  > > > out command line options.
  >
  > Actually Vincent Bernat maintains a PPA with rebuilds of
our Debian
  > packages from experimental, which should be handy for
Ubuntu users:
  >
  > https://launchpad.net/~__vbernat/+archive/haproxy-1.5

  >
  > >
  > > I think we're getting close to a release so we should not
 harrass distro
  > > maintainers with that *now* (but we could have done
years ago).
 That
  > > reminds me that I tend not to always realize how much time
 slips between
  > > versions, and to forget that sometimes a previous
version has some
  > > bugs.
  > >
  > > What I'd expect from our users is to sometimes
complain loudly
 and insist
  > > for having a new dev release when the latest snapshot has
 become more
  > > reliable than the last dev release if that makes their
lifes
 easier. A
  > > new version doesn't cost much (1 hour to read the
changelog,
 write a
  > > human-friendly summary in an announce e-mail and
update the site).
  >
  > With my Debian hat on, I'd like to "complain" a bit
about 1.5. Of
 course
  > we appreciate your dedication to making HAProxy
rock-solid and
  > feature-complete and at this point as a user 1.5 has
been pretty
 stable
  > for me (and the new features are definitely worth the wait).
  >
  > However, as Debian maintainers we probably will not
replace 1.4
 with 1.5
  > in o

Re: Recommended strategy for running 1.5 in production

[pablo platt]

So I "only" need to setup a VM, download a file from a personal PPA, add
the source code and learn how to build a deb package...
And every user need to do the same...
Easy :)


On Wed, Apr 16, 2014 at 11:12 PM, Ramin K  wrote:

> It's easy to build your own packages with the files from the PPA
> which is what we do. BTW, thanks Vincent for doing the groundwork.
>
> Simplest method might be:
> -  Decide which archs you need to target. Follow a how-to for setting up a
> environment to build deb packages. I recommend dedicating a VM for each
> arch though we only build for x86_64.
> - Use the debian.tar.gz file from the PPA to populate the ./debian/ dir
> - make a orig.tar.gz from dev22 or preferred revision
> - probably take you 2-3 tries to get everything in the right place
> - make the deb packages
> - copy deb packages to your internal repo.
>
> Ramin
>
>
> On 4/16/2014 12:07 PM, pablo platt wrote:
>
>> The Ubuntu PPA is great but it is not 'official' and I couldn't find
>> Ubuntu 14.04 package.
>> https://launchpad.net/~vbernat/+archive/haproxy-1.5
>> 
>>
>>
>> Ubuntu 14.04 LTS will be out tomorrow which means that haproxy-1.5 will
>> be included only in the next LTS release 2 years from now.
>> That's why an official Ubuntu repo will be very useful.
>> Nginx and MongoDB for example has one.
>>
>> Is there a script that we can use to build a deb package?
>>
>>
>> On Wed, Apr 16, 2014 at 9:55 PM, Willy Tarreau > > wrote:
>>
>> Hi Apollon,
>>
>> On Wed, Apr 16, 2014 at 09:22:56PM +0300, Apollon Oikonomopoulos
>> wrote:
>>  > (Cc'ing the Debian maintainers as well)
>>  >
>>  > Hi all,
>>  >
>>  > On 19:28 Wed 16 Apr , Willy Tarreau wrote:
>>  > > On Wed, Apr 16, 2014 at 07:14:31PM +0300, pablo platt wrote:
>>  > > > An official Ubuntu dev repo will also make testing easier.
>>  > > > It's much easier to use a apt-get than building from source
>> and figuring
>>  > > > out command line options.
>>  >
>>  > Actually Vincent Bernat maintains a PPA with rebuilds of our Debian
>>  > packages from experimental, which should be handy for Ubuntu users:
>>  >
>>  > https://launchpad.net/~vbernat/+archive/haproxy-1.5
>>  >
>>  > >
>>  > > I think we're getting close to a release so we should not
>> harrass distro
>>  > > maintainers with that *now* (but we could have done years ago).
>> That
>>  > > reminds me that I tend not to always realize how much time
>> slips between
>>  > > versions, and to forget that sometimes a previous version has
>> some
>>  > > bugs.
>>  > >
>>  > > What I'd expect from our users is to sometimes complain loudly
>> and insist
>>  > > for having a new dev release when the latest snapshot has
>> become more
>>  > > reliable than the last dev release if that makes their lifes
>> easier. A
>>  > > new version doesn't cost much (1 hour to read the changelog,
>> write a
>>  > > human-friendly summary in an announce e-mail and update the
>> site).
>>  >
>>  > With my Debian hat on, I'd like to "complain" a bit about 1.5. Of
>> course
>>  > we appreciate your dedication to making HAProxy rock-solid and
>>  > feature-complete and at this point as a user 1.5 has been pretty
>> stable
>>  > for me (and the new features are definitely worth the wait).
>>  >
>>  > However, as Debian maintainers we probably will not replace 1.4
>> with 1.5
>>  > in our main track (unstable -> testing -> wheezy-backports) until
>>  > 1.5-final is out; we would like to make sure that we will end up
>> with a
>>  > proper 1.5 release in Debian Jessie (and not with a development
>> snapshot
>>  > at any rate) that both, upstream and ourselves will be willing to
>>  > support.
>>  >
>>  > Unfortunately, this means that 1.5 currently gets less user
>> exposure (at
>>  > least via Debian and Ubuntu), potentially slowing down the
>> stabilization
>>  > process. So please, leave some features aside for 1.6 ;-)
>>
>> I know and the goal clearly is not to add new features to 1.5, but
>> to fix
>> what still remains to be fixed before the release otherwise we'd have
>> to
>> risk breaking some supposed stable setups later when backporting
>> fixes :
>>
>>- fix the HTTP body parser to get rid of the mess it is when mixing
>>  redispatch with check_post, not to mention compression.
>>
>>- fix the compression to re-enable compression of chunked-encoded
>>  responses
>>
>>- adapt the check agent to the final API we agreed on the list a
>> few
>>  weeks ago
>>
>>- fix the bind-process lameness.
>>
>> I'm still working on point #1 and making progress (I was even
>> writing some
>> architecture doc on it to engrave the changes so tha

Re: Recommended strategy for running 1.5 in production

[Ramin K]

	It's easy to build your own packages with the files from the PPA which 
is what we do. BTW, thanks Vincent for doing the groundwork.


Simplest method might be:
-  Decide which archs you need to target. Follow a how-to for setting up 
a environment to build deb packages. I recommend dedicating a VM for 
each arch though we only build for x86_64.

- Use the debian.tar.gz file from the PPA to populate the ./debian/ dir
- make a orig.tar.gz from dev22 or preferred revision
- probably take you 2-3 tries to get everything in the right place
- make the deb packages
- copy deb packages to your internal repo.

Ramin

On 4/16/2014 12:07 PM, pablo platt wrote:

The Ubuntu PPA is great but it is not 'official' and I couldn't find
Ubuntu 14.04 package.
https://launchpad.net/~vbernat/+archive/haproxy-1.5


Ubuntu 14.04 LTS will be out tomorrow which means that haproxy-1.5 will
be included only in the next LTS release 2 years from now.
That's why an official Ubuntu repo will be very useful.
Nginx and MongoDB for example has one.

Is there a script that we can use to build a deb package?


On Wed, Apr 16, 2014 at 9:55 PM, Willy Tarreau mailto:w...@1wt.eu>> wrote:

Hi Apollon,

On Wed, Apr 16, 2014 at 09:22:56PM +0300, Apollon Oikonomopoulos wrote:
 > (Cc'ing the Debian maintainers as well)
 >
 > Hi all,
 >
 > On 19:28 Wed 16 Apr , Willy Tarreau wrote:
 > > On Wed, Apr 16, 2014 at 07:14:31PM +0300, pablo platt wrote:
 > > > An official Ubuntu dev repo will also make testing easier.
 > > > It's much easier to use a apt-get than building from source
and figuring
 > > > out command line options.
 >
 > Actually Vincent Bernat maintains a PPA with rebuilds of our Debian
 > packages from experimental, which should be handy for Ubuntu users:
 >
 > https://launchpad.net/~vbernat/+archive/haproxy-1.5
 >
 > >
 > > I think we're getting close to a release so we should not
harrass distro
 > > maintainers with that *now* (but we could have done years ago).
That
 > > reminds me that I tend not to always realize how much time
slips between
 > > versions, and to forget that sometimes a previous version has some
 > > bugs.
 > >
 > > What I'd expect from our users is to sometimes complain loudly
and insist
 > > for having a new dev release when the latest snapshot has
become more
 > > reliable than the last dev release if that makes their lifes
easier. A
 > > new version doesn't cost much (1 hour to read the changelog,
write a
 > > human-friendly summary in an announce e-mail and update the site).
 >
 > With my Debian hat on, I'd like to "complain" a bit about 1.5. Of
course
 > we appreciate your dedication to making HAProxy rock-solid and
 > feature-complete and at this point as a user 1.5 has been pretty
stable
 > for me (and the new features are definitely worth the wait).
 >
 > However, as Debian maintainers we probably will not replace 1.4
with 1.5
 > in our main track (unstable -> testing -> wheezy-backports) until
 > 1.5-final is out; we would like to make sure that we will end up
with a
 > proper 1.5 release in Debian Jessie (and not with a development
snapshot
 > at any rate) that both, upstream and ourselves will be willing to
 > support.
 >
 > Unfortunately, this means that 1.5 currently gets less user
exposure (at
 > least via Debian and Ubuntu), potentially slowing down the
stabilization
 > process. So please, leave some features aside for 1.6 ;-)

I know and the goal clearly is not to add new features to 1.5, but
to fix
what still remains to be fixed before the release otherwise we'd have to
risk breaking some supposed stable setups later when backporting fixes :

   - fix the HTTP body parser to get rid of the mess it is when mixing
 redispatch with check_post, not to mention compression.

   - fix the compression to re-enable compression of chunked-encoded
 responses

   - adapt the check agent to the final API we agreed on the list a few
 weeks ago

   - fix the bind-process lameness.

I'm still working on point #1 and making progress (I was even
writing some
architecture doc on it to engrave the changes so that we avoid breaking
that soon again). #2 should follow shortly after that. #3 is apparently
easy (I had a beginning of patch 2 months ago to start on it) but we
noticed
that the check agent touches many intimate parts of the checks and I
expect
a few surprizes again. However, I don't care much about minor bugs
for the
final release provided that the architecture is ready to accept fixes
without putting users at risk. For #4, I think we can keep the users
in a
safe working area to prepare them for upgrades by simp

Re: Recommended strategy for running 1.5 in production

[pablo platt]

The Ubuntu PPA is great but it is not 'official' and I couldn't find Ubuntu
14.04 package.
https://launchpad.net/~vbernat/+archive/haproxy-1.5

Ubuntu 14.04 LTS will be out tomorrow which means that haproxy-1.5 will be
included only in the next LTS release 2 years from now.
That's why an official Ubuntu repo will be very useful.
Nginx and MongoDB for example has one.

Is there a script that we can use to build a deb package?


On Wed, Apr 16, 2014 at 9:55 PM, Willy Tarreau  wrote:

> Hi Apollon,
>
> On Wed, Apr 16, 2014 at 09:22:56PM +0300, Apollon Oikonomopoulos wrote:
> > (Cc'ing the Debian maintainers as well)
> >
> > Hi all,
> >
> > On 19:28 Wed 16 Apr , Willy Tarreau wrote:
> > > On Wed, Apr 16, 2014 at 07:14:31PM +0300, pablo platt wrote:
> > > > An official Ubuntu dev repo will also make testing easier.
> > > > It's much easier to use a apt-get than building from source and
> figuring
> > > > out command line options.
> >
> > Actually Vincent Bernat maintains a PPA with rebuilds of our Debian
> > packages from experimental, which should be handy for Ubuntu users:
> >
> >   https://launchpad.net/~vbernat/+archive/haproxy-1.5
> >
> > >
> > > I think we're getting close to a release so we should not harrass
> distro
> > > maintainers with that *now* (but we could have done years ago). That
> > > reminds me that I tend not to always realize how much time slips
> between
> > > versions, and to forget that sometimes a previous version has some
> > > bugs.
> > >
> > > What I'd expect from our users is to sometimes complain loudly and
> insist
> > > for having a new dev release when the latest snapshot has become more
> > > reliable than the last dev release if that makes their lifes easier. A
> > > new version doesn't cost much (1 hour to read the changelog, write a
> > > human-friendly summary in an announce e-mail and update the site).
> >
> > With my Debian hat on, I'd like to "complain" a bit about 1.5. Of course
> > we appreciate your dedication to making HAProxy rock-solid and
> > feature-complete and at this point as a user 1.5 has been pretty stable
> > for me (and the new features are definitely worth the wait).
> >
> > However, as Debian maintainers we probably will not replace 1.4 with 1.5
> > in our main track (unstable -> testing -> wheezy-backports) until
> > 1.5-final is out; we would like to make sure that we will end up with a
> > proper 1.5 release in Debian Jessie (and not with a development snapshot
> > at any rate) that both, upstream and ourselves will be willing to
> > support.
> >
> > Unfortunately, this means that 1.5 currently gets less user exposure (at
> > least via Debian and Ubuntu), potentially slowing down the stabilization
> > process. So please, leave some features aside for 1.6 ;-)
>
> I know and the goal clearly is not to add new features to 1.5, but to fix
> what still remains to be fixed before the release otherwise we'd have to
> risk breaking some supposed stable setups later when backporting fixes :
>
>   - fix the HTTP body parser to get rid of the mess it is when mixing
> redispatch with check_post, not to mention compression.
>
>   - fix the compression to re-enable compression of chunked-encoded
> responses
>
>   - adapt the check agent to the final API we agreed on the list a few
> weeks ago
>
>   - fix the bind-process lameness.
>
> I'm still working on point #1 and making progress (I was even writing some
> architecture doc on it to engrave the changes so that we avoid breaking
> that soon again). #2 should follow shortly after that. #3 is apparently
> easy (I had a beginning of patch 2 months ago to start on it) but we
> noticed
> that the check agent touches many intimate parts of the checks and I expect
> a few surprizes again. However, I don't care much about minor bugs for the
> final release provided that the architecture is ready to accept fixes
> without putting users at risk. For #4, I think we can keep the users in a
> safe working area to prepare them for upgrades by simply emitting a few
> warnings in the configs leading to a corner case.
>
> I really think we're on the right track, we must just not stop efforts.
> And the fact that we get lots of bug reports is a good sign as well!
>
> Cheers,
> Willy
>
>

Re: Recommended strategy for running 1.5 in production

[Willy Tarreau]

Hi Apollon,

On Wed, Apr 16, 2014 at 09:22:56PM +0300, Apollon Oikonomopoulos wrote:
> (Cc'ing the Debian maintainers as well)
> 
> Hi all,
> 
> On 19:28 Wed 16 Apr , Willy Tarreau wrote:
> > On Wed, Apr 16, 2014 at 07:14:31PM +0300, pablo platt wrote:
> > > An official Ubuntu dev repo will also make testing easier.
> > > It's much easier to use a apt-get than building from source and figuring
> > > out command line options.
> 
> Actually Vincent Bernat maintains a PPA with rebuilds of our Debian 
> packages from experimental, which should be handy for Ubuntu users:
> 
>   https://launchpad.net/~vbernat/+archive/haproxy-1.5
> 
> > 
> > I think we're getting close to a release so we should not harrass distro
> > maintainers with that *now* (but we could have done years ago). That
> > reminds me that I tend not to always realize how much time slips between
> > versions, and to forget that sometimes a previous version has some 
> > bugs.
> > 
> > What I'd expect from our users is to sometimes complain loudly and insist
> > for having a new dev release when the latest snapshot has become more
> > reliable than the last dev release if that makes their lifes easier. A
> > new version doesn't cost much (1 hour to read the changelog, write a
> > human-friendly summary in an announce e-mail and update the site).
> 
> With my Debian hat on, I'd like to "complain" a bit about 1.5. Of course 
> we appreciate your dedication to making HAProxy rock-solid and 
> feature-complete and at this point as a user 1.5 has been pretty stable 
> for me (and the new features are definitely worth the wait).
> 
> However, as Debian maintainers we probably will not replace 1.4 with 1.5 
> in our main track (unstable -> testing -> wheezy-backports) until 
> 1.5-final is out; we would like to make sure that we will end up with a 
> proper 1.5 release in Debian Jessie (and not with a development snapshot 
> at any rate) that both, upstream and ourselves will be willing to 
> support.
> 
> Unfortunately, this means that 1.5 currently gets less user exposure (at 
> least via Debian and Ubuntu), potentially slowing down the stabilization 
> process. So please, leave some features aside for 1.6 ;-)

I know and the goal clearly is not to add new features to 1.5, but to fix
what still remains to be fixed before the release otherwise we'd have to
risk breaking some supposed stable setups later when backporting fixes :

  - fix the HTTP body parser to get rid of the mess it is when mixing
redispatch with check_post, not to mention compression.

  - fix the compression to re-enable compression of chunked-encoded
responses

  - adapt the check agent to the final API we agreed on the list a few
weeks ago

  - fix the bind-process lameness.

I'm still working on point #1 and making progress (I was even writing some
architecture doc on it to engrave the changes so that we avoid breaking
that soon again). #2 should follow shortly after that. #3 is apparently
easy (I had a beginning of patch 2 months ago to start on it) but we noticed
that the check agent touches many intimate parts of the checks and I expect
a few surprizes again. However, I don't care much about minor bugs for the
final release provided that the architecture is ready to accept fixes
without putting users at risk. For #4, I think we can keep the users in a
safe working area to prepare them for upgrades by simply emitting a few
warnings in the configs leading to a corner case.

I really think we're on the right track, we must just not stop efforts.
And the fact that we get lots of bug reports is a good sign as well!

Cheers,
Willy

Re: Recommended strategy for running 1.5 in production

[Apollon Oikonomopoulos]

(Cc'ing the Debian maintainers as well)

Hi all,

On 19:28 Wed 16 Apr , Willy Tarreau wrote:
> On Wed, Apr 16, 2014 at 07:14:31PM +0300, pablo platt wrote:
> > An official Ubuntu dev repo will also make testing easier.
> > It's much easier to use a apt-get than building from source and figuring
> > out command line options.

Actually Vincent Bernat maintains a PPA with rebuilds of our Debian 
packages from experimental, which should be handy for Ubuntu users:

  https://launchpad.net/~vbernat/+archive/haproxy-1.5

> 
> I think we're getting close to a release so we should not harrass distro
> maintainers with that *now* (but we could have done years ago). That
> reminds me that I tend not to always realize how much time slips between
> versions, and to forget that sometimes a previous version has some 
> bugs.
> 
> What I'd expect from our users is to sometimes complain loudly and insist
> for having a new dev release when the latest snapshot has become more
> reliable than the last dev release if that makes their lifes easier. A
> new version doesn't cost much (1 hour to read the changelog, write a
> human-friendly summary in an announce e-mail and update the site).

With my Debian hat on, I'd like to "complain" a bit about 1.5. Of course 
we appreciate your dedication to making HAProxy rock-solid and 
feature-complete and at this point as a user 1.5 has been pretty stable 
for me (and the new features are definitely worth the wait).

However, as Debian maintainers we probably will not replace 1.4 with 1.5 
in our main track (unstable -> testing -> wheezy-backports) until 
1.5-final is out; we would like to make sure that we will end up with a 
proper 1.5 release in Debian Jessie (and not with a development snapshot 
at any rate) that both, upstream and ourselves will be willing to 
support.

Unfortunately, this means that 1.5 currently gets less user exposure (at 
least via Debian and Ubuntu), potentially slowing down the stabilization 
process. So please, leave some features aside for 1.6 ;-)

Regards,
Apollon

Re: Recommended strategy for running 1.5 in production

[Willy Tarreau]

On Wed, Apr 16, 2014 at 07:14:31PM +0300, pablo platt wrote:
> An official Ubuntu dev repo will also make testing easier.
> It's much easier to use a apt-get than building from source and figuring
> out command line options.

I think we're getting close to a release so we should not harrass distro
maintainers with that *now* (but we could have done years ago). That
reminds me that I tend not to always realize how much time slips between
versions, and to forget that sometimes a previous version has some bugs.

What I'd expect from our users is to sometimes complain loudly and insist
for having a new dev release when the latest snapshot has become more
reliable than the last dev release if that makes their lifes easier. A
new version doesn't cost much (1 hour to read the changelog, write a
human-friendly summary in an announce e-mail and update the site).

Of course you probably all realize that I like to mix bug fixes and new
features in order to get some feedback on new features, but that's not
mean this is systematic either!

Cheers,
Willy

Re: Recommended strategy for running 1.5 in production

[pablo platt]

An official Ubuntu dev repo will also make testing easier.
It's much easier to use a apt-get than building from source and figuring
out command line options.


On Wed, Apr 16, 2014 at 7:05 PM, Philipp <
e1c1bac6253dc54a1e89ddc046585...@posteo.net> wrote:

> Am 16.04.2014 17:40 schrieb Willy Tarreau:
>
>> I think you summarized very well how to carefully use a development
>> version in prod. That requires a bit of care, but with that you can
>> get both nice features and quick fixes.
>>
>
> Indeed :)
>
>  After 1.5 is released, I'd like to switch to a faster and more regular
>> release cycle with less constraints on the features.
>>
>
> And with above said: I, personally, give a rats a** if a version is called
> alpha, rc123, -dev or whatever fancy version string it has.
>
> Test the thing and find out the hairy bits after it hits production :-)
>
> I was sooo often burned by "oh, finally release" and then it was worse
> then the "RC" before the actual release whatsoever.
>
> My kudos to Willy and the other developers of haproxy, awesome work
> overall AND in the nitbits :-).
>
>

Re: Recommended strategy for running 1.5 in production

[Philipp]

Am 16.04.2014 17:40 schrieb Willy Tarreau:

I think you summarized very well how to carefully use a development
version in prod. That requires a bit of care, but with that you can
get both nice features and quick fixes.


Indeed :)


After 1.5 is released, I'd like to switch to a faster and more regular
release cycle with less constraints on the features.


And with above said: I, personally, give a rats a** if a version is 
called

alpha, rc123, -dev or whatever fancy version string it has.

Test the thing and find out the hairy bits after it hits production :-)

I was sooo often burned by "oh, finally release" and then it was worse
then the "RC" before the actual release whatsoever.

My kudos to Willy and the other developers of haproxy, awesome work
overall AND in the nitbits :-).

Re: Recommended strategy for running 1.5 in production

[Willy Tarreau]

Hi Apollon,

On Wed, Apr 16, 2014 at 11:33:37AM +0300, Apollon Oikonomopoulos wrote:
(...)
> We run a -dev version - not necessarily the latest one, just one that is 
> running stable and has no security issues. In general I follow the list 
> on a daily basis (not reading through every mail though) and I always 
> keep a clone of the git repository around and periodically check the 
> output of
> 
>   git log --grep BUG/ v1.5-dev${my_release}..
> 
> for anything serious (especially BUG/MAJOR commits). If anything too 
> serious arises, we are willing to cherry-pick changes on top of our 
> production version (which is always deployed using debian packages). The 
> bugfix commits usually refer to either the commit, or the version that 
> introduced the bug, so it is easy to determine whether we are affected 
> by the bug or not.

I think you summarized very well how to carefully use a development
version in prod. That requires a bit of care, but with that you can
get both nice features and quick fixes. Someone recently told me that
1.5 had less bugs than 1.4 because I release 1.4 less often... I must
admit that sometimes it's true.

After 1.5 is released, I'd like to switch to a faster and more regular
release cycle with less constraints on the features. A bit like Linux
after all. We'll work with a new feature window and a freeze period,
with pre-defined deadlines. That will certainly improve the rhythm and
remove some frustration that may come from using devel in prod.

Cheers,
Willy

Re: Recommended strategy for running 1.5 in production

[Apollon Oikonomopoulos]

Hi Jonathan,

On 22:27 Mon 14 Apr , Jonathan Matthews wrote:
> Hi all -
> 
> I've been running 1.4 for a number of years, but am pondering moving
> some as-yet-unreleased apps to 1.5, for SSL and ACL-ish reasons.
> 
> I'd like to ask how you, 1.5 sysadmins and devs, track the development
> version, and how you decide which version to run in production.

We switched from 1.4 to 1.5 last September, after running 1.4 for 2+ 
years. The switch was made primarily due to SSL and proper IPv6 support; 
we had been using nginx for SSL termination previously, which ran fine, 
but made our setup a bit more complex than we'd like. We've been very 
satisfied with 1.5, both in terms of performance and features.

The features we've been looking at more recently are stick-tables and 
counters, and we're planning to use them primarily for abuse detection 
and throttling.

> Do you just run 1.5-dev${LATEST}? The latest snapshot? Do you follow
> the list here and cherry-pick important bug fixes?
>
> I don't feel I have a firm understanding of the status of the
> different, co-existing codebases that one could call "1.5" at any
> given time. And nor do I have the C-skills and time to review every
> commit.
> 
> What do /you/ do, fellow sysadmins? How do you run, upgrade and
> maintain confidence in your chosen version of 1.5 in production?

We run a -dev version - not necessarily the latest one, just one that is 
running stable and has no security issues. In general I follow the list 
on a daily basis (not reading through every mail though) and I always 
keep a clone of the git repository around and periodically check the 
output of

  git log --grep BUG/ v1.5-dev${my_release}..

for anything serious (especially BUG/MAJOR commits). If anything too 
serious arises, we are willing to cherry-pick changes on top of our 
production version (which is always deployed using debian packages). The 
bugfix commits usually refer to either the commit, or the version that 
introduced the bug, so it is easy to determine whether we are affected 
by the bug or not.

We do upgrades in a three-step process, upgrading the backup node of our 
active-backup setup, then failing over traffic to it while maintaining 
the old active as a backup with the previous version. After everything 
runs smoothly for a week or so, we upgrade the other node as well.

Personally I consider 1.5 pretty stable at this point and most bugs I've 
seen (even the important ones) are either corner-cases or bugs in new 
features that won't break existing configurations. The only exception 
was -dev20, which introduced lots of changes and new functionality and 
it was probably expected that there would be some breakage. Thus said, I 
prefer to run "milestone" releases rather than daily snapshots or latest 
git.

Regards,
Apollon

Re: Recommended strategy for running 1.5 in production

[Remy van Elst]

Andy Walker schreef op 15/04/14 01:19:
That's pretty close to our method as well. We've been using haproxy 
for just over a year, and started with 1.5 because of the built-in SSL 
capabilities... specifically, version 1.5-dev17. We actually did a 
fairly minimal amount of testing, and just started by moving a low 
amount of production traffic over to it. Since then, we've slowly 
ramped up to our current 120-140 million hits per day.


We've done 2 upgrades -- 1.5-dev19 and 1.5-dev22. In both instances, 
we just did a bit of testing in a dev environment, and then upgraded 
the passive nodes in our active/passive haproxy pairs, let them chug 
along for a couple of days days, and then upgraded the others. We 
haven't had any issues with any regressions, and there haven't been 
any real backwards compatibility issues. The only time we needed to 
change our configs during an upgrade was with the latest health code 
check changes, but everything worked out just fine!


In short, we've been much more happy with haproxy -- both during 
upgrades and as a whole -- than we have with all of the closed-source 
enterprisey software we run.


Thanks, Willy!!


--
Andy Walker
System Administrator
FBS - creators of flexmls
3415 39th St S
Fargo, ND  58104
701-235-7300


On Mon, Apr 14, 2014 at 5:27 PM, Ramin K > wrote:


On 4/14/2014 2:27 PM, Jonathan Matthews wrote:

Hi all -

I've been running 1.4 for a number of years, but am pondering
moving
some as-yet-unreleased apps to 1.5, for SSL and ACL-ish reasons.

I'd like to ask how you, 1.5 sysadmins and devs, track the
development
version, and how you decide which version to run in production.

Do you just run 1.5-dev${LATEST}? The latest snapshot? Do you
follow
the list here and cherry-pick important bug fixes?

I don't feel I have a firm understanding of the status of the
different, co-existing codebases that one could call "1.5" at any
given time. And nor do I have the C-skills and time to review
every
commit.

What do /you/ do, fellow sysadmins? How do you run, upgrade and
maintain confidence in your chosen version of 1.5 in production?

All opinions and information welcome!
Jonathan


Our method was to pick an actual release (-dev19), put it on
stage, and after a week move it to prod. Update as new releases
come out, but wait a day or two for any major bug (dev20 was a
short lived release) to shake out before it goes on stage. We've
been running 1.5-devxx in prod for over six months.

Our system is relatively small trafficwise and ssl termination was
the only driver of the move to 1.5. However we are looking at
making use of the new health check code. ymmv.

Ramin


About the same here. SSL termination and ACL's were the reason to go to 
1.5 in production. First in the testing environment, then after a month 
or so without issues to acceptance, then after a month to production. We 
do this with every release, however, only if it has been out for at 
least a month (As in, see 1.5-dev20).


We have not had many issues yet, as in, only on our side were the 
applications did not follow protocols correctly and haproxy did.


For the rest, it is an improvement over our F5. Configuration can be 
done with Puppet way more easily, the code is out there and it feels 
more stable.




smime.p7s
Description: S/MIME-cryptografische ondertekening

Re: Recommended strategy for running 1.5 in production

[Lukas Tribus]

Hi,


key is certainly to do some fair amount of testing, don't use a release
with a lot of very recent activity, but let it "cool down" a few days,
or better, 1 - 3 weeks (and then check fixed bugs 'git log --oneline \
v1.5-devXY..master | grep BUG' to see if anything is relevant for you).

You also probably don't want to upgrade to every new -dev release, as
validating them costs a certain amount of time and there is a always
a risk involved, even though you do some testing with it.

At the same time you should not stick to a certain release for an infinite
amount of time, but restart this process periodically. Otherwise you don't
benefit from the fixes.



Regards,

Lukas

  

Fwd: Re: Recommended strategy for running 1.5 in production

[Philipp]

Missed the reply-to :)

 Originalnachricht 
Thanks for the data point, Philipp. If you resend your reply to the
list, that might be useful for people other than just me :-)

J

On 15 April 2014 09:26, Philipp
 wrote:

Am 14.04.2014 23:27 schrieb Jonathan Matthews:


What do /you/ do, fellow sysadmins? How do you run, upgrade and
maintain confidence in your chosen version of 1.5 in production?



We run 1.5-dev19 (2013/06/17) in production. The last restart was 
28days ago
and the two peers cluster delivered around 100GB of traffic with some 
25

million http/200
in this period (as of hatop).

I had no visible bug since installing this version and thus I am not
upgrading :-).

(on a sidenote, no openssl 1.0.1 so not bleeding ;-) )

Re: Recommended strategy for running 1.5 in production

[Andy Walker]

That's pretty close to our method as well. We've been using haproxy for
just over a year, and started with 1.5 because of the built-in SSL
capabilities... specifically, version 1.5-dev17. We actually did a fairly
minimal amount of testing, and just started by moving a low amount of
production traffic over to it. Since then, we've slowly ramped up to our
current 120-140 million hits per day.

We've done 2 upgrades -- 1.5-dev19 and 1.5-dev22. In both instances, we
just did a bit of testing in a dev environment, and then upgraded the
passive nodes in our active/passive haproxy pairs, let them chug along for
a couple of days days, and then upgraded the others. We haven't had any
issues with any regressions, and there haven't been any real backwards
compatibility issues. The only time we needed to change our configs during
an upgrade was with the latest health code check changes, but everything
worked out just fine!

In short, we've been much more happy with haproxy -- both during upgrades
and as a whole -- than we have with all of the closed-source enterprisey
software we run.

Thanks, Willy!!


--
Andy Walker
System Administrator
FBS - creators of flexmls
3415 39th St S
Fargo, ND  58104
701-235-7300


On Mon, Apr 14, 2014 at 5:27 PM, Ramin K  wrote:

> On 4/14/2014 2:27 PM, Jonathan Matthews wrote:
>
>> Hi all -
>>
>> I've been running 1.4 for a number of years, but am pondering moving
>> some as-yet-unreleased apps to 1.5, for SSL and ACL-ish reasons.
>>
>> I'd like to ask how you, 1.5 sysadmins and devs, track the development
>> version, and how you decide which version to run in production.
>>
>> Do you just run 1.5-dev${LATEST}? The latest snapshot? Do you follow
>> the list here and cherry-pick important bug fixes?
>>
>> I don't feel I have a firm understanding of the status of the
>> different, co-existing codebases that one could call "1.5" at any
>> given time. And nor do I have the C-skills and time to review every
>> commit.
>>
>> What do /you/ do, fellow sysadmins? How do you run, upgrade and
>> maintain confidence in your chosen version of 1.5 in production?
>>
>> All opinions and information welcome!
>> Jonathan
>>
>>
> Our method was to pick an actual release (-dev19), put it on stage, and
> after a week move it to prod. Update as new releases come out, but wait a
> day or two for any major bug (dev20 was a short lived release) to shake out
> before it goes on stage. We've been running 1.5-devxx in prod for over six
> months.
>
> Our system is relatively small trafficwise and ssl termination was the
> only driver of the move to 1.5. However we are looking at making use of the
> new health check code. ymmv.
>
> Ramin
>
>

Re: Recommended strategy for running 1.5 in production

[Ramin K]

On 4/14/2014 2:27 PM, Jonathan Matthews wrote:

Hi all -

I've been running 1.4 for a number of years, but am pondering moving
some as-yet-unreleased apps to 1.5, for SSL and ACL-ish reasons.

I'd like to ask how you, 1.5 sysadmins and devs, track the development
version, and how you decide which version to run in production.

Do you just run 1.5-dev${LATEST}? The latest snapshot? Do you follow
the list here and cherry-pick important bug fixes?

I don't feel I have a firm understanding of the status of the
different, co-existing codebases that one could call "1.5" at any
given time. And nor do I have the C-skills and time to review every
commit.

What do /you/ do, fellow sysadmins? How do you run, upgrade and
maintain confidence in your chosen version of 1.5 in production?

All opinions and information welcome!
Jonathan



Our method was to pick an actual release (-dev19), put it on stage, and 
after a week move it to prod. Update as new releases come out, but wait 
a day or two for any major bug (dev20 was a short lived release) to 
shake out before it goes on stage. We've been running 1.5-devxx in prod 
for over six months.


Our system is relatively small trafficwise and ssl termination was the 
only driver of the move to 1.5. However we are looking at making use of 
the new health check code. ymmv.


Ramin

Recommended strategy for running 1.5 in production

[Jonathan Matthews]

Hi all -

I've been running 1.4 for a number of years, but am pondering moving
some as-yet-unreleased apps to 1.5, for SSL and ACL-ish reasons.

I'd like to ask how you, 1.5 sysadmins and devs, track the development
version, and how you decide which version to run in production.

Do you just run 1.5-dev${LATEST}? The latest snapshot? Do you follow
the list here and cherry-pick important bug fixes?

I don't feel I have a firm understanding of the status of the
different, co-existing codebases that one could call "1.5" at any
given time. And nor do I have the C-skills and time to review every
commit.

What do /you/ do, fellow sysadmins? How do you run, upgrade and
maintain confidence in your chosen version of 1.5 in production?

All opinions and information welcome!
Jonathan