Re: Trying to build the EL6 packages, getting the EL7 ones

[Khosrow Moossavi]

Hi Vladimir

You need to build the RPMs *on* CentOS 6 machines to be able to have el6 in
the name.
Or as an alternative you can use
https://khos2ow.github.io/cloudstack-rpm-builder/ which contains all the
required tools to build RPMs.




On Thu, Apr 25, 2019 at 7:59 AM Andrija Panic 
wrote:

> Correct - yes.
>
> Best,
>
> On Thu, 25 Apr 2019 at 13:45, Vladimir Melnik  wrote:
>
> > Thank you Andrija! I've pulled the image, started /bin/bash in this
> > contaner, is it an environment to build ACS for EL6, right?
> >
> > On Wed, Apr 24, 2019 at 05:31:21PM +, Andrija Panic wrote:
> > > Hi Vladimir,
> > >
> > > You have to use CentOS 6 host or do some magic with Docker on whatever
> > host: https://hub.docker.com/r/bhaisaab/centos6-cloudstack-slave
> > >
> > > Best,
> > > Andrija
> > >
> > >
> > > andrija.pa...@shapeblue.com
> > > www.shapeblue.com
> > > Amadeus House, Floral Street, London  WC2E 9DPUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Vladimir Melnik 
> > > Sent: 24 April 2019 08:34
> > > To: dev@cloudstack.apache.org
> > > Subject: Trying to build the EL6 packages, getting the EL7 ones
> > >
> > > Hello,
> > >
> > > I asked this question in users@, but didn't get any ideas, so I'll be
> > hoping that dev@ could help.
> > >
> > > I'm trying to build the CentOS6-compatible packages (using this
> > document:
> >
> http://docs.cloudstack.apache.org/en/latest/installguide/building_from_source.html#building-rpms-from-source
> ),
> > running `./package.sh -d centos63`, but the files I get have the "el7"
> > suffix in their names:
> > >   cloudstack-agent-4.11.2.0-1.el7.centos.x86_64.rpm
> > >   cloudstack-baremetal-agent-4.11.2.0-1.el7.centos.x86_64.rpm
> > >   cloudstack-cli-4.11.2.0-1.el7.centos.x86_64.rpm
> > >   cloudstack-common-4.11.2.0-1.el7.centos.x86_64.rpm
> > >   cloudstack-integration-tests-4.11.2.0-1.el7.centos.x86_64.rpm
> > >   cloudstack-management-4.11.2.0-1.el7.centos.x86_64.rpm
> > >   cloudstack-marvin-4.11.2.0-1.el7.centos.x86_64.rpm
> > >   cloudstack-usage-4.11.2.0-1.el7.centos.x86_64.rpm
> > >
> > > Am I doing it wrong?
> > >
> > > Thanks in advance for the clue!
> > >
> > > --
> > > V.Melnik
> >
> > --
> > V.Melnik
> >
>
>
> --
>
> Andrija Panić
>

Re: New VP of CloudStack: Paul Angus

[Khosrow Moossavi]

Congratulations Paul!




On Tue, Mar 12, 2019 at 9:29 AM Boris Varbanov  wrote:

> Congratulations Paul!
> Excellent news! Definitely well deserved!
>
> Best,
> Boris Varbanov
>
> On Mon, Mar 11, 2019 at 11:16 AM Tutkowski, Mike <
> mike.tutkow...@netapp.com>
> wrote:
>
> > Hi everyone,
> >
> > As you may know, the role of VP of CloudStack (Chair of the CloudStack
> > PMC) has a one-year term. My term has now come and gone.
> >
> > I’m happy to announce that the CloudStack PMC has elected Paul Angus as
> > our new VP of CloudStack.
> >
> > As many already know, Paul has been an active member of the CloudStack
> > Community for over six years now. I’ve worked with Paul on and off
> > throughout much of that time and I believe he’ll be a great fit for this
> > role.
> >
> > Please join me in welcoming Paul as the new VP of Apache CloudStack!
> >
> > Thanks,
> > Mike
> >
>

Re: Build CloudStack 4.10.0 Error

[Khosrow Moossavi]

As Rafael mentioned this was a known issue and fixed (the commit he sent).

You can check the version of java running in the build tools like:

docker run -it --entrypoint environment-info.sh
khos2ow/cloudstack-deb-builder:ubuntu1404

java version: openjdk version "1.8.0_171" OpenJDK Runtime Environment
(build 1.8.0_171-8u171-b11-2~14.04-b11) OpenJDK 64-Bit Server VM (build
25.171-b11, mixed mode)


On Tue, Nov 13, 2018 at 11:18 AM li jerry  wrote:

> Think Rafael Weingärtner
>
>
>
> I have already passed the build after the change. Is this because the
> openssl version of my build machine is too low?
>
>
>
> Root@59e9a8871fe8:/mnt/build# openssl version -a
>
> OpenSSL 1.0.1f 6 Jan 2014
>
> Built on: Wed Apr 18 18:30:39 UTC 2018
>
> Platform: debian-amd64
>
> Options: bn(64,64) rc4(16x,int) des(idx,cisc,16,int) blowfish(idx)
>
> Compiler: cc -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT
> -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -g -O2
> -fstack-protector --param=ssp-buffer-size=4 -Wformat
> -Werror=format-security -D_FORTIFY_SOURCE = 2 -Wl, -Bsymbolic-functions
> -Wl, -z, relro -Wa, - noexecstack -Wall -DMD32_REG_T = int
> -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5
> -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM
> -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
>
> OPENSSLDIR: "/usr/lib/ssl"
>
>
>
> 
> 发件人: Rafael Weingärtner 
> 发送时间: Tuesday, November 13, 2018 11:44:37 PM
> 收件人: dev
> 抄送: users
> 主题: Re: Build CloudStack 4.10.0 Error
>
> This is a known problem, and it is fixed with:
> https://github.com/apache/cloudstack/pull/2674
>
> On Tue, Nov 13, 2018 at 1:42 PM Rohit Yadav 
> wrote:
>
> > Can you try with 4.11 branch or latest master? 4.10 is not a maintained
> > branch and likely have env caused errors due to supported ciphers and tls
> > versions in jre/jdk.
> >
> > Regards.
> >
> >
> > Regards,
> > Rohit Yadav
> >
> > 
> > From: li jerry 
> > Sent: Tuesday, November 13, 2018 9:04:34 PM
> > To: us...@cloudstack.apache.org; dev@cloudstack.apache.org
> > Subject: Build CloudStack 4.10.0 Error
> >
> > Hi All
> >
> >I made a mistake in compiling cloudstack 4.10.0 through docker
> > image (khos2ow/cloudstack-deb-builder: 14.04).
> >
> > Please help me. Thank you!
> >
> > ---
> > T E S T S
> > ---
> > Running streamer.BaseElementTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.103 sec
> > - in streamer.BaseElementTest
> > Running streamer.ByteBufferTest
> > Tests run: 400, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.091
> > sec - in streamer.ByteBufferTest
> > Running rdpclient.MockServerTest
> > Error in mock server: Received fatal alert: handshake_failure
> > javax.net.ssl.SSLHandshakeException: Received fatal alert:
> > handshake_failure
> >at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> >at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
> >at
> sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2038)
> >at
> > sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1135)
> >at
> >
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
> >at
> > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
> >at
> > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
> >at streamer.debug.MockServer.run(MockServer.java:122)
> >at java.lang.Thread.run(Thread.java:748)
> > Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.186 sec
> > <<< FAILURE! - in rdpclient.MockServerTest
> > testIsMockServerCanUpgradeConnectionToSsl(rdpclient.MockServerTest)  Time
> > elapsed: 0.18 sec  <<< ERROR!
> > javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is
> > disabled or cipher suites are inappropriate)
> >at sun.security.ssl.Handshaker.activate(Handshaker.java:529)
> >at
> >
> sun.security.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.java:1492)
> >at
> >
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1361)
> >at
> > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
> >at
> > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
> >at
> >
> rdpclient.MockServerTest.testIsMockServerCanUpgradeConnectionToSsl(MockServerTest.java:166)
> >
> > Running common.ClientTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262 sec
> > - in common.ClientTest
> >
> > Results :
> >
> > Tests in error:
> >   MockServerTest.testIsMockServerCanUpgradeConnectionToSsl:166 ?
> > SSLHandshake No...
> >
> > Tests run: 404, Failures: 0, Errors: 1, Skipped: 0
> >
> >
> >
> > 

Wrong tags pushed to Github

[Khosrow Moossavi]

Hi Dev

Today I noticed bunch of tags have been pushed upstream which I believe
this wasn't intended.

 * [new tag]   4.11.2-snap-> 4.11.2-snap
 * [new tag]   4.6.2-rc1  -> 4.6.2-rc1
 * [new tag]   4.7.0-rc1  -> 4.7.0-rc1
 * [new tag]   4.9.1.0-rc1-> 4.9.1.0-rc1
 * [new tag]   shapeblue-4.3.2-01 -> shapeblue-4.3.2-01
 * [new tag]   shapeblue-4.4.4-00 -> shapeblue-4.4.4-00
 * [new tag]   shapeblue-4.5.1-00 -> shapeblue-4.5.1-00
 * [new tag]   shapeblue-4.5.2-00 -> shapeblue-4.5.2-00
 * [new tag]   shapeblue-4.6.0-00 -> shapeblue-4.6.0-00
 * [new tag]   shapeblue-4.6.1-00 -> shapeblue-4.6.1-00

Cheers

Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>

Removing IAM services

[Khosrow Moossavi]

Hello Community

Following up after PR#2613[1] to cleanup POMs across the whole repository,
now in a new
PR #2817 we are going to remove services/iam projects which seems to be
disabled and
not maintained since May 2014.

Moving forward with the PR, corresponding tables will be deleted from
database too.

Please let us know if you have any questions, comments, concerns about this
removal or if
you are willing to revive the project in some shape or form.

[1]: https://github.com/apache/cloudstack/pull/2613
[2]: https://github.com/apache/cloudstack/pull/2817

Khosrow Moossavi

Cloud Infrastructure Developer

<https://goo.gl/NYZ8KK>

Re: Github Issues

[Khosrow Moossavi]

I sort of agree with Marc-Aurèle and Will, and like github issues way
better than Jira.

It definitely is easier that both the issues and fix for those issues live
in the same place
and easily can be referenced from one another. The only thing is that we
need to come
up with good set of labels (for both issues and PRs) for tracking purpose.

Discussing the issue at hand under the issue itself can be even good, it
will leave a
trail of what has been discussed around the issue which led to the fix and
potentially
discussion can be continued under PR itself. Essentially they are targeting
the same
"problem".

As for the point Ron brought up, if one issue was that big that required
multiple PR for
it to be fixed, it only makes sens to me to create subset of issues all
referencing the
"parent" issue, and each individual PR fixes one of those smaller issue.

Khosrow


On Tue, Jul 17, 2018 at 11:08 AM Will Stevens  wrote:

> Ron, keep in mind that PRs on Github are different from Issues.  They are
> two different features.
>
> There will be a much cleaner, tighter integration between issues and the
> solution when everything is on Github.
>
> will
>
> On Tue, Jul 17, 2018, 9:33 AM Ron Wheeler 
> wrote:
>
> > I may have voiced my concerns earlier but as a user, I think that JIRA
> > issues are easier to follow than PRs.
> > - As Paul said an issue may affect more than one version.
> > - It may also require more than one PR to fully resolve the issue.
> > - Issues tend to be described in terms of a problem that the user would
> > recognize while PRs are most often described as what was done to fix the
> > problem. The JIRA could be much easier to relate to what the user is
> > seeing and more likely to show up in Google.
> >
> > Ron
> > On 17/07/2018 4:53 AM, Paul Angus wrote:
> > > Hi All,
> > >
> > > We have been trialling replacing Jira with Github Issues.   I think
> that
> > we should have a conversation about it before it become the new standard
> by
> > default.
> > >
> > >  From my perspective, I don't like it.  Searching has become far more
> > difficult, categorising has also. When there is a bug fix it can only be
> > targeted for a single version, which makes them easy to lose track of,
> and
> > when looking at milestones issues and PRs get jumbled up and people are
> > commenting on issues when it should by the PR and vice-versa (yes I've
> done
> > it too).
> > > In summary, from an administrative point of view it causes a lot more
> > problems than it solves.
> > >
> > > I yield the floor to hear other people's opinions...
> > >
> > >
> > > Kind regards,
> > >
> > > Paul Angus
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> >
> > --
> > Ron Wheeler
> > President
> > Artifact Software Inc
> > email: rwhee...@artifact-software.com
> > skype: ronaldmwheeler
> > phone: 866-970-2435, ext 102
> >
> >
>

Re: Can't build master

[Khosrow Moossavi]

Mike

I have no idea about your issue on OS X but if you only need and care to
build the artifacts,
you can use docker based cloudstack-deb-builder[1] or if you only want to
run maven phases
(test, verify, etc) you can override docker image entrypoint, such as:

docker run  -it ... --entrypoint "/usr/bin/mvn"
khos2ow/cloudstack-deb-builder 

[1]: https://khos2ow.github.io/cloudstack-deb-builder/




On Tue, May 22, 2018 at 2:58 PM Tutkowski, Mike 
wrote:

> Hi Rohit,
>
> I’ve tried a few things so far, but none seem to install genisoimage in
> /usr/bin as the test indicates is required.
>
> From
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+Up+a+CloudStack+Development+Environment+on+Mac+OS+X,
> I’ve tried these steps:
>
> • sudo port install cdrtools; or using brew: brew install cdrtools (could
> take a long time)
> 'brew install cdrtools' did not work for me on OSX 10.9.  However, 'brew
> install dvdrtools' did work for me...
> • NOTE - If after the above steps, for any reason, mkisofs is still not
> installed, download it from the net. One good link to get mkisofs for mac
> is - http://www.helios.de/viewart.html?id=1000-en#download . Follow the
> instructions in the section "Download HELIOS “mkisofs” tested binary
> versions". Use the macosx86 binary if you're running mac os x on an intel
> platform. After downloading the mkisofs binary, copy it over to
> /usr/local/bin/.
>
> I only use Mac OS X to build the code locally. I don’t actually run the
> management server from this machine (I run it on Ubuntu).
>
> For the time being at least, I can just use –DskipTests=true when building
> on Mac OS X.
>
> Talk to you later,
> Mike
>
> On 5/22/18, 12:19 AM, "Rohit Yadav"  wrote:
>
> Hi Mike,
>
>
> Is genisoimage or mkisofs available on osx? This is usually installed
> at /usr/bin/ on CentOS6/CentOS7/Ubuntu Linux. Can you try brew or something
> else to install it?
>
> They are also used by injectkeys.sh/.py when the management server
> starts. The change is part of a recent PR I did and added a unit test for
> it where it tries to build a config drive ISO file. If genisoimage is not
> availabe on OSX, we can add some environment check to the unit test to skip
> on non-Linux environments.
>
>
> - Rohit
>
> 
>
>
>
> 
> From: Tutkowski, Mike 
> Sent: Tuesday, May 22, 2018 2:13:23 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Can't build master
>
> Just an FYI that this is on OS X Version 10.11.6.
>
> From: "Tutkowski, Mike" 
> Date: Monday, May 21, 2018 at 2:42 PM
> To: "dev@cloudstack.apache.org" 
> Subject: Can't build master
>
> Hi,
>
> Did I miss an e-mail or something? I’m having trouble building master
> (below).
>
> Thanks!
> Mike
>
> Running
> org.apache.cloudstack.storage.configdrive.ConfigDriveBuilderTest
> log4j:WARN No appenders could be found for logger
> (org.apache.cloudstack.storage.configdrive.ConfigDriveBuilder).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.296
> sec <<< FAILURE! - in
> org.apache.cloudstack.storage.configdrive.ConfigDriveBuilderTest
>
> testConfigDriveBuild(org.apache.cloudstack.storage.configdrive.ConfigDriveBuilderTest)
> Time elapsed: 0.278 sec  <<< ERROR!
> com.cloud.utils.exception.CloudRuntimeException: Unable to create iso
> file: i-x-y.iso due to java.io.IOException: Cannot run program
> "/usr/bin/genisoimage": error=2, No such file or directory
> at
> java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
> at
> com.cloud.utils.script.Script.execute(Script.java:215)
> at
> com.cloud.utils.script.Script.execute(Script.java:183)
> at
> org.apache.cloudstack.storage.configdrive.ConfigDriveBuilder.buildConfigDrive(ConfigDriveBuilder.java:152)
> at
> org.apache.cloudstack.storage.configdrive.ConfigDriveBuilderTest.testConfigDriveBuild(ConfigDriveBuilderTest.java:56)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> 

Re: John Kinsella and Wido den Hollander now ASF members

[Khosrow Moossavi]

That's awesome! Congratulations!




On Wed, May 2, 2018 at 12:19 PM Tutkowski, Mike 
wrote:

> Congratulations, guys! :-)
>
> > On May 2, 2018, at 9:58 AM, David Nalley  wrote:
> >
> > Hi folks,
> >
> > As noted in the press release[1] John Kinsella and Wido den Hollander
> > have been elected to the ASF's membership.
> >
> > Members are the 'shareholders' of the foundation, elect the board of
> > directors, and help guide the future of the ASF.
> >
> > Congrats to both of you, very well deserved.
> >
> > --David
> >
> > [1] https://s.apache.org/ysxx
>

Re: [DISCUSS] New VPN implementation based on IKEv2 backed by Vault

[Khosrow Moossavi]

Thanks Ilya for the feedback.

The way I currently implemented it, two items need to be set in global
settings beforehand:

- you need to specify the VPN implementation (either L2TP or IKEv2)
- then select the PKI engine backend (Vault or Default)

so there won't be any immediate and blocking coupling between
management-server and Vault.

But...

I would argue that we should leverage these new *tools* in our own infra
where Cloudstack runs, such as ELK, RabbitMQ, Kafka, Redis, MongoDB,
or Vault in this particular case. I would assume everyone of us does use
these tools one way or another to keep the infrastructure up an running.
Why not provide an easy, OOB way to do so from ACS itself?

On the other hand, I totally agree that ACS must not fully depend on these
so if any of these were not available ACS won't work. But at the end of the
day ACS is a *webapp* -which does something special of its own- and it
should get help from all the *cool kids*.

On code quality, test coverage and integrations tests I completely 100%
agree.





On Wed, Apr 4, 2018 at 4:53 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> To complement one thing that Ilya mentioned here. I do not worry much about
> the “requirement” for Vault systems to test ACS. This would be the case if
> Khosrow, when developing, only created tests using what the community calls
> integration tests.
>
> However, it is an implementation from scratch and as such it can and should
> use a very high bar for code quality, which enables proper unit testing for
> all methods. This means, that we can check all of the code in our domain
> (code base) without requiring third-party software. It does not mean that
> we do not need “integration tests” checking the system integration with
> Vault, but we could then restrict this execution to RCs.
>
> On Wed, Apr 4, 2018 at 5:29 PM, ilya musayev <ilya.mailing.li...@gmail.com
> >
> wrote:
>
> > Khosrow
> >
> > My 2c, little less than ideal to manage yet another external end point
> > like.
> >
> > While i understand that it makes it easier to manage certificates - it
> also
> > means going forward - Vault implementation will become a requirement to
> > validate future ACS release.
> >
> > With that said - i do like the proposal and not against it, but:
> > 1) Please consider decoupling it from cloudstack-management server - and
> > release it as server plugin
> > 2) Test coverage must be sufficient enough to validate the functionality
> > (perhaps mock vault endpoints and response)
> >
> > Regards,
> > ilya
> >
> > On Wed, Apr 4, 2018 at 10:49 AM, Khosrow Moossavi <
> kmooss...@cloudops.com>
> > wrote:
> >
> > > Thanks Paul, the proposed feature will enable the functionality to use
> > > Vault to
> > > act as CA if enabled in ACS, otherwise will fall back to "default"
> > > implementation
> > > which Rohit has already done.
> > >
> > >
> > > On Wed, Apr 4, 2018 at 12:29 PM, Paul Angus <paul.an...@shapeblue.com>
> > > wrote:
> > >
> > > > You guys should speak to Rohit about the CA framework.  CloudStack
> can
> > > > manage certificates now, including creating them itself and acting
> as a
> > > > root CA.
> > > >
> > > >
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Paul Angus
> > > >
> > > > paul.an...@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: Rafael Weingärtner <rafaelweingart...@gmail.com>
> > > > Sent: 04 April 2018 16:51
> > > > To: dev <dev@cloudstack.apache.org>
> > > > Subject: Re: [DISCUSS] New VPN implementation based on IKEv2 backed
> by
> > > > Vault
> > > >
> > > > Thanks for sharing the details. Now I have a better perspective of
> the
> > > > proposal.It is an interesting integration of CloudStack VPN service
> > with
> > > > Vault PKI feature.
> > > >
> > > > On Wed, Apr 4, 2018 at 12:38 PM, Khosrow Moossavi <
> > > kmooss...@cloudops.com>
> > > > wrote:
> > > >
> > > > > One of the things Vault does is essentially one of the thing Let's
> > > > > Encrypt does, acting as CA and generating/signing certificates.
> > > > >
> > > > > Fr

Re: [DISCUSS] New VPN implementation based on IKEv2 backed by Vault

[Khosrow Moossavi]

Thanks Paul, the proposed feature will enable the functionality to use
Vault to
act as CA if enabled in ACS, otherwise will fall back to "default"
implementation
which Rohit has already done.


On Wed, Apr 4, 2018 at 12:29 PM, Paul Angus <paul.an...@shapeblue.com>
wrote:

> You guys should speak to Rohit about the CA framework.  CloudStack can
> manage certificates now, including creating them itself and acting as a
> root CA.
>
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Rafael Weingärtner <rafaelweingart...@gmail.com>
> Sent: 04 April 2018 16:51
> To: dev <dev@cloudstack.apache.org>
> Subject: Re: [DISCUSS] New VPN implementation based on IKEv2 backed by
> Vault
>
> Thanks for sharing the details. Now I have a better perspective of the
> proposal.It is an interesting integration of CloudStack VPN service with
> Vault PKI feature.
>
> On Wed, Apr 4, 2018 at 12:38 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > One of the things Vault does is essentially one of the thing Let's
> > Encrypt does, acting as CA and generating/signing certificates.
> >
> > From the Vault website itself:
> >
> > "HashiCorp Vault secures, stores, and tightly controls access to
> > tokens, passwords, certificates, API keys, and other secrets in modern
> > computing. Vault handles leasing, key revocation, key rolling, and
> > auditing. Through a unified API, users can access an encrypted
> > Key/Value store and network encryption-as-a-service, or generate AWS
> > IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH
> > credentials, and more."
> >
> > In our case we are going to use Vault as PKI backend engine, to act as
> > Root CA, sign certificates, handle CRL (Certificate Revocation List),
> > etc.
> > Technically we can
> > do these with Let's Encrypt, but I haven't started exploring the
> > possibilities or potential limitation. Using external services (such
> > as Let's Encrypt) or going forward with Bring You Own Certificate
> > model would be for future, it they ever made sense to do.
> >
> >
> >
> > On Wed, Apr 4, 2018 at 11:20 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > Got it. Thanks for the explanations.
> > > There is one other thing I do not understand. This Vault thing that
> > > you mention, how does it work? Is it similar to let's encrypt?
> > >
> > > On Wed, Apr 4, 2018 at 12:15 PM, Khosrow Moossavi <
> > kmooss...@cloudops.com>
> > > wrote:
> > >
> > > > On Wed, Apr 4, 2018 at 10:36 AM, Rafael Weingärtner <
> > > > rafaelweingart...@gmail.com> wrote:
> > > >
> > > > > So, you need a certificate that is signed by the CA that is used
> > > > > by
> > the
> > > > VPN
> > > > > service. Is that it?
> > > > >
> > > > >
> > > > Correct, a self signed "server certificate" against CA, to be
> > > > installed directly on VR.
> > > >
> > > >
> > > > >
> > > > > It has been a while that I do not configure these VPN systems;
> > > > > do you
> > > > need
> > > > > access to the private key of the CA? Or, does the program simply
> > > validate
> > > > > the user (VPN client) certificate to see if it is issued by a
> > specific
> > > > CA?
> > > > > I believe it also needs the public key of the user to execute
> > > > > the
> > > > handshake
> > > > > and create the connection.
> > > > >
> > > > >
> > > > >
> > > > No, end user only needs to have Root CA at hand, to *trust* it.
> > > > Both
> > the
> > > > "Server
> > > > Certificate" and "Server Private Key" are sensitive information
> > > > and
> > only
> > > > exist  on
> > > > VR.
> > > >
> > > > User then can go ahead and install the Root CA on their local
> > > > machine
> > and
> > > > open
> > > > up VPN connection with strongSwan client of the correspondning OS
> > they're
> > > > on
> > > > import the Root CA, and their credential (EAP on VPN side), and
> > > &g

Re: [DISCUSS] New VPN implementation based on IKEv2 backed by Vault

[Khosrow Moossavi]

One of the things Vault does is essentially one of the thing Let's Encrypt
does,
acting as CA and generating/signing certificates.

>From the Vault website itself:

"HashiCorp Vault secures, stores, and tightly controls access to tokens,
passwords,
certificates, API keys, and other secrets in modern computing. Vault
handles leasing,
key revocation, key rolling, and auditing. Through a unified API, users can
access an
encrypted Key/Value store and network encryption-as-a-service, or generate
AWS
IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH
credentials,
and more."

In our case we are going to use Vault as PKI backend engine, to act as Root
CA,
sign certificates, handle CRL (Certificate Revocation List), etc.
Technically we can
do these with Let's Encrypt, but I haven't started exploring the
possibilities or potential
limitation. Using external services (such as Let's Encrypt) or going
forward with
Bring You Own Certificate model would be for future, it they ever made
sense to do.



On Wed, Apr 4, 2018 at 11:20 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Got it. Thanks for the explanations.
> There is one other thing I do not understand. This Vault thing that you
> mention, how does it work? Is it similar to let's encrypt?
>
> On Wed, Apr 4, 2018 at 12:15 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > On Wed, Apr 4, 2018 at 10:36 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > So, you need a certificate that is signed by the CA that is used by the
> > VPN
> > > service. Is that it?
> > >
> > >
> > Correct, a self signed "server certificate" against CA, to be installed
> > directly on VR.
> >
> >
> > >
> > > It has been a while that I do not configure these VPN systems; do you
> > need
> > > access to the private key of the CA? Or, does the program simply
> validate
> > > the user (VPN client) certificate to see if it is issued by a specific
> > CA?
> > > I believe it also needs the public key of the user to execute the
> > handshake
> > > and create the connection.
> > >
> > >
> > >
> > No, end user only needs to have Root CA at hand, to *trust* it. Both the
> > "Server
> > Certificate" and "Server Private Key" are sensitive information and only
> > exist  on
> > VR.
> >
> > User then can go ahead and install the Root CA on their local machine and
> > open
> > up VPN connection with strongSwan client of the correspondning OS they're
> > on
> > import the Root CA, and their credential (EAP on VPN side), and that's
> it.
> >
> >
> > >
> > >
> > > On Wed, Apr 4, 2018 at 11:22 AM, Khosrow Moossavi <
> > kmooss...@cloudops.com>
> > > wrote:
> > >
> > > > Rafael,
> > > >
> > > > We cannot use SshKeyPair functionality because the proposed VPN
> > > > implementation
> > > > does need a signed certificate and not a ssh key pair. The process is
> > as
> > > > follow:
> > > >
> > > > 1) generate root CA (if doesn't exist)
> > > > 2) generate bunch of intermediate steps (config urls, CRLs, role
> name,
> > > ...)
> > > > [I'm not going
> > > > in detail now, here, for simplicity]
> > > > 3) self sign a certificate against the root CA (regenerate every time
> > > start
> > > > VPN command
> > > > executed)
> > > >
> > > > This will produce:
> > > >
> > > > 1) Root CA cert (one per domain in cloudstack)
> > > > 2) Server cert (one per VR)
> > > > 3) Server private key (one per VR)
> > > >
> > > > Then all the above will be pushed to the said VR we want to start VPN
> > on,
> > > > and start
> > > > ipsec service on it (with extra configuration - which will be
> available
> > > in
> > > > codebase) and
> > > > finally present Root CA for user to download and install on their
> local
> > > > machine to be
> > > > able to "trust" VR they are VPNing to.
> > > >
> > > >
> > > >
> > > > On Wed, Apr 4, 2018 at 6:19 AM, Rafael Weingärtner <
> > > > rafaelweingart...@gmail.com> wrote:
> > > >
> > > > > Khosrow thanks for the interesting feature. You mention two
> possible
> > > > > methods to manage certificates; one using the CA framework, and
> 

Re: [DISCUSS] New VPN implementation based on IKEv2 backed by Vault

[Khosrow Moossavi]

On Wed, Apr 4, 2018 at 10:36 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> So, you need a certificate that is signed by the CA that is used by the VPN
> service. Is that it?
>
>
Correct, a self signed "server certificate" against CA, to be installed
directly on VR.


>
> It has been a while that I do not configure these VPN systems; do you need
> access to the private key of the CA? Or, does the program simply validate
> the user (VPN client) certificate to see if it is issued by a specific CA?
> I believe it also needs the public key of the user to execute the handshake
> and create the connection.
>
>
>
No, end user only needs to have Root CA at hand, to *trust* it. Both the
"Server
Certificate" and "Server Private Key" are sensitive information and only
exist  on
VR.

User then can go ahead and install the Root CA on their local machine and
open
up VPN connection with strongSwan client of the correspondning OS they're on
import the Root CA, and their credential (EAP on VPN side), and that's it.


>
>
> On Wed, Apr 4, 2018 at 11:22 AM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > Rafael,
> >
> > We cannot use SshKeyPair functionality because the proposed VPN
> > implementation
> > does need a signed certificate and not a ssh key pair. The process is as
> > follow:
> >
> > 1) generate root CA (if doesn't exist)
> > 2) generate bunch of intermediate steps (config urls, CRLs, role name,
> ...)
> > [I'm not going
> > in detail now, here, for simplicity]
> > 3) self sign a certificate against the root CA (regenerate every time
> start
> > VPN command
> > executed)
> >
> > This will produce:
> >
> > 1) Root CA cert (one per domain in cloudstack)
> > 2) Server cert (one per VR)
> > 3) Server private key (one per VR)
> >
> > Then all the above will be pushed to the said VR we want to start VPN on,
> > and start
> > ipsec service on it (with extra configuration - which will be available
> in
> > codebase) and
> > finally present Root CA for user to download and install on their local
> > machine to be
> > able to "trust" VR they are VPNing to.
> >
> >
> >
> > On Wed, Apr 4, 2018 at 6:19 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > Khosrow thanks for the interesting feature. You mention two possible
> > > methods to manage certificates; one using the CA framework, and other
> > using
> > > third party such as Vault and Let’s Encrypt.
> > >
> > > Have you considered using the sshKeyPair API methods (is it part of the
> > CA
> > > framework?)? I mean, users already can generate key pairs via ACS, and
> > then
> > > they are presented with the private key. You could simply list these
> > > certificates for the user when they want to configure a new certificate
> > for
> > > a VPN or generate one in runtime using this feature. Reading your
> feature
> > > proposal I did not understand how you are binding certificated with a
> VPN
> > > (are you always generating new ones and simply returning the private
> key
> > to
> > > users?).
> > >
> > > Moreover, as the sshKeyPair methods, I do believe you should only
> return
> > > the private key once. Therefore, you should not store it in ACS.
> > >
> > > On Mon, Apr 2, 2018 at 4:36 PM, Khosrow Moossavi <
> kmooss...@cloudops.com
> > >
> > > wrote:
> > >
> > > > Hi Community
> > > >
> > > > I want to open up a discussion around the new Remote Access VPN
> > > > implementation on VRs. Currently
> > > > we have only L2TP implementation, which lacks different features
> (such
> > as
> > > > verbos logging), so we
> > > > decided to start developing new implementation based on IKEv2 (on top
> > of
> > > > the existing strongSwan).
> > > >
> > > > We have this feature working locally for over a week now, and seems
> to
> > be
> > > > ready for opening up a
> > > > PR on official repo. But before doing so we agreed to open up a
> > > discussion
> > > > here first.
> > > >
> > > > The current implementation we use EAP + Public Key for
> authentication,
> > so
> > > > we need to have a PKI
> > > > Engine somewhere. Rather than start re-inventing the wheel (and start
> > > > extending the current CA Framework
> > &

Re: [DISCUSS] New VPN implementation based on IKEv2 backed by Vault

[Khosrow Moossavi]

Rafael,

We cannot use SshKeyPair functionality because the proposed VPN
implementation
does need a signed certificate and not a ssh key pair. The process is as
follow:

1) generate root CA (if doesn't exist)
2) generate bunch of intermediate steps (config urls, CRLs, role name, ...)
[I'm not going
in detail now, here, for simplicity]
3) self sign a certificate against the root CA (regenerate every time start
VPN command
executed)

This will produce:

1) Root CA cert (one per domain in cloudstack)
2) Server cert (one per VR)
3) Server private key (one per VR)

Then all the above will be pushed to the said VR we want to start VPN on,
and start
ipsec service on it (with extra configuration - which will be available in
codebase) and
finally present Root CA for user to download and install on their local
machine to be
able to "trust" VR they are VPNing to.



On Wed, Apr 4, 2018 at 6:19 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Khosrow thanks for the interesting feature. You mention two possible
> methods to manage certificates; one using the CA framework, and other using
> third party such as Vault and Let’s Encrypt.
>
> Have you considered using the sshKeyPair API methods (is it part of the CA
> framework?)? I mean, users already can generate key pairs via ACS, and then
> they are presented with the private key. You could simply list these
> certificates for the user when they want to configure a new certificate for
> a VPN or generate one in runtime using this feature. Reading your feature
> proposal I did not understand how you are binding certificated with a VPN
> (are you always generating new ones and simply returning the private key to
> users?).
>
> Moreover, as the sshKeyPair methods, I do believe you should only return
> the private key once. Therefore, you should not store it in ACS.
>
> On Mon, Apr 2, 2018 at 4:36 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > Hi Community
> >
> > I want to open up a discussion around the new Remote Access VPN
> > implementation on VRs. Currently
> > we have only L2TP implementation, which lacks different features (such as
> > verbos logging), so we
> > decided to start developing new implementation based on IKEv2 (on top of
> > the existing strongSwan).
> >
> > We have this feature working locally for over a week now, and seems to be
> > ready for opening up a
> > PR on official repo. But before doing so we agreed to open up a
> discussion
> > here first.
> >
> > The current implementation we use EAP + Public Key for authentication, so
> > we need to have a PKI
> > Engine somewhere. Rather than start re-inventing the wheel (and start
> > extending the current CA Framework
> > which was done by Rohit) we decided to delegate this functionality to
> > HashiCorp Vault, which will act as
> > a PKI backend engine for Cloudstack.
> >
> > The way I implemented this specific part of the code, is that it can
> easily
> > be extended/implemented with other
> > concrete classes or designs (such as going forward with in-house PKI
> > engine, or even use external services
> > such as Let's Encrypt), but at the end of the day we strongly suggest to
> > use Vault, as it is really easy to use.
> >
> >
> > Please find the design document here[1], and share your feedback. I will
> > open up a PR -as is- soon to be able
> > to have a source code to discuss around it as well.
> >
> > [1]:
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> > VPN+Implementation+based+on+IKEv2+backed+by+Vault+as+PKI+Engine
> >
> >
> > Thanks
> >
> > Khosrow Moossavi
> >
> > Cloud Infrastructure Developer
> >
> > t 514.447.3456
> >
> > <https://goo.gl/NYZ8KK>
> >
>
>
>
> --
> Rafael Weingärtner
>

[DISCUSS] New VPN implementation based on IKEv2 backed by Vault

[Khosrow Moossavi]

Hi Community

I want to open up a discussion around the new Remote Access VPN
implementation on VRs. Currently
we have only L2TP implementation, which lacks different features (such as
verbos logging), so we
decided to start developing new implementation based on IKEv2 (on top of
the existing strongSwan).

We have this feature working locally for over a week now, and seems to be
ready for opening up a
PR on official repo. But before doing so we agreed to open up a discussion
here first.

The current implementation we use EAP + Public Key for authentication, so
we need to have a PKI
Engine somewhere. Rather than start re-inventing the wheel (and start
extending the current CA Framework
which was done by Rohit) we decided to delegate this functionality to
HashiCorp Vault, which will act as
a PKI backend engine for Cloudstack.

The way I implemented this specific part of the code, is that it can easily
be extended/implemented with other
concrete classes or designs (such as going forward with in-house PKI
engine, or even use external services
such as Let's Encrypt), but at the end of the day we strongly suggest to
use Vault, as it is really easy to use.


Please find the design document here[1], and share your feedback. I will
open up a PR -as is- soon to be able
to have a source code to discuss around it as well.

[1]:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/VPN+Implementation+based+on+IKEv2+backed+by+Vault+as+PKI+Engine


Thanks

Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>

Re: Committee to Sort through CCC Presentation Submissions

[Khosrow Moossavi]

I can help as well.




On Tue, Mar 27, 2018 at 4:19 PM, Will Stevens  wrote:

> I can support this.
>
> Cheers,
>
> Will
>
> On Tue, Mar 27, 2018, 12:39 PM Tutkowski, Mike,  >
> wrote:
>
> > Hi everyone,
> >
> > As you may be aware, this coming September in Montreal, the CloudStack
> > Community will be hosting the CloudStack Collaboration Conference:
> >
> > http://ca.cloudstackcollab.org/
> >
> > Even though the event is six months away, we are on a tight schedule with
> > regards to the Call For Participation (CFP):
> >
> > https://www.apachecon.com/acna18/schedule.html
> >
> > If you are interested in submitting a talk, please do so before March
> 30th.
> >
> > That being said, as usual, we will have need of a small committee to sort
> > through these presentation submissions.
> >
> > If you are interested in helping out in this process, please reply to
> this
> > message.
> >
> > Thanks!
> > Mike
> >
>

Re: Dockerizing Cloudstack package builders

[Khosrow Moossavi]

Thanks for the feedback Wido.

I, personally, wouldn't put them in *cloudstack* repo itself, since these
are complementary tools. To me it would
make more sense to put them under apache/ namespace on GitHub as we already
have bunch of apache/cloudstack-*
repos. And under apache/ or cloudstackl/ namespace on Docker Hub (whichever
makes more sense).

If community agrees with going forward with this proposal, I would be more
than happy to transfer ownership of
the GH and DH repos I created.

That's actually a great point. I was assuming these are going to be used by
only developers/maintainers which have

the source code at their disposals. I'm gonna wait for others
feedback/suggestions before enhancing this though.

On Mon, Mar 26, 2018 at 3:28 PM, Wido den Hollander <w...@widodh.nl> wrote:

>
>
> On 03/26/2018 06:21 PM, Khosrow Moossavi wrote:
> > Hi community
> >
> > To make building Cloudstack's RPM or DEB packages more easily, portable
> and
> > reduce the need to
> > have a *correct* hardware to build them, I've Dockerized them. This is
> > essentially a follow up of what
> > Wido had done (only for deb packages though).
> >
>
> Very nice! I didn't have the time to work on this, but it kept popping
> up. I'd love to have a automated builder on download.cloudstack.org
> which does this for us automatically.
>
> > The proposed Builders can be found here[1] and here[2] on Github and
> > here[3] and here[4] on Docker Hub.
> >
>
> Cool!
>
> > I would appreciate it if you can give your feedback of them and if need
> be,
> > promote them on the official wiki/doc/repo.
> >
>
> Getting them on a official repo should be possible I think, we could put
> them in the /packaging directory of our main repo. Or maybe request a
> second repo called 'cloudstack-docker' or something.
>
> Right now I see that your builder wants to have the cloudstack source
> ready to be used (just as I did), but how about adding the option where
> it downloads a tarball from the website if no source and just a version
> is provided? Or it can do a git clone from Github and check out a
> specific tag/version.
>
> Again, very much appreciated!
>
> Wido
>
> > [1]: https://github.com/khos2ow/cloudstack-rpm-builder
> > [2]: https://github.com/khos2ow/cloudstack-deb-builder
> > [3]: https://hub.docker.com/r/khos2ow/cloudstack-deb-builder/
> > [4]: https://hub.docker.com/r/khos2ow/cloudstack-rpm-builder/
> >
> > Thanks
> >
> > Khosrow Moossavi
> >
> > Cloud Infrastructure Developer
> >
> > t 514.447.3456
> >
> > <https://goo.gl/NYZ8KK>
> >
>

Dockerizing Cloudstack package builders

[Khosrow Moossavi]

Hi community

To make building Cloudstack's RPM or DEB packages more easily, portable and
reduce the need to
have a *correct* hardware to build them, I've Dockerized them. This is
essentially a follow up of what
Wido had done (only for deb packages though).

The proposed Builders can be found here[1] and here[2] on Github and
here[3] and here[4] on Docker Hub.

I would appreciate it if you can give your feedback of them and if need be,
promote them on the official wiki/doc/repo.

[1]: https://github.com/khos2ow/cloudstack-rpm-builder
[2]: https://github.com/khos2ow/cloudstack-deb-builder
[3]: https://hub.docker.com/r/khos2ow/cloudstack-deb-builder/
[4]: https://hub.docker.com/r/khos2ow/cloudstack-rpm-builder/

Thanks

Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>

Re: [VOTE] Move to Github issues

[Khosrow Moossavi]

+1


On Mon, Mar 26, 2018 at 9:05 AM, Will Stevens  wrote:

> +1
>
> On Mon, Mar 26, 2018, 5:51 AM Nicolas Vazquez, <
> nicolas.vazq...@shapeblue.com> wrote:
>
> > +1
> >
> > 
> > From: Dag Sonstebo 
> > Sent: Monday, March 26, 2018 5:06:29 AM
> > To: us...@cloudstack.apache.org; dev@cloudstack.apache.org
> > Subject: Re: [VOTE] Move to Github issues
> >
> > +1
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > On 26/03/2018, 07:33, "Rohit Yadav"  wrote:
> >
> > All,
> >
> > Based on the discussion last week [1], I would like to start a vote
> to
> > put
> > the proposal into effect:
> >
> > - Enable Github issues, wiki features in CloudStack repositories.
> > - Both user and developers can use Github issues for tracking issues.
> > - Developers can use #id references while fixing an existing/open
> > issue in
> > a PR [2]. PRs can be sent without requiring to open/create an issue.
> > - Use Github milestone to track both issues and pull requests
> towards a
> > CloudStack release, and generate release notes.
> > - Relax requirement for JIRA IDs, JIRA still to be used for
> historical
> > reference and security issues. Use of JIRA will be discouraged.
> > - The current requirement of two(+) non-author LGTMs will continue
> for
> > PR
> > acceptance. The two(+) PR non-authors can advise resolution to any
> > issue
> > that we've not already discussed/agreed upon.
> >
> > For sanity in tallying the vote, can PMC members please be sure to
> > indicate
> > "(binding)" with their vote?
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> > Vote will be open for 120 hours. If the vote passes the following
> > actions
> > will be taken:
> > - Get Github features enabled from ASF INFRA
> > - Update CONTRIBUTING.md and other relevant cwiki pages.
> > - Update project website
> >
> > [1] https://markmail.org/message/llodbwsmzgx5hod6
> > [2]
> > https://blog.github.com/2013-05-14-closing-issues-via-pull-requests/
> >
> > Regards,
> > Rohit Yadav
> >
> >
> >
> > dag.sonst...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > nicolas.vazq...@shapeblue.com
> > www.shapeblue.com
> > ,
> > @shapeblue
> >
> >
> >
> >
>

Re: Marketing CCC Montreal

[Khosrow Moossavi]

Well whatever the dates it will not be in May, it will be in September!




On Thu, Mar 22, 2018 at 10:55 AM, Will Stevens <wstev...@cloudops.com>
wrote:

> The dates are somewhat unclear, so its not really 'wrong' yet.  We know we
> start on the 24th.  We are not sure what the second day we will get from
> ApacheCon is yet.  I am working on getting a space together for a hackathon
> day on either the 25th or 26th depending on what the second day we get from
> ApacheCon is, so there are still some logistics in play still.
>
> I am advertising the 24-26th in the absence of official details as I am
> confident I can get another space online for a hackathon day.  More details
> will be available once we get details from ApacheCon and I work out a space
> for the hackathon.
>
> I hope that is not too difficult to understand.
>
> Cheers,
>
> *Will Stevens*
> Chief Technology Officer
> c 514.826.0190
>
> <https://goo.gl/NYZ8KK>
>
> On Thu, Mar 22, 2018 at 10:49 AM, Khosrow Moossavi <kmooss...@cloudops.com
> >
> wrote:
>
> > Somehow related, the tweet sent out earlier today, has the date wrong.
> >
> > https://twitter.com/CloudStack/status/976819870426902530?s=19
> >
> >
> > On Mar 22, 2018 10:37, "Will Stevens" <sw...@apache.org> wrote:
> >
> > > Hey Everyone,
> > > We need your help to promote the visibility of the CCC conference we
> are
> > > running later this year in September.
> > >
> > > I have put together an information website to help us promote the CCC:
> > > http://ca.cloudstackcollab.org/
> > >
> > > I will keep the website updated with additional detail as we get more
> > > information.
> > >
> > > *PLEASE NOTE: Call For Papers closes on March 30th.  Please get your
> > topics
> > > in ASAP.*  On that note, make sure the title of your talk includes the
> > word
> > > 'CloudStack' so we are able to get the talk in the correct CCC track.
> > >
> > > Additionally, in order to get more visibility for the event, please
> > > consider adding the following footer to your email signature (ideally
> > > linking to the CCC site).  The footer is available here:
> > > http://ca.cloudstackcollab.org/img/ccc_mtl_footer.png
> > >
> > > Let us know if you have any questions...
> > >
> > > *Will Stevens*
> > >
> > >
> > > * <http://ca.cloudstackcollab.org/>*
> > >
> >
>

Re: Marketing CCC Montreal

[Khosrow Moossavi]

Somehow related, the tweet sent out earlier today, has the date wrong.

https://twitter.com/CloudStack/status/976819870426902530?s=19


On Mar 22, 2018 10:37, "Will Stevens"  wrote:

> Hey Everyone,
> We need your help to promote the visibility of the CCC conference we are
> running later this year in September.
>
> I have put together an information website to help us promote the CCC:
> http://ca.cloudstackcollab.org/
>
> I will keep the website updated with additional detail as we get more
> information.
>
> *PLEASE NOTE: Call For Papers closes on March 30th.  Please get your topics
> in ASAP.*  On that note, make sure the title of your talk includes the word
> 'CloudStack' so we are able to get the talk in the correct CCC track.
>
> Additionally, in order to get more visibility for the event, please
> consider adding the following footer to your email signature (ideally
> linking to the CCC site).  The footer is available here:
> http://ca.cloudstackcollab.org/img/ccc_mtl_footer.png
>
> Let us know if you have any questions...
>
> *Will Stevens*
>
>
> * *
>

Re: [DISCUSS] Relax strict requirement of JIRA ID for PRs

[Khosrow Moossavi]

Thanks Rohit, that's a really great sum-up of proposition!

According to the private issue topic, as far as I understand, we don't have
any private issue per se, unless they are security, vulnerability
or CVE related and the process of reporting them -being really private
until they are fixed- are well-defined. So I would say the mentioned
security@ ML and have an interim ticket in Jira or the ML itself works and
when the fix is out, for sake of archive, the issue can be created
in GH and set as closed right away.



On Wed, Mar 21, 2018 at 6:38 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> I think that works as well. Let's see what others think about it.
>
> On Wed, Mar 21, 2018 at 4:15 AM, Rohit Yadav 
> wrote:
>
> > Thanks Rafael for your inputs. You're right about access control
> > limitation on Github.
> >
> >
> > However, I think having another repository to track security stuff can
> add
> > overhead (and to manage its custom ACLs with INFRA, as all cloudstack*
> > repos are allowed access to all PMC/committers now).
> >
> >
> > Users are advised to report security issues on security@, so how about
> we
> > continue to use JIRA for security issues (logging, tracking, and
> > discussions) and limit the proposed change to just moving to Github
> issues
> > as a first step?
> >
> >
> > - Rohit
> >
> > 
> >
> >
> >
> > 
> > From: Rafael Weingärtner 
> > Sent: Tuesday, March 20, 2018 11:46:32 PM
> > To: dev
> > Cc: us...@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Relax strict requirement of JIRA ID for PRs
> >
> > It looks like you have done all of the homework here. If it comes to a
> > vote, I am +1 on migrating issues to Github, and even the Wiki in the
> > future.  The Github would be able to pretty much provide everything that
> we
> > have in both Wiki and Jira. Therefore, it feels better to work on a
> single
> > platform than to spread information across them. However, we still have
> one
> > problem. The security issues/tickets in Jira. How can we manage them in
> > Github? As far as I know, there is no way to control the access to
> certain
> > issues/tickets in Github.
> >
> > To tackle that problem with security issues we could open a ticket with
> > Github; and in the meantime, we could set up a private repository in the
> > Apache organization to hold the security issues (e.g.
> cloudstack-security).
> >
> >
> > Thanks Rohit.
> >
> >
> >
> > On Mon, Mar 19, 2018 at 7:58 AM, Rohit Yadav 
> > wrote:
> >
> > > (I've cc-ed user ML to gather feedback from users for this email as
> > well.)
> > > All,
> > > Thank you for your feedbacks, discussions, and suggestions. I have
> tried
> > > to take on board all the feedback from the discussion and I propose the
> > > following:
> > > # Problem
> > > Let me summarize the problems we're facing and propose some solution
> > > (which may require voting) in the next section:
> > > - Search:
> > > The source of truth is in the git repository (Github or asf mirror),
> > > however, our issue tracking and wiki systems are different. Therefore
> any
> > > task requires us to move back and forth between these various
> > > portals/systems. As an example - a contributor trying to find whether
> an
> > > issue was fixed, requires them to search both JIRA, Github for pull
> > > requests or commits (and sometimes the release notes and MLs).
> > > - Audience/Platform:
> > > From an audience's perspective, the user ML and JIRA issue are for
> users
> > > to be able to reach the community and seek help with a bug or request a
> > new
> > > feature.
> > > The dev ML, and github PR are ways that developers usually use to
> > > fix/address an issue or develop a new feature, and get them accepted
> > > towards a release.
> > > CWiki is used by both to track articles, documentation and FSs, the
> docs
> > > website hosts docs for install/admin/release notes is user-facing. Both
> > > JIRA and Confluence are slow compared to Github. The docs website is a
> > > static website and is fast.
> > > - Relationship and discovery:
> > > Historically, the main reason for having a JIRA id with a PR/commit is
> to
> > > be able to track changes for an open bug and gather such a list in the
> > > release notes. It also helps with cross-searching of a PR against a
> JIRA
> > > ticket and searching a JIRA id for possible discussion from an accepted
> > PR.
> > > A git commit (for example on github) can also help by telling us which
> > tags
> > > or branches the commit was included in, so helps in knowing which
> version
> > > of ACS will have that in.
> > >  - Behaviours:
> > > With the current strict requirement of JIRA ids for each PR, we see
> these
> > > behaviours:
> > > (a) many times the author may not engage after sending the PR and may
> not
> > > add a JIRA id causing that PR to get blocked indefinitely,
> > 

Re: New committer: Dag Sonstebo

[Khosrow Moossavi]

Congratulations Dag!



On Wed, Mar 21, 2018 at 6:57 AM, Daan Hoogland 
wrote:

> hey Dag, keep up the good work, thanks and welcome
>
> On Tue, Mar 20, 2018 at 9:21 PM, Simon Weller 
> wrote:
>
> > Congrats Dag, much deserved!
> >
> > 
> > From: John Kinsella 
> > Sent: Tuesday, March 20, 2018 8:58 AM
> > To: 
> > Subject: New committer: Dag Sonstebo
> >
> > The Project Management Committee (PMC) for Apache CloudStack has
> > invited Dag Sonsteboto become a committer and we are pleased to
> > announce that he has accepted.
> >
> > I’ll take a moment here to remind folks that being an ASF committer
> > isn’t purely about code - Dag has been helping out for quite a while
> > on users@, and seems to have a strong interest around ACS and the
> > community. We welcome this activity, and encourage others to help
> > out as they can - it doesn’t necessarily have to be purely code-related.
> >
> > Being a committer enables easier contribution to the project since
> > there is no need to go via the patch submission process. This should
> > enable better productivity.
> >
> > Please join me in welcoming Dag!
> >
> > John
> >
>
>
>
> --
> Daan
>

Re: Notice that Gabriel Bräscher now works at PCextreme

[Khosrow Moossavi]

Congratulations Gabriel!



On Wed, Mar 21, 2018 at 5:40 AM, Daan Hoogland 
wrote:

> nice going Gabriel and congratulations to PCextreme!
>
> On Tue, Mar 20, 2018 at 8:36 PM, Gabriel Beims Bräscher <
> gabrasc...@gmail.com> wrote:
>
> > Thank you, guys :)
> >
> > 2018-03-20 16:23 GMT-03:00 Wei ZHOU :
> >
> > > Congratulations Gabriel!
> > >
> > > Welcome to NL :)
> > >
> > >
> > >
> > > 2018-03-20 14:50 GMT+01:00 Wido den Hollander :
> > >
> > >> Hi,
> > >>
> > >> Just wanted to let you know that Gabriel Bräscher started working at
> > >> PCextreme this week.
> > >>
> > >> He'll be committing and developing on CloudStack for PCextreme and the
> > >> community.
> > >>
> > >> Just so everybody knows that we are colleagues now.
> > >>
> > >> Let's make CloudStack even better!
> > >>
> > >> Wido
> > >>
> > >
> > >
> >
>
>
>
> --
> Daan
>

Re: [DISCUSS] CloudStack Connection Pools

[Khosrow Moossavi]

Why would we want to expose this choice to administrator of Cloudstack
whose responsibility
is to keep it running and not knowing about the inner-mechanic of how it
works. right? It's not
like that we're giving them a choice of which database to connect to.

So on that note, I would say we need to agree on any of those CP libraries
and implement, the
same way we chose for example log4j or slf4j over one another, or any other
_library_ we use.

Khosrow Moossavi

CloudOps



On Wed, Mar 14, 2018 at 10:36 AM, Nicolas Vazquez <
nicolas.vazq...@shapeblue.com> wrote:

> Thanks Khosrow and Rafael. You both agree on Spring Data as the best
> option, I see it would require a big effort and commitment to migrate to
> it, therefore it can take some (long) time to achieve it.
>
> As a more viable option, would you agree on supporting different
> connection pool management libraries and letting the administrator choose
> which one to use? (DBCP 1.4 as default)
>
> 
> From: Rafael Weingärtner <rafaelweingart...@gmail.com>
> Sent: Tuesday, March 13, 2018 8:52:50 AM
> To: dev
> Subject: Re: [DISCUSS] CloudStack Connection Pools
>
> Spring data would be awesome. It is very flexible and has a very good API.
> However, this would require commitment from our side to slowly migrate
> things to it.
>
> Regarding the connection pool management libraries; I would prefer either
> C3P0 or 2.* DBCP. The other two sound trendy, but I worry about this type
> of project in the long run. Both DBCP from Apache and C3P0 from Hibernate
> (RedHat) sound a more reasonable selection for me. They have been around
> for years, and have a solid community base already.
>
> On Mon, Mar 12, 2018 at 11:31 PM, Khosrow Moossavi <kmooss...@cloudops.com
> >
> wrote:
>
> > Hi Nicolas
> >
> > From my past experiences, I prefer 1) HikariCP 2) Tomcat Pool 3) C3P0 4)
> > DBCP in that order. Although I don't have
> > any benchmark of my own to provide, and the ones you mentioned are really
> > informative anyway.
> >
> > To me the broader subject is the _one_ who uses the pool, I mean if the
> > transactions are handled in a faster way and
> > released sooner and with shorter locks, generally speaking if it's more
> > efficient, I don't think from ACS point of view
> > there won't be much difference between the above mentioned options.
> >
> > On the same subject, it might be more interesting to use Spring Boot in
> > general and Spring Boot Data in particular
> > rather than only changing the CP functionality, and slowly
> refactor/retire
> > the DAO layer in favor of Spring Boot equivalent
> > implementation.
> >
> >
> > Khosrow Moossavi
> >
> > CloudOps
> >
> >
> >
> > On Mon, Mar 12, 2018 at 9:32 PM, Nicolas Vazquez <
> > nicolas.vazq...@shapeblue.com> wrote:
> >
> > > Hi all,
> > >
> > >
> > > I would like to introduce a topic for discussion, regarding DB
> connection
> > > pools used in CloudStack, currently Apache Commons DBCP 1.4 (
> > > http://commons.apache.org/) is used. I've been investigating this
> topic
> > > as we are having complains of random issues on MySQL connection pool on
> > > large environments. Please let me know if this topic has already been
> > > discussed before.
> > >
> > >
> > > First of all, DBCP 1.4 has been released on 2010 (
> > > https://commons.apache.org/proper/commons-dbcp/changes-report.html),
> and
> > > no minor/patch version has been released since then. It seems to work
> in
> > > high performance with relatively low traffic and low load applications.
> > > However, it is single threaded, and in order to be thread-safe, the
> > entire
> > > pool needs to be locked. It is also reported that an CPU and concurrent
> > > threads increases, the performance gets affected. This is a serious
> issue
> > > on highly concurrent systems, such as CloudStack.
> > >
> > >
> > > I've been investigating some options to replace it:
> > > - The first option can be upgrading to version 2.x. Issues on
> performance
> > > and concurrency could be solved using this version.
> > > - Tomcat JDBC Connection Pool. Please check:
> https://tomcat.apache.org/
> > > tomcat-7.0-doc/jdbc-pool.html.
> > >
> > > - Other replacement options found: BoneCP, C3P0, HikariCP
> > >
> > >
> > > Given these options, I've been looking for benchmarks to compare them
> > (*).
> > > Looks like HikariCP (http://brettwooldr

Re: [DISCUSS] Relax strict requirement of JIRA ID for PRs

[Khosrow Moossavi]

I'm completely +1 on using GH as source of truth, both PR and issue wise,
with Daan comment regarding Apache rules in mind.
At least it doesn't need to have "yet another" integration to do automated
actions on an issue (such as auto close an issue by "Fixes NUMBER",
"Closes NUMBER") directly from commit or PR body.

Khosrow Moossavi
CloudOps

On Tue, Mar 13, 2018 at 10:11 AM, Syed Ahmed <sah...@cloudops.com> wrote:

> I agree with the relaxation as Rohit pointed out. At this point we should
> ask if Jira is really needed. Most people here I believe agree that it is
> not. The only reason we have Jira is to track releases. This could easily
> be replicated in GitHub as I see that GitHub is the place where we all
> collaborate. I would be completely in if we use GitHub issues and like it
> with Jira as we do with our PRs.
>
>
> On Tue, Mar 13, 2018 at 10:03 AM Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > I was checking and for some reason ACS does not have an issue tab (
> > https://github.com/apache/cloudstack/issues). It might be some
> > configuration in the repository.
> >
> > On Tue, Mar 13, 2018 at 10:54 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > What do you mean by issue? PR or issue (Github issue)?
> > >
> > > On Tue, Mar 13, 2018 at 10:53 AM, Daan Hoogland <
> daan.hoogl...@gmail.com
> > >
> > > wrote:
> > >
> > >> right, also when an issue is created?
> > >>
> > >>
> > >> On Tue, Mar 13, 2018 at 2:50 PM, Rafael Weingärtner <
> > >> rafaelweingart...@gmail.com> wrote:
> > >>
> > >> > We already have. All messages on ACS' GH go to commits@c.a.o
> > >> >
> > >> > On Tue, Mar 13, 2018 at 10:49 AM, Daan Hoogland <
> > >> daan.hoogl...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Ok, one issue there is Apache foundation rules. If we copy every
> > thing
> > >> > into
> > >> > > jira or the mail list we are fine, where ever we have our
> > discussions.
> > >> > The
> > >> > > only thing is that we need a Apache hosted record. (as in not
> > github)
> > >> > >
> > >> > > On Tue, Mar 13, 2018 at 2:09 PM, Rafael Weingärtner <
> > >> > > rafaelweingart...@gmail.com> wrote:
> > >> > >
> > >> > > > I prefer the workflow in Github as you guy, but to be fair with
> > Jira
> > >> > > ticket
> > >> > > > system I mentioned it.
> > >> > > >
> > >> > > > @Marc, yes Jira can facilitate a lot the management. However, we
> > do
> > >> not
> > >> > > use
> > >> > > > it fully. In our workflow, there is no planning/roadmap for the
> > next
> > >> > > > release per se. Things seem to work in an ad-hoc fashion. On the
> > >> other
> > >> > > > hand, when you need to break down milestones into
> > >> issues/tickets/tasks
> > >> > > and
> > >> > > > then divide them into sprints, and manage a team of developers,
> > the
> > >> > > > oversight provided by Jira system is very good; specially, when
> > >> issues
> > >> > > > start to take more than a single sprint to finish.
> > >> > > >
> > >> > > > On Tue, Mar 13, 2018 at 9:44 AM, Marc-Aurèle Brothier <
> > >> > ma...@exoscale.ch
> > >> > > >
> > >> > > > wrote:
> > >> > > >
> > >> > > > > @rafael, you said: they all required Jira tickets to track the
> > >> > > discussion
> > >> > > > > and facilitate the management
> > >> > > > >
> > >> > > > > I can see the discussion happening in the PR on github, but
> the
> > >> Jira
> > >> > > > ticket
> > >> > > > > by itself doesn't do much, except copy/pasting the github
> > >> discussion.
> > >> > > > Then
> > >> > > > > it's down to "facilitate the management", which I only see as
> > >> listing
> > >> > > the
> > >> > > > > changes for a release as far as I know. But this can be
> achieved
> > >>

Re: [DISCUSS] CloudStack Connection Pools

[Khosrow Moossavi]

Hi Nicolas

>From my past experiences, I prefer 1) HikariCP 2) Tomcat Pool 3) C3P0 4)
DBCP in that order. Although I don't have
any benchmark of my own to provide, and the ones you mentioned are really
informative anyway.

To me the broader subject is the _one_ who uses the pool, I mean if the
transactions are handled in a faster way and
released sooner and with shorter locks, generally speaking if it's more
efficient, I don't think from ACS point of view
there won't be much difference between the above mentioned options.

On the same subject, it might be more interesting to use Spring Boot in
general and Spring Boot Data in particular
rather than only changing the CP functionality, and slowly refactor/retire
the DAO layer in favor of Spring Boot equivalent
implementation.


Khosrow Moossavi

CloudOps



On Mon, Mar 12, 2018 at 9:32 PM, Nicolas Vazquez <
nicolas.vazq...@shapeblue.com> wrote:

> Hi all,
>
>
> I would like to introduce a topic for discussion, regarding DB connection
> pools used in CloudStack, currently Apache Commons DBCP 1.4 (
> http://commons.apache.org/) is used. I've been investigating this topic
> as we are having complains of random issues on MySQL connection pool on
> large environments. Please let me know if this topic has already been
> discussed before.
>
>
> First of all, DBCP 1.4 has been released on 2010 (
> https://commons.apache.org/proper/commons-dbcp/changes-report.html), and
> no minor/patch version has been released since then. It seems to work in
> high performance with relatively low traffic and low load applications.
> However, it is single threaded, and in order to be thread-safe, the entire
> pool needs to be locked. It is also reported that an CPU and concurrent
> threads increases, the performance gets affected. This is a serious issue
> on highly concurrent systems, such as CloudStack.
>
>
> I've been investigating some options to replace it:
> - The first option can be upgrading to version 2.x. Issues on performance
> and concurrency could be solved using this version.
> - Tomcat JDBC Connection Pool. Please check: https://tomcat.apache.org/
> tomcat-7.0-doc/jdbc-pool.html.
>
> - Other replacement options found: BoneCP, C3P0, HikariCP
>
>
> Given these options, I've been looking for benchmarks to compare them (*).
> Looks like HikariCP (http://brettwooldridge.github.io/HikariCP/) could be
> the best replacement, improving performance and stability. Another good
> replacement option could be Tomcat.
>
>
> I've been also examining the codebase, data source initialization is done
> on TransactionLegacy class under the cloud-framework-db project.
> Replacement work should be done on this class. Instead of pure replacement,
> a global setting can be introduced to make the admins able to select which
> connection pool to use.
>
> What do you think? Any possitive/negative feedback is welcome as well as
> new ideas. As mentioned before, I don't know if it has been discussed
> before, sorry in advance if it has.
>
> Kind regards,
> Nicolas
>
> (*) Links to benchmarks and comparissons:
> https://www.wix.engineering/single-post/how-many-threads-
> does-it-take-to-fill-a-pool
> https://www.wix.engineering/single-post/how-does-hikaricp-
> compare-to-other-connection-pools
> <https://www.wix.engineering/single-post/how-does-hikaricp-
> compare-to-other-connection-pools>https://beansroasted.
> wordpress.com/2017/07/29/connection-pool-analysis/
> https://beansroasted.wordpress.com/tag/connection-pool-comparison/
> <https://beansroasted.wordpress.com/tag/connection-pool-comparison/>
> https://github.com/brettwooldridge/HikariCP/wiki/%22My-benchmark-
> doesn't-show-a-difference.%22
> http://www.trustiv.co.uk/2014/06/battle-connection-pools
>
> nicolas.vazq...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>
>

Re: GSoC ideas for the 2018 edition

[Khosrow Moossavi]

Cool! +1


On Mon, Mar 5, 2018 at 4:13 PM, Daan Hoogland <daan.hoogl...@gmail.com>
wrote:

> fixed
>
> On Mon, Mar 5, 2018 at 7:43 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > Hi Daan
> >
> > I just noticed filter for issues on gsoc2018 [1] doesn't show all of them
> > when filtered by CLOUDSTACK
> > project, if you haven't logged in. Is there a security setting somewhere
> to
> > make them public?
> >
> > I guess it would be better for anyone to see the list, and not make them
> to
> > register first.
> >
> > [1]: http://s.apache.org/gsoc2018ideas
> >
> > Khosrow Moossavi
> >
> > <https://goo.gl/NYZ8KK>
> >
> >
> >
> > On Mon, Jan 22, 2018 at 4:21 AM, Daan Hoogland <daan.hoogl...@gmail.com>
> > wrote:
> >
> > > devs,
> > >
> > > Google Summer of Code is coming again. Apache is applying as
> organisation
> > > and we can hitch on that.
> > > I know we have a lot of ideas with varying probability of success for
> > which
> > > we have no time ourselves.
> > > Let me share some relevant information:
> > >
> > >
> > > Google Summer of Code [1] is a program sponsored by Google allowing
> > > students to spend their summer
> > > working on open source software. Students will receive stipends for
> > > developing open source software
> > > full-time for three months. Projects will provide mentoring and project
> > > ideas, and in return have
> > > the chance to get new code developed and - most importantly - to
> identify
> > > and bring in new committers.
> > >
> > > The ASF will apply as a participating organization meaning individual
> > > projects don't have to apply
> > > separately.
> > >
> > > If you want to participate with your project we ask you to do the
> > following
> > > things as soon as
> > > possible but please no later than 2017-01-30:
> > >
> > > 1. understand what it means to be a mentor [2].
> > >
> > > 2. record your project ideas.
> > >
> > > Just create issues in JIRA, label them with gsoc2018, and they will
> show
> > up
> > > at [3]. Please be as
> > > specific as possible when describing your idea. Include the programming
> > > language, the tools and
> > > skills required, but try not to scare potential students away. They are
> > > supposed to learn what's
> > > required before the program starts.
> > >
> > > ​Mark those projects with the label GSoC2018 and
> > > ​u​
> > > se
> > > ​ additional​
> > > labels, e.g. for the programming language (java, c, c++, erlang,
> python,
> > > brainfuck, ...) or
> > > technology area (cloud, xml, web, foo, bar, ...)
> > > Share those label with the rest of us devs, for instance on this
> thread.
> > >
> > >
> > > ​
> > > [4] contains some additional information (will be updated for 2017
> > > shortly).
> > >
> > > 3. subscribe to ment...@community.apache.org; restricted to potential
> > > mentors, meant to be used as a
> > > private list - general discussions on the public
> > d...@community.apache.org
> > > list as much as possible
> > > please). Use a recognized address when subscribing (@apache.org or one
> > of
> > > your alias addresses on
> > > record).
> > >
> > > Note that the ASF isn't accepted as a participating organization yet,
> > > nevertheless you *have to*
> > > start recording your ideas now or we will not get accepted.
> > >
> > > Over the years we were able to complete hundreds of projects
> > successfully.
> > > Some of our prior
> > > students are active contributors now! Let's make this year a success
> > again!
> > >
> > >
> > > [1] https://summerofcode.withgoogle.com/
> > > [2] http://community.apache.org/guide-to-being-a-mentor.html
> > > [3] http://s.apache.org/gsoc2018ideas
> > > [4] http://community.apache.org/gsoc.html
> > >
> > > --
> > > Daan
> > >
> >
>
>
>
> --
> Daan
>

Re: GSoC ideas for the 2018 edition

[Khosrow Moossavi]

Hi Daan

I just noticed filter for issues on gsoc2018 [1] doesn't show all of them
when filtered by CLOUDSTACK
project, if you haven't logged in. Is there a security setting somewhere to
make them public?

I guess it would be better for anyone to see the list, and not make them to
register first.

[1]: http://s.apache.org/gsoc2018ideas

Khosrow Moossavi

<https://goo.gl/NYZ8KK>



On Mon, Jan 22, 2018 at 4:21 AM, Daan Hoogland <daan.hoogl...@gmail.com>
wrote:

> devs,
>
> Google Summer of Code is coming again. Apache is applying as organisation
> and we can hitch on that.
> I know we have a lot of ideas with varying probability of success for which
> we have no time ourselves.
> Let me share some relevant information:
>
>
> Google Summer of Code [1] is a program sponsored by Google allowing
> students to spend their summer
> working on open source software. Students will receive stipends for
> developing open source software
> full-time for three months. Projects will provide mentoring and project
> ideas, and in return have
> the chance to get new code developed and - most importantly - to identify
> and bring in new committers.
>
> The ASF will apply as a participating organization meaning individual
> projects don't have to apply
> separately.
>
> If you want to participate with your project we ask you to do the following
> things as soon as
> possible but please no later than 2017-01-30:
>
> 1. understand what it means to be a mentor [2].
>
> 2. record your project ideas.
>
> Just create issues in JIRA, label them with gsoc2018, and they will show up
> at [3]. Please be as
> specific as possible when describing your idea. Include the programming
> language, the tools and
> skills required, but try not to scare potential students away. They are
> supposed to learn what's
> required before the program starts.
>
> ​Mark those projects with the label GSoC2018 and
> ​u​
> se
> ​ additional​
> labels, e.g. for the programming language (java, c, c++, erlang, python,
> brainfuck, ...) or
> technology area (cloud, xml, web, foo, bar, ...)
> Share those label with the rest of us devs, for instance on this thread.
>
>
> ​
> [4] contains some additional information (will be updated for 2017
> shortly).
>
> 3. subscribe to ment...@community.apache.org; restricted to potential
> mentors, meant to be used as a
> private list - general discussions on the public d...@community.apache.org
> list as much as possible
> please). Use a recognized address when subscribing (@apache.org or one of
> your alias addresses on
> record).
>
> Note that the ASF isn't accepted as a participating organization yet,
> nevertheless you *have to*
> start recording your ideas now or we will not get accepted.
>
> Over the years we were able to complete hundreds of projects successfully.
> Some of our prior
> students are active contributors now! Let's make this year a success again!
>
>
> [1] https://summerofcode.withgoogle.com/
> [2] http://community.apache.org/guide-to-being-a-mentor.html
> [3] http://s.apache.org/gsoc2018ideas
> [4] http://community.apache.org/gsoc.html
>
> --
> Daan
>

Re: POM version in 4.11 branch

[Khosrow Moossavi]

I see, thanks Rohit.



On Mon, Feb 26, 2018 at 10:58 AM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Thanks for sharing Khosrow, yes that's an error. There was an issue around
> db failure for 4.10.0.0 users upgrade to 4.11.0.0 -- which was sorted in
> release notes instead of doing a new 4.11.0.1 release.
>
>
> Let me fix that.
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Khosrow Moossavi <kmooss...@cloudops.com>
> Sent: Monday, February 26, 2018 4:46:36 PM
> To: dev@cloudstack.apache.org
> Subject: POM version in 4.11 branch
>
> Hi
>
> While working on a PR, Rafael and myself noticed that the POM version of
> 4.11 branch
> is 4.11.0.1-SNAPSHOT and not 4.11.1.0-SNAPSHOT.
> Was this planned? Shouldn't we be on micro version rather than patch
> version?
>
> Khosrow Moossavi
>
> CloudOps
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

POM version in 4.11 branch

[Khosrow Moossavi]

Hi

While working on a PR, Rafael and myself noticed that the POM version of
4.11 branch
is 4.11.0.1-SNAPSHOT and not 4.11.1.0-SNAPSHOT.
Was this planned? Shouldn't we be on micro version rather than patch
version?

Khosrow Moossavi

CloudOps

Re: I'd like to introduce you to Khosrow

[Khosrow Moossavi]

Thank you Pierre-Luc,
I'm super excited to be part of the community.

On Feb 22, 2018 18:42, "Rafael Weingärtner" <rafaelweingart...@gmail.com>
wrote:

> Welcome!
> Congratualations for the great job done so far...
>
> On Thu, Feb 22, 2018 at 8:40 PM, Pierre-Luc Dion <pd...@cloudops.com>
> wrote:
>
> > Hi fellow colleagues,
> >
> > I might be a bit late with this email...
> >
> > I'd like to introduce Khosrow Moossavi, who recently join our team and
> his
> > focus is currently exclusively on dev for Cloudstack with cloud.ca.
> >
> > Our 2 current priorities are:
> > -fixing VRs,SVMs to run has HVM VMs in xenserver.
> > - redesign, or rewrite, the remote management vpn for vpc, poc in
> progress
> > for IKEv2...
> >
> >
> >
> > Some of you might have interact with him already.
> >
> >
> > Also, we are going to be more active for the upcomming 4.12 release.
> >
> >
> > Cheers!
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: Potential backward incompatibility problem in building SystemVM

[Khosrow Moossavi]

Fair enough. I modified our Jenkins job to differentiate between versions.
I might open a PR to emphasis this in README, to prevent further confusion.

Thanks





On Mon, Feb 19, 2018 at 4:35 AM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Khosrow,
>
>
> Your argument about the ability to have a given name be used as the final
> artifact name is not correct for prior 4.11 versions, as that only was a
> specific case/condition for systemvm template to copy/rename and then use
> an existing definition, and not with rest of the veewee definitions that
> existed in the folder.
>
>
> Even if the name of the folder was systemvm64template, you build job may
> still fail due to the build process and tool/environment changes. Finally,
> you can always rename the build artifacts. The issue IMO is with your build
> job and not the current build scripts.
>
>
> The README file already mentions what arguments can be used to build
> templates but can indeed get some improvements:
>
> https://github.com/apache/cloudstack/blob/master/tools/appliance/README.md
>
>
> Both your suggestions are okay with me, you may improve the README or send
> a PR that modifies the build.sh script to handle exporting appliances to
> custom name (but as a general option, not specific to systemvmtemplate).
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Khosrow Moossavi <kmooss...@cloudops.com>
> Sent: Monday, February 19, 2018 3:07:41 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Potential backward incompatibility problem in building
> SystemVM
>
> Daan, Rohit
>
> With the new packer build (4.11+) one cannot give "blah" to be the final
> name of the template.
> The script will look for a folder called "blah" in appliance folder which
> does not exist. But in before
> packer (prior to 4.11) one can give "blah" to be the final name, because
> the script would copy
> "definition" to "blah" folder and continue the script.
>
> In our own case, for instance, we needed to change the Jenkins script
> because it was failing with
> "systemvm64template" as the name on 4.11.0.1.
>
> So I guess my point is either 1) we need to change the script to still
> handle custom name, 2) have
> this documented somewhere (applicane/README may be) that the only accepted
> name will be
> "systemvmtemplate".
>
> Minor change either way.
>
>
>
> On Sat, Feb 17, 2018 at 2:30 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
> wrote:
>
> > Khosrow,
> >
> >
> > The name 'systemvmtemplate' refers to the name of the folder, the
> build.sh
> > script now accepts a folder that has the packer definitions such as the
> > built-in one or any other future packer based templates. The systemvm
> > template's file name is never used for compatibilities sake, one can
> choose
> > whatever name they want and they will be used okay as long as that name
> is
> > correctly configured in the global settings. I don't understand the bit
> > about name/compatbility.
> >
> >
> > Historically, we used to a 32bit template with its definition defined in
> > systemvmtemplate and then we moved to 64-bit template with introduction
> of
> > definitions in systemvm64template folder, and when we did that we added
> > that constraint to remove and rename folders while are not needed/useful
> > anymore.
> >
> >
> > Wrt building it's not backward compatible as well, the build process has
> > been changed from virtualbox+veewee/ruby based to packer+qemu/kvm based
> so
> > the old script/jobs are broken as well.
> >
> >
> > - Rohit
> >
> > <https://cloudstack.apache.org>
> >
> >
> >
> > 
> > From: Khosrow Moossavi <kmooss...@cloudops.com>
> > Sent: Friday, February 16, 2018 5:58:59 PM
> > To: dev@cloudstack.apache.org
> > Subject: Potential backward incompatibility problem in building SystemVM
> >
> > Hi
> >
> > I just noticed that the changes [1] in tools/applince/build.sh may break
> > backward compatibility
> > of the building process of systemvmtremplate.
> >
> > In the highlighted (and now removed) line, we used to have a predefined
> > name as "systemvm64template"
> > and if one still wants to execute "build.sh systemvm64template ..." (or
> any
> > other name they
> > want) the build will break (becauase of the now new if condition).
> >
> > Was this intentional to always have a new

Re: Potential backward incompatibility problem in building SystemVM

[Khosrow Moossavi]

Daan, Rohit

With the new packer build (4.11+) one cannot give "blah" to be the final
name of the template.
The script will look for a folder called "blah" in appliance folder which
does not exist. But in before
packer (prior to 4.11) one can give "blah" to be the final name, because
the script would copy
"definition" to "blah" folder and continue the script.

In our own case, for instance, we needed to change the Jenkins script
because it was failing with
"systemvm64template" as the name on 4.11.0.1.

So I guess my point is either 1) we need to change the script to still
handle custom name, 2) have
this documented somewhere (applicane/README may be) that the only accepted
name will be
"systemvmtemplate".

Minor change either way.



On Sat, Feb 17, 2018 at 2:30 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Khosrow,
>
>
> The name 'systemvmtemplate' refers to the name of the folder, the build.sh
> script now accepts a folder that has the packer definitions such as the
> built-in one or any other future packer based templates. The systemvm
> template's file name is never used for compatibilities sake, one can choose
> whatever name they want and they will be used okay as long as that name is
> correctly configured in the global settings. I don't understand the bit
> about name/compatbility.
>
>
> Historically, we used to a 32bit template with its definition defined in
> systemvmtemplate and then we moved to 64-bit template with introduction of
> definitions in systemvm64template folder, and when we did that we added
> that constraint to remove and rename folders while are not needed/useful
> anymore.
>
>
> Wrt building it's not backward compatible as well, the build process has
> been changed from virtualbox+veewee/ruby based to packer+qemu/kvm based so
> the old script/jobs are broken as well.
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Khosrow Moossavi <kmooss...@cloudops.com>
> Sent: Friday, February 16, 2018 5:58:59 PM
> To: dev@cloudstack.apache.org
> Subject: Potential backward incompatibility problem in building SystemVM
>
> Hi
>
> I just noticed that the changes [1] in tools/applince/build.sh may break
> backward compatibility
> of the building process of systemvmtremplate.
>
> In the highlighted (and now removed) line, we used to have a predefined
> name as "systemvm64template"
> and if one still wants to execute "build.sh systemvm64template ..." (or any
> other name they
> want) the build will break (becauase of the now new if condition).
>
> Was this intentional to always have a new "systemvmtemplate" as the name or
> the new if
> condition should be fixed? Super simple to fix anyway.
>
> if [ "systemvmtemplate" != "${appliance_build_name}" ]; then
>
> instead of:
>
> if [ "${appliance}" != "${appliance_build_name}" ]; then
>
>
> [1]
> https://github.com/apache/cloudstack/commit/3839239a21fc14a64acc18900ae303
> 961036ef91#diff-68ae31f5f30dae8f541e26b8acbd75eeL247
>
> Khosrow Moossavi
> CloudOps
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Potential backward incompatibility problem in building SystemVM

[Khosrow Moossavi]

Hi

I just noticed that the changes [1] in tools/applince/build.sh may break
backward compatibility
of the building process of systemvmtremplate.

In the highlighted (and now removed) line, we used to have a predefined
name as "systemvm64template"
and if one still wants to execute "build.sh systemvm64template ..." (or any
other name they
want) the build will break (becauase of the now new if condition).

Was this intentional to always have a new "systemvmtemplate" as the name or
the new if
condition should be fixed? Super simple to fix anyway.

if [ "systemvmtemplate" != "${appliance_build_name}" ]; then

instead of:

if [ "${appliance}" != "${appliance_build_name}" ]; then


[1]
https://github.com/apache/cloudstack/commit/3839239a21fc14a64acc18900ae303961036ef91#diff-68ae31f5f30dae8f541e26b8acbd75eeL247

Khosrow Moossavi
CloudOps

Re: [VOTE] Apache Cloudstack 4.11.0.0 LTS [RC2]

[Khosrow Moossavi]

Awesome! Good job everyone +1

Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>



On Mon, Feb 5, 2018 at 2:50 PM, Tutkowski, Mike <mike.tutkow...@netapp.com>
wrote:

> Congratulations, everyone!
>
> On 2/5/18, 3:09 AM, "Rohit Yadav" <rohit.ya...@shapeblue.com> wrote:
>
> Copy/paste error, the vote count was:
>
>
> +1 (PMC / binding)
> 4 person (Mike, Daan, Wido, Rohit)
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Rohit Yadav <ro...@apache.org>
> Sent: Monday, February 5, 2018 10:57:17 AM
> To: dev@cloudstack.apache.org
> Cc: us...@cloudstack.apache.org
> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 LTS [RC2]
>
> Hi All,
>
> The vote for CloudStack 4.11.0.0 *passes* with 4 PMC + 2 non-PMC votes.
>
> +1 (PMC / binding)
> 2 person (Mike, Daan, Wido, Rohit)
>
> +1 (non binding)
> 2 person (Lucian, Boris)
>
> 0
> none
>
> -1
> none
>
> Thanks to everyone participating.
>
> I will now prepare the release announcement to go out after 48 hours to
> give the mirrors time to catch up.
>
> Regards.
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On Fri, Jan 26, 2018 at 1:19 PM, Rohit Yadav <ro...@apache.org> wrote:
>
> > Hi All,
> >
> > I've created a 4.11.0.0 release (RC2), with the following artifacts
> up for
> > testing and a vote:
> >
> > Git Branch and Commit SH:
> > https://gitbox.apache.org/repos/asf?p=cloudstack.git;a=
> > shortlog;h=refs/heads/4.11.0.0-RC20180126T1313
> > Commit: 5dada1f7ed5fb6a8ee261c763f744583e586f8bf
> >
> > Source release (checksums and signatures are available at the same
> > location):
> > https://dist.apache.org/repos/dist/dev/cloudstack/4.11.0.0/
> >
> > PGP release keys (signed using 5ED1E1122DC5E8A4A45112C2484248
> 210EE3D884):
> > https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> >
> > The vote will be open till end of next week, 2nd Feb 2018.
> >
> > For sanity in tallying the vote, can PMC members please be sure to
> > indicate "(binding)" with their vote?
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> > Additional information:
> >
> > For users' convenience, I've built packages from
> > 5dada1f7ed5fb6a8ee261c763f744583e586f8bf and published RC2
> repository
> > here:
> > http://cloudstack.apt-get.eu/testing/4.11-rc2
> > (the packages are being built and will be available shortly)
> >
> > The release notes are still work-in-progress, but the
> systemvmtemplate
> > upgrade section has been updated. You may refer the following for
> > systemvmtemplate upgrade testing:
> > http://docs.cloudstack.apache.org/projects/cloudstack-
> > release-notes/en/latest/index.html
> >
> > 4.11 systemvmtemplates are available from here:
> > https://download.cloudstack.org/systemvm/4.11/
> >
> > Regards,
> > Rohit Yadav
> >
>
>
>

Enhance package.sh script

[Khosrow Moossavi]

Hi community

I fixed and enhanced package.sh script in PR #2433 [1]. It's based on 4.11
branch
and since it's only touches packaging script, it should be safe to be
included in
4.11.0.0 and/or 4.11.1.0.

The idea was to be able to "brand" the final package name to distinguish
the custom
built packages from community build ones just by looking at the file name.

If POM version contains branding string (x.y.z.a-NAME[-SNAPSHOT]) it will
generate
packages like: cloudstack-package-x.y.z.a-NAME.NUMBER.el7.centos.x86_64

[1] https://github.com/apache/cloudstack/pull/2433

Khosrow Moossavi
CloudOps

Re: Squeeze another PR (#2398) in 4.11 milestone

[Khosrow Moossavi]

Rafael,

It got changed on this PR:
https://github.com/apache/cloudstack/pull/1749/files#diff-6eeb1a2fb818cccb14785ee80c93a561R560



On Tue, Jan 9, 2018 at 4:44 PM, Khosrow Moossavi <kmooss...@cloudops.com>
wrote:

> That is correct Mike. The quoted part above was misleading, it should have
> been "at any given point in time *when transaction finished*"
> Removal of "other" or "current failed" snapshot happens at the very end of
> the method. The state of SR throughout time would be something like:
>
> 1) snapshot-01 (at rest)
> 2) snapshot-01, snapshot-02 (while taking snapshot-02 on primary storage
> and sending to secondary storage)
> 3) snapshot-02 (at rest again, after successful)
> OR
> 3) snapshot-01 (at rest again, after failure)
>
>
> Khosrow Moossavi
>
> Cloud Infrastructure Developer
>
> t 514.447.3456 <(514)%20447-3456>
>
> <https://goo.gl/NYZ8KK>
>
>
>
> On Tue, Jan 9, 2018 at 4:33 PM, Tutkowski, Mike <mike.tutkow...@netapp.com
> > wrote:
>
>> “technically we should only have "one" on primary storage at any given
>> point in time”
>>
>> I just wanted to follow up on this one.
>>
>> When we are copying a delta from the previous snapshot, we should
>> actually have two snapshots on primary storage for a time.
>>
>> If the delta copy is successful, then we delete the older snapshot. If
>> the delta copy fails, then we delete the newest snapshot.
>>
>> Is that correct?
>>
>> > On Jan 9, 2018, at 11:36 AM, Khosrow Moossavi <kmooss...@cloudops.com>
>> wrote:
>> >
>> > "We are already deleting snapshots in the primary storage, but we always
>> > leave behind the last one"
>> >
>> > This issue doesn't happen only when something fails. We are not
>> deleting the
>> > snapshots from primary storage (not on XenServer 6.25+ and not since Feb
>> > 2017)
>> >
>> > The fix of this PR is:
>> >
>> > 1) when transferred successfully to secondary storage everything except
>> > "this"
>> > snapshot get removed (technically we should only have "one" on primary
>> > storage
>> > at any given point in time) [towards the end of try block]
>> > 2) when transferring to secondary storage fails, only "this" in-progress
>> > snapshot
>> > gets deleted. [finally block]
>> >
>> >
>> >
>> > On Tue, Jan 9, 2018 at 1:01 PM, Rafael Weingärtner <
>> > rafaelweingart...@gmail.com> wrote:
>> >
>> >> Khosrow, I have seen this issue as well. It happens when there are
>> problems
>> >> to transfer the snapshot from the primary to the secondary storage.
>> >> However, we need to clarify one thing. We are already deleting
>> snapshots in
>> >> the primary storage, but we always leave behind the last one. The
>> problem
>> >> is that if an error happens, during the transfer of the VHD from the
>> >> primary to the secondary storage. The failed snapshot VDI is left
>> behind in
>> >> primary storage (for XenServer). These failed snapshots can accumulate
>> with
>> >> time and cause the problem you described because XenServer will not be
>> able
>> >> to coalesce the VHD files of the VM. Therefore, what you are
>> addressing in
>> >> this PR are cases when an exception happens during the transfer from
>> >> primary to secondary storage.
>> >>
>> >> On Tue, Jan 9, 2018 at 3:25 PM, Khosrow Moossavi <
>> kmooss...@cloudops.com>
>> >> wrote:
>> >>
>> >>> Hi community
>> >>>
>> >>> We've found [1] and fixed [2] an issue in 4.10 regarding snapshots
>> >>> remaining on primary storage (XenServer + Swift) which causes VDI
>> chain
>> >>> gets full after some time and user cannot take another snapshot.
>> >>>
>> >>> Please include this in 4.11 milestone if you see fit.
>> >>>
>> >>> [1]: https://issues.apache.org/jira/browse/CLOUDSTACK-10222
>> >>> [2]: https://github.com/apache/cloudstack/pull/2398
>> >>>
>> >>> Thanks
>> >>> Khosrow
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Rafael Weingärtner
>> >>
>>
>
>

Re: Squeeze another PR (#2398) in 4.11 milestone

[Khosrow Moossavi]

That is correct Mike. The quoted part above was misleading, it should have
been "at any given point in time *when transaction finished*"
Removal of "other" or "current failed" snapshot happens at the very end of
the method. The state of SR throughout time would be something like:

1) snapshot-01 (at rest)
2) snapshot-01, snapshot-02 (while taking snapshot-02 on primary storage
and sending to secondary storage)
3) snapshot-02 (at rest again, after successful)
OR
3) snapshot-01 (at rest again, after failure)


Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>



On Tue, Jan 9, 2018 at 4:33 PM, Tutkowski, Mike <mike.tutkow...@netapp.com>
wrote:

> “technically we should only have "one" on primary storage at any given
> point in time”
>
> I just wanted to follow up on this one.
>
> When we are copying a delta from the previous snapshot, we should actually
> have two snapshots on primary storage for a time.
>
> If the delta copy is successful, then we delete the older snapshot. If the
> delta copy fails, then we delete the newest snapshot.
>
> Is that correct?
>
> > On Jan 9, 2018, at 11:36 AM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
> >
> > "We are already deleting snapshots in the primary storage, but we always
> > leave behind the last one"
> >
> > This issue doesn't happen only when something fails. We are not deleting
> the
> > snapshots from primary storage (not on XenServer 6.25+ and not since Feb
> > 2017)
> >
> > The fix of this PR is:
> >
> > 1) when transferred successfully to secondary storage everything except
> > "this"
> > snapshot get removed (technically we should only have "one" on primary
> > storage
> > at any given point in time) [towards the end of try block]
> > 2) when transferring to secondary storage fails, only "this" in-progress
> > snapshot
> > gets deleted. [finally block]
> >
> >
> >
> > On Tue, Jan 9, 2018 at 1:01 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> >> Khosrow, I have seen this issue as well. It happens when there are
> problems
> >> to transfer the snapshot from the primary to the secondary storage.
> >> However, we need to clarify one thing. We are already deleting
> snapshots in
> >> the primary storage, but we always leave behind the last one. The
> problem
> >> is that if an error happens, during the transfer of the VHD from the
> >> primary to the secondary storage. The failed snapshot VDI is left
> behind in
> >> primary storage (for XenServer). These failed snapshots can accumulate
> with
> >> time and cause the problem you described because XenServer will not be
> able
> >> to coalesce the VHD files of the VM. Therefore, what you are addressing
> in
> >> this PR are cases when an exception happens during the transfer from
> >> primary to secondary storage.
> >>
> >> On Tue, Jan 9, 2018 at 3:25 PM, Khosrow Moossavi <
> kmooss...@cloudops.com>
> >> wrote:
> >>
> >>> Hi community
> >>>
> >>> We've found [1] and fixed [2] an issue in 4.10 regarding snapshots
> >>> remaining on primary storage (XenServer + Swift) which causes VDI chain
> >>> gets full after some time and user cannot take another snapshot.
> >>>
> >>> Please include this in 4.11 milestone if you see fit.
> >>>
> >>> [1]: https://issues.apache.org/jira/browse/CLOUDSTACK-10222
> >>> [2]: https://github.com/apache/cloudstack/pull/2398
> >>>
> >>> Thanks
> >>> Khosrow
> >>>
> >>
> >>
> >>
> >> --
> >> Rafael Weingärtner
> >>
>

Re: Squeeze another PR (#2398) in 4.11 milestone

[Khosrow Moossavi]

"We are already deleting snapshots in the primary storage, but we always
leave behind the last one"

This issue doesn't happen only when something fails. We are not deleting the
snapshots from primary storage (not on XenServer 6.25+ and not since Feb
2017)

The fix of this PR is:

1) when transferred successfully to secondary storage everything except
"this"
snapshot get removed (technically we should only have "one" on primary
storage
at any given point in time) [towards the end of try block]
2) when transferring to secondary storage fails, only "this" in-progress
snapshot
gets deleted. [finally block]



On Tue, Jan 9, 2018 at 1:01 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Khosrow, I have seen this issue as well. It happens when there are problems
> to transfer the snapshot from the primary to the secondary storage.
> However, we need to clarify one thing. We are already deleting snapshots in
> the primary storage, but we always leave behind the last one. The problem
> is that if an error happens, during the transfer of the VHD from the
> primary to the secondary storage. The failed snapshot VDI is left behind in
> primary storage (for XenServer). These failed snapshots can accumulate with
> time and cause the problem you described because XenServer will not be able
> to coalesce the VHD files of the VM. Therefore, what you are addressing in
> this PR are cases when an exception happens during the transfer from
> primary to secondary storage.
>
> On Tue, Jan 9, 2018 at 3:25 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > Hi community
> >
> > We've found [1] and fixed [2] an issue in 4.10 regarding snapshots
> > remaining on primary storage (XenServer + Swift) which causes VDI chain
> > gets full after some time and user cannot take another snapshot.
> >
> > Please include this in 4.11 milestone if you see fit.
> >
> > [1]: https://issues.apache.org/jira/browse/CLOUDSTACK-10222
> > [2]: https://github.com/apache/cloudstack/pull/2398
> >
> > Thanks
> > Khosrow
> >
>
>
>
> --
> Rafael Weingärtner
>

Squeeze another PR (#2398) in 4.11 milestone

[Khosrow Moossavi]

Hi community

We've found [1] and fixed [2] an issue in 4.10 regarding snapshots
remaining on primary storage (XenServer + Swift) which causes VDI chain
gets full after some time and user cannot take another snapshot.

Please include this in 4.11 milestone if you see fit.

[1]: https://issues.apache.org/jira/browse/CLOUDSTACK-10222
[2]: https://github.com/apache/cloudstack/pull/2398

Thanks
Khosrow

Re: [DISCUSS] Freezing master for 4.11

[Khosrow Moossavi]

+1 Daan and Rohit

Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>



On Mon, Jan 8, 2018 at 7:58 AM, Boris Stoyanov <boris.stoya...@shapeblue.com
> wrote:

> Yes let’s do that.
>
>
> boris.stoya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> > On 8 Jan 2018, at 14:41, Wido den Hollander <w...@widodh.nl> wrote:
> >
> >
> >
> > On 01/08/2018 01:30 PM, Rohit Yadav wrote:
> >> All,
> >> Thank you everyone for your feedback. Given we're in agreement with
> Daan's proposal, I'll summarize and add strategy:
> >> - We'll freeze the 4.11 milestone to only these 9 PRs:
> https://github.com/apache/cloudstack/milestone/3
> >> - In addition, only blocker, test fixes, and release/packaging related
> fixes may be accepted in master now.
> >> - If we don't hear from authors within a day, the PRs may be removed
> from the milestone.
> >> - We'll re-assess the list again by EOD, Wed 10 Jan 2018.
> >> Does this look agreeable? Thanks.
> >
> > Yes, it does! When the first RC comes out we should be able to run
> tests, fix what's broken and focus in the release and 4.12 afterwards.
> >
> > Wido
> >
> >> - Rohit
> >> <https://cloudstack.apache.org>
> >> 
> >> From: Daan Hoogland <daan.hoogl...@gmail.com>
> >> Sent: Monday, January 8, 2018 4:21:00 PM
> >> To: dev
> >> Subject: Re: [DISCUSS] Freezing master for 4.11
> >> slow display of arrogance in reacting this time ; yeeaahhh
> >> On Mon, Jan 8, 2018 at 11:22 AM, Voloshanenko Igor <
> >> igor.voloshane...@gmail.com> wrote:
> >>> You again faster than me )))
> >>>
> >>> 2018-01-08 12:21 GMT+02:00 Voloshanenko Igor <
> igor.voloshane...@gmail.com
> >>>> :
> >>>
> >>>> :D tnx )
> >>>> Updated by my colleague already
> >>>>
> >>>> 2018-01-08 12:06 GMT+02:00 Daan Hoogland <daan.hoogl...@gmail.com>:
> >>>>
> >>>>> yeah, way ahead of you Igor ;) I asked a question about it
> >>>>>
> >>>>> On Mon, Jan 8, 2018 at 11:05 AM, Voloshanenko Igor <
> >>>>> igor.voloshane...@gmail.com> wrote:
> >>>>>
> >>>>>> Updates posted to https://github.com/apache/cloudstack/pull/2389
> >>>>>> Can you please review?
> >>>>>>
> >>>>>> 2018-01-08 11:57 GMT+02:00 Voloshanenko Igor <
> >>>>> igor.voloshane...@gmail.com
> >>>>>>> :
> >>>>>>
> >>>>>>> Sure. Got it.
> >>>>>>>
> >>>>>>> Will post update soon
> >>>>>>>
> >>>>>>> 2018-01-08 11:38 GMT+02:00 Daan Hoogland <daan.hoogl...@gmail.com
> >:
> >>>>>>>
> >>>>>>>> Igor, I remember your PR and think it is fine. It can also be
> >>> argued
> >>>>>> that
> >>>>>>>> it needs to go in as a security feature. For an RM it is
> >>> unthinkably
> >>>>>> late,
> >>>>>>>> but fortunately it is very small. I will however -1 it if it leads
> >>>>> to a
> >>>>>>>> plethora of last minute PRs to include.
> >>>>>>>>
> >>>>>>>> On Mon, Jan 8, 2018 at 10:33 AM, Voloshanenko Igor <
> >>>>>>>> igor.voloshane...@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Guys, can we please include https://github.com/apache/clou
> >>>>>>>> dstack/pull/2389
> >>>>>>>>> into 4.11
> >>>>>>>>> PR very small and updates will be published in next few hours.
> >>>>>>>>>
> >>>>>>>>> As we have this for a while in production for 4.8 branch.
> >>>>>>>>>
> >>>>>>>>> 2018-01-08 11:15 GMT+02:00 Boris Stoyanov <
> >>>>>> boris.stoya...@shapeblue.com
> >>>>>>>>> :
> >>>>>>>>>
> >>>>>>>>>> +1 Daan
> >>>>>>>>>>
> >>>>>>>>>>
>

Re: [VOTE] Clean up old and obsolete branches.

[Khosrow Moossavi]

+1

Khosrow Moossavi
CloudOps

On Jan 2, 2018 14:07, "Nicolas Vazquez" <nicolas.vazq...@shapeblue.com>
wrote:

> +1
>
> 
> From: Simon Weller <swel...@ena.com.INVALID>
> Sent: Tuesday, January 2, 2018 3:38:00 PM
> To: dev
> Subject: Re: [VOTE] Clean up old and obsolete branches.
>
> +0
>
> 
> From: Daan Hoogland <daan.hoogl...@gmail.com>
> Sent: Tuesday, January 2, 2018 12:19 PM
> To: dev
> Subject: Re: [VOTE] Clean up old and obsolete branches.
>
> 0
>
> On Tue, Jan 2, 2018 at 1:51 PM, Gabriel Beims Bräscher <
> gabrasc...@gmail.com
> > wrote:
>
> > +1
> >
> > 2018-01-02 9:46 GMT-02:00 Rafael Weingärtner <
> rafaelweingart...@gmail.com
> > >:
> >
> > > Hope you guys had great holy days!
> > >
> > > Resuming the discussion we started last year in [1]. It is time to vote
> > and
> > > then to push (if the vote is successful) the protocol defined to our
> > wiki.
> > > Later we can start enforcing it.
> > > I will summarize the protocol for branches in the official repository.
> > >
> > >1. We only maintain the master and major release branches. We
> > currently
> > >have a system of X.Y.Z.S. I define major release here as a release
> > that
> > >changes either ((X or Y) or (X and Y));
> > >2. We will use tags for versioning. Therefore, all versions we
> release
> > >are tagged accordingly, including minor and security releases;
> > >3. When releasing the “SNAPSHOT” is removed and the branch of the
> > >version is created (if the version is being cut from master). Rule
> (1)
> > > one
> > >is applied here; therefore, only major releases will receive
> branches.
> > >Every release must have a tag according to the format X.Y.Z.S. After
> > >releasing, we bump the POM of the version to next available
> SNAPSHOT;
> > >4. If there's a need to fix an old version, we work on HEAD of
> > >corresponding release branch. For instance, if we want to fix
> > something
> > > in
> > >release 4.1.1.0, we will work on branch 4.1, which will have the POM
> > > set to
> > >4.1.2.0-SNAPSHOT;
> > >5. People should avoid (it is not forbidden though) using the
> official
> > >apache repository to store working branches. If we want to work
> > > together on
> > >some issues, we can set up a fork and give permission to interested
> > > parties
> > >(the official repository is restricted to committers). If one uses
> the
> > >official repository, the branch used must be cleaned right after
> > > merging;
> > >6. Branches not following these rules will be removed if they have
> not
> > >received attention (commits) for over 6 (six) months;
> > >7. Before the removal of a branch in the official repository it is
> > >mandatory to create a Jira ticket and send a notification email to
> > >CloudStack’s dev mailing list. If there are no objections, the
> branch
> > > can
> > >be deleted seven (7) business days after the notification email is
> > sent;
> > >8. After the branch removal, the Jira ticket must be closed.
> > >
> > > Let’s go to the poll:
> > > (+1) – I want to work using this protocol
> > > (0) – Indifferent to me
> > > (-1) – I prefer the way it is not, without any protocol/guidelines
> > >
> > >
> > > [1]
> > > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> > > 201711.mbox/%3CCAHGRR8ozDBX%3DJJewLz_cu-YP9vA3TEmesvxGArTDBPerAOj8Cw%
> > > 40mail.gmail.com%3E
> > >
> > > --
> > > Rafael Weingärtner
> > >
> >
>
>
>
> --
> Daan
>
> nicolas.vazq...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>
>

Re: [DISCUSS] Changing events to include UUIDs, could it break your integration

[Khosrow Moossavi]

+1 on API versioning and make it really RESTful (cherry on top).
I'm willing to participate on this change if voted for.

Khosrow Moossavi
CloudOps



On Thu, Dec 28, 2017 at 11:07 AM, Rene Moser <m...@renemoser.net> wrote:

> Hi
>
> On 12/28/2017 10:52 AM, Rohit Yadav wrote:
> > All,
> >
> >
> > We've come across a pull request which changes the event description to
> use/export UUIDs instead of the numeric internal ID of a resource. I'm not
> sure if this could potentially break any external integration such as
> billing, crms etc. so wanted to get your feedback on this. My understanding
> is external billing/intergrations would consume from the usage related
> tables for data than events table.
> >
> >
> > The PR is https://github.com/apache/cloudstack/pull/1940
> >
> >
> > Comments, thoughts? Thanks.
>
> Even though I am +1 with this change, we should work towards versioning
> the API to prevent breaking anything out there.
>
> René
>

Re: Master Blockers and Criticals

[Khosrow Moossavi]

@Paul you can assign CLOUDSTACK-9862 to me, we already have it fixed in our
own fork.



On Mon, Dec 18, 2017 at 12:05 PM, Paul Angus 
wrote:

> Hi All, here is an updated summary of the open Critical and Blocker Issues
> in Jira.
> If you are working on any of these issues, please whether you believe that
> you will have this issue closed by 8th Jan.
>
> @Jayapal Reddy please respond to the pings on the subject of the blocker
> that you have raised and are working on.
>
> Key PrioritySummary
>  Assignee
> Reporter
> CLOUDSTACK-9885 Blocker VPC RVR: On deleting first tier and configuring
> Private GW both VRs becoming MASTER Jayapal Reddy   Jayapal
> Reddy
> CLOUDSTACK-10127Critical4.9 / 4.10 KVM + openvswitch + vpc
> + static nat / secondary ip on eth2? Frank Maximus
>  Sven Vogel
> CLOUDSTACK-9892 CriticalPrimary storage resource check is broken
> when using root disk size override to deploy VMKoushik Das
>  Koushik Das
> CLOUDSTACK-9862 Criticallist template with id= no longer work as
> domain admin   Unassigned
> Pierre-Luc Dion
> CLOUDSTACK-10128CriticalTemplate from snapshot not merging
> vhd filesRafael
> Weingärtner  Marcelo Lima
> CLOUDSTACK-9964 CriticalSnapahots are getting deleted if VM is
> assigned to another user Pavan Kumar
> Aravapalli  Pavan Kumar Aravapalli
> CLOUDSTACK-9855 CriticalVPC RVR when master guest interface is
> down backup VR is not switched to Master Jayapal Reddy   Jayapal
> Reddy
> CLOUDSTACK-9837 CriticalUpon stoping and starting an InternalLbVM
> device from CloudStack, HAProxy service on the device is not being started
> properly
>
>   Unassigned  Mani
> Prashanth Varma Manthena
>
> project = CLOUDSTACK AND issuetype = Bug AND status in (Open, "In
> Progress", Reopened) AND priority in (Blocker, Critical) AND
> affectedVersion in (4.10.0.0, 4.10.1.0, 4.11.0.0, Future) ORDER BY priority
> DESC, updated DESC
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Paul Angus [mailto:paul.an...@shapeblue.com]
> Sent: 11 December 2017 11:00
> To: dev@cloudstack.apache.org
> Cc: Boris Stoyanov ; Rohit Yadav <
> rohit.ya...@shapeblue.com>
> Subject: Master Blockers and Criticals
>
> Hi All,
> Please find a summary - of open critical and blocker bugs in 4.11 If you
> know of one which is missing please update Jira accordingly.  I will chase
> Assignees individually (via ML) to get status updates...
>
>
> KeySummary
>
>
>  Priority Assignee  Reporter
> CLOUDSTACK-9885  VPC RVR: On deleting first tier and configuring
> Private GW both VRs becoming MASTER   Blocker Jayapal
> Reddy   Jayapal Reddy
> CLOUDSTACK-10164   UI - not able to create a VPC
>
>   Blocker Sigert Goeminne
> Sigert Goeminne
> CLOUDSTACK-10127   4.9 / 4.10 KVM + openvswitch + vpc + static nat /
> secondary ip on eth2?
> Critical   Frank Maximus Sven Vogel
> CLOUDSTACK-9892  Primary storage resource check is broken when
> using root disk size override to deploy VMCritical
>  Koushik DasKoushik Das
> CLOUDSTACK-10140   When template is created from snapshot
> template.properties are corrupted
> Critical   UnassignedIvan Kudryavtsev
> CLOUDSTACK-10128   Template from snapshot not merging vhd files
>
> Critical   UnassignedMarcelo Lima
> CLOUDSTACK-9964  Snapahots are getting deleted if VM is assigned
> to another user
>  Critical   Pavan Kumar Aravapalli Pavan Kumar Aravapalli
> CLOUDSTACK-9855  VPC RVR when master guest interface is down
> backup VR is not switched to Master  Critical
>  Jayapal Reddy   Jayapal Reddy
> CLOUDSTACK-9862  list template with id= no longer work as domain
> admin
>  Critical   UnassignedPierre-Luc Dion
> CLOUDSTACK-9837  Upon stoping and starting an InternalLbVM device
> from CloudStack, HAProxy service on the device is not being started properly
> Critical   UnassignedMani Prashanth Varma Manthena
>
>
> Kind regards,
>
> Paul Angus
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>

Re: XenServer 7.1 and 7.2

[Khosrow Moossavi]

Apparently XenServer "xen-tools" has been renamed from version 7.0 onward
to "guest-tools".
https://docs.citrix.com/content/dam/docs/en-us/xenserver/xenserver-7-0/downloads/xenserver-7-0-quick-start-guide.pdf
(Section 4.2, point 3)

And this comment:
https://issues.apache.org/jira/browse/CLOUDSTACK-9839?focusedCommentId=16039096=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039096

I'm gonna open a PR today.



On Mon, Dec 18, 2017 at 2:09 PM, Rohit Yadav 
wrote:

> Thanks Paul, the PR has been merged after reviewing and based on
> smoketests. Would you also like to add support for XenServer 7.3?
>
> Regards.
> 
> From: Paul Angus 
> Sent: Wednesday, December 13, 2017 11:39:28 PM
> To: dev
> Cc: Syed Ahmed; Pierre-Luc Dion
> Subject: XenServer 7.1 and 7.2
>
> Hi All,
>
> I’ve raised a PR to add XenServer 7.1 and 7.2 to  CloudStack’s hypervisor
> list and added new OS mappings.
> I’ve also not added deprecated OSes to the new mappings (if that makes
> sense).
>
> https://github.com/apache/cloudstack/pull/2346
>
> Tests pass with PR rebased to master as of a couple of days ago.  Only
> ‘usual’ failures being seen in smoke tests.
>
> Can we get some review-love going please… 
>
>
>
> Kind regards,
>
> Paul Angus
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: Clean up old and obsolete branches

[Khosrow Moossavi]

I agree Erik. I updated the list in CLOUDSTACK-10169 with more information
(last updated, last commit, HEAD on master and PR status/number) to give us
more immediate visibility of the status of those branches. So any branches
can
be deleted if:

- which its HEAD exists on master
- its PR was merged or closed (which surprisingly are not so many)
- it's old (last updated in 2015 or before?)

and the rest of them can be deleted after more examination (if need be).


On Mon, Dec 4, 2017 at 6:37 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> I thought someone might bring that up. The problem with using branches in
> the official repo is that only committers will be able to commit there. So,
> we would restrict the group of people that might be able to participate in
> this type of cooperation. I do not see the difficulty for a
> contributor/committer to give permissions for others in their own
> repository that is a fork from our official one. I have done that with some
> friends before.
>
> Also, do not worry Erik; the idea is not to delete anything right away. We
> are only bringing the topic for a broader discussion here.
>
> On Mon, Dec 4, 2017 at 8:58 AM, Erik Weber <terbol...@gmail.com> wrote:
>
> > On Thu, Nov 30, 2017 at 9:05 PM, Khosrow Moossavi
> > <kmooss...@cloudops.com> wrote:
> > > Hi Community
> > >
> > > I would like to start the discussion around deleting old and obsolete
> > > branches on github repository. This will help newcomers (including
> > myself)
> > > to keep track of which branches are important and which are not. And
> > since
> > > almost everyone's working on their own forks there is no need to keep
> > > feature/bugfix/hotfix branches around in the main official repository.
> > >
> > > I've created an issue which contains full list of branches in
> > > GH/apache/cloudstack repo as of time of writing this email and the
> > > proposition of which one of them can be deleted.
> > >
> > > https://issues.apache.org/jira/browse/CLOUDSTACK-10169
> > >
> > > I would appreciate your questions, comments, suggestions.
> >
> > Do note that there might be branches with unmerged changes, I don't
> > think we should just automatically delete those without reflecting
> > over its content.
> > Although these branch might be stray now, there could be pieces there
> > that someone else could use at a later point.
> >
> > As for old feature/fix branches that has been merged, I'm +1 to
> > cleanup up those.
> >
> > --
> > Erik
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: Clean up old and obsolete branches

[Khosrow Moossavi]

@Daan That's a good point, I'll try to update the list to
CLOUDSTACK- wherever applicable. But the majority of branches are
4.1.x to 4.6.x, which might be able to be cleaned up easily.

- I feel we might be able to delete everything 4.1.x, 4.2.x, 4.3.x, 4.4.x,
4.5.x, 4.6.x (almost blindly).
- the only branches of 4.7.x are the RCs (should be safe to be deleted)
- the only branches of 4.8.x are the RCs (should be safe to be deleted)
- the only branches of 4.9.x are the RCs and 3 develop branches (should be
safe to be deleted)
- there are bunch of CID- which I don't know what they are. There
are no corresponding CLOUDSTACK tickets for those number. (might be safe to
be deleted)

@Rafael I agree which this approach. We can have master and release
branches with names as "major.minor.micro.x" (e.g. 4.11.0.x) in which their
HEAD's pom version always have SNAPSHOT (e.g. 4.11.0.1-SNAPSHOT) and on
releasing:

- remove the SNAPSHOT from pom
- tag it (with full qualified pom version)
- bump pom version on the branch to next available SNAPSHOT

and if there's a need to fix on older releases, one can either 1) create a
branch out of that tag 2) fix on HEAD of corresponding release branch. (I,
personally, like the second approach better)


Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>



On Fri, Dec 1, 2017 at 5:05 AM, Daan Hoogland <daan.hoogl...@gmail.com>
wrote:

> also I think we can tolerate collective work on our repo. Not everything
> has to go on forks.
>
> On Fri, Dec 1, 2017 at 11:04 AM, Daan Hoogland <daan.hoogl...@gmail.com>
> wrote:
>
> > Rafael, I don't think that works. the versions in the pom.xml files are
> > updated to non snapshot versions on per release mini branches. I like the
> > principle but be carefull not to remove the GA branches.
> >
> > On Fri, Dec 1, 2017 at 10:41 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> >> Thanks for the initiative and the hard worki Khosrow!
> >>
> >> In my opinion, we should only maintain the master and major release
> >> branches. Then, for minor versions, we can keep track of them using
> tags.
> >> There is no need to have things such as GA-4.4.1, GA-4.4.2, and so
> forth.
> >> Instead, we should keep only the branch 4.4, and the minor versions are
> >> built on top of that branch (the branch would always have the top minor
> >> version of the major version). The versioning is done using tags, and
> not
> >> branches. Moreover, people should not use the official apache repository
> >> to
> >> store working branches. Working branches should be kept at the
> developer’s
> >> personal repository on Github.
> >>
> >> To the initial list, I would also remove things such as GA-4.4.1,
> >> GA-4.4.2,
> >> and so on. As I said, we only need on branch per major release. The
> >> versioning is executed through tags, and fixes on past releases should
> be
> >> done in the branch of the release. Also, there are things like
> >> “add_XS_71_72”, “cloudearlyinit”, “new-location”, and
> >> “debian9-systemvmtemplate”; none of them should be there. They are
> working
> >> branch from contributors/committers. These branches can be at their own
> >> personal forks.
> >>
> >> On Fri, Dec 1, 2017 at 4:16 AM, Daan Hoogland <daan.hoogl...@gmail.com>
> >> wrote:
> >>
> >> > thanks for that list Khosrow,  Also very usefull for cleaning people
> to
> >> > clean their own fork.
> >> > I think you can start with the lowest pom versions but I changed one
> >> > because the referred ticket isn't closed. It's my own and I'll have a
> >> look
> >> > later today. For a lot of the branches the ticket aren't clear because
> >> only
> >> >  or CS- is in the titel. Only when
> >> CLOUDSTACK- >> > number> is in the titel you can see immediately that it is closed by
> the
> >> > automatic strikethrough that happens. just a heads-up.
> >> >
> >> > +1
> >> >
> >> >
> >> > On Fri, Dec 1, 2017 at 2:13 AM, Gabriel Beims Bräscher <
> >> > gabrasc...@gmail.com
> >> > > wrote:
> >> >
> >> > > Thanks for the initiative, Khosrow.
> >> > >
> >> > > +1 on removing obsolete branches.
> >> > >
> >> > > 2017-11-30 18:05 GMT-02:00 Khosrow Moossavi <kmooss...@cloudops.com
> >:
> >> > >
> >> > > > Hi Community
> >> > > >
> >> > > &

Clean up old and obsolete branches

[Khosrow Moossavi]

Hi Community

I would like to start the discussion around deleting old and obsolete
branches on github repository. This will help newcomers (including myself)
to keep track of which branches are important and which are not. And since
almost everyone's working on their own forks there is no need to keep
feature/bugfix/hotfix branches around in the main official repository.

I've created an issue which contains full list of branches in
GH/apache/cloudstack repo as of time of writing this email and the
proposition of which one of them can be deleted.

https://issues.apache.org/jira/browse/CLOUDSTACK-10169

I would appreciate your questions, comments, suggestions.

Thanks

Khosrow Moossavi

Cloud Infrastructure Developer

CloudOps

Re: Using Maven wrapper

[Khosrow Moossavi]

Not at all, I'll open one soon.



On Wed, Nov 15, 2017 at 7:28 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> This seems to be interesting. Would you mind opening a PR?
>
> On Tue, Nov 14, 2017 at 7:53 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > No changes required in pom whatsover. Only the scripts/guides. For
> examaple
> > in:
> >
> > /cloudstack/tools/travis/install.sh
> > /cloudstack/deps/install-non-oss.sh
> > /cloudstack/packaging/package.sh (even lines 63-69 will become useless)
> > ...
> >
> > --
> > Khosrow
> >
> >
> >
> > On Tue, Nov 14, 2017 at 4:33 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > This seems to be something interesting to be used. Do we need to make
> > > changes to our pom.xml files to use it? Or is it only a matter of
> > > protocol/guidelines?
> > >
> > > On Tue, Nov 14, 2017 at 7:06 PM, Khosrow Moossavi <
> > kmooss...@cloudops.com>
> > > wrote:
> > >
> > > > Well basically anywhere (either manual or in scripts/CI) maven is
> > > required
> > > > to do something with ACS.
> > > > Basically using *"./mvnw foo bar -Pbaz" *instead of *"mvn foo bar
> > -Pbaz"*
> > > > in which mvnw is an executable file in the repo not installed in the
> > OS.
> > > >
> > > > Khosrow Moossavi
> > > >
> > > > Cloud Infrastructure Developer
> > > >
> > > > t 514.447.3456
> > > >
> > > > <https://goo.gl/NYZ8KK>
> > > >
> > > >
> > > >
> > > > On Tue, Nov 14, 2017 at 2:48 PM, Rafael Weingärtner <
> > > > rafaelweingart...@gmail.com> wrote:
> > > >
> > > > > You mean, to use this tool to help us managing maven versions in
> ACS'
> > > > > community distributed CIs and devs environments?
> > > > >
> > > > >
> > > > > On Tue, Nov 14, 2017 at 5:15 PM, Khosrow Moossavi <
> > > > kmooss...@cloudops.com>
> > > > > wrote:
> > > > >
> > > > > > Hi
> > > > > >
> > > > > > I'm new to ACS community, so I'm not sure if this topic has
> already
> > > > come
> > > > > up
> > > > > > before.
> > > > > >
> > > > > > I wanted to suggest using Maven Wrapper (
> > > > > > https://github.com/takari/maven-wrapper) and actually check it
> in
> > in
> > > > the
> > > > > > repo rather than relying on existence of it on dev, ci, build,
> > host,
> > > > etc
> > > > > > machine.
> > > > > >
> > > > > > With this we can get portability and consistency between builds.
> > > > > >
> > > > > > Regards
> > > > > > ---
> > > > > > Khosrow
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Rafael Weingärtner
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Rafael Weingärtner
> > >
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: Using Maven wrapper

[Khosrow Moossavi]

No changes required in pom whatsover. Only the scripts/guides. For examaple
in:

/cloudstack/tools/travis/install.sh
/cloudstack/deps/install-non-oss.sh
/cloudstack/packaging/package.sh (even lines 63-69 will become useless)
...

--
Khosrow



On Tue, Nov 14, 2017 at 4:33 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> This seems to be something interesting to be used. Do we need to make
> changes to our pom.xml files to use it? Or is it only a matter of
> protocol/guidelines?
>
> On Tue, Nov 14, 2017 at 7:06 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > Well basically anywhere (either manual or in scripts/CI) maven is
> required
> > to do something with ACS.
> > Basically using *"./mvnw foo bar -Pbaz" *instead of *"mvn foo bar -Pbaz"*
> > in which mvnw is an executable file in the repo not installed in the OS.
> >
> > Khosrow Moossavi
> >
> > Cloud Infrastructure Developer
> >
> > t 514.447.3456
> >
> > <https://goo.gl/NYZ8KK>
> >
> >
> >
> > On Tue, Nov 14, 2017 at 2:48 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > You mean, to use this tool to help us managing maven versions in ACS'
> > > community distributed CIs and devs environments?
> > >
> > >
> > > On Tue, Nov 14, 2017 at 5:15 PM, Khosrow Moossavi <
> > kmooss...@cloudops.com>
> > > wrote:
> > >
> > > > Hi
> > > >
> > > > I'm new to ACS community, so I'm not sure if this topic has already
> > come
> > > up
> > > > before.
> > > >
> > > > I wanted to suggest using Maven Wrapper (
> > > > https://github.com/takari/maven-wrapper) and actually check it in in
> > the
> > > > repo rather than relying on existence of it on dev, ci, build, host,
> > etc
> > > > machine.
> > > >
> > > > With this we can get portability and consistency between builds.
> > > >
> > > > Regards
> > > > ---
> > > > Khosrow
> > > >
> > >
> > >
> > >
> > > --
> > > Rafael Weingärtner
> > >
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: Using Maven wrapper

[Khosrow Moossavi]

Well basically anywhere (either manual or in scripts/CI) maven is required
to do something with ACS.
Basically using *"./mvnw foo bar -Pbaz" *instead of *"mvn foo bar -Pbaz"*
in which mvnw is an executable file in the repo not installed in the OS.

Khosrow Moossavi

Cloud Infrastructure Developer

t 514.447.3456

<https://goo.gl/NYZ8KK>



On Tue, Nov 14, 2017 at 2:48 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> You mean, to use this tool to help us managing maven versions in ACS'
> community distributed CIs and devs environments?
>
>
> On Tue, Nov 14, 2017 at 5:15 PM, Khosrow Moossavi <kmooss...@cloudops.com>
> wrote:
>
> > Hi
> >
> > I'm new to ACS community, so I'm not sure if this topic has already come
> up
> > before.
> >
> > I wanted to suggest using Maven Wrapper (
> > https://github.com/takari/maven-wrapper) and actually check it in in the
> > repo rather than relying on existence of it on dev, ci, build, host, etc
> > machine.
> >
> > With this we can get portability and consistency between builds.
> >
> > Regards
> > ---
> > Khosrow
> >
>
>
>
> --
> Rafael Weingärtner
>

Using Maven wrapper

[Khosrow Moossavi]

Hi

I'm new to ACS community, so I'm not sure if this topic has already come up
before.

I wanted to suggest using Maven Wrapper (
https://github.com/takari/maven-wrapper) and actually check it in in the
repo rather than relying on existence of it on dev, ci, build, host, etc
machine.

With this we can get portability and consistency between builds.

Regards
---
Khosrow