Re: [Owncloud] External (git) code repositories
Hi Bart, I think we have to look at different scenarios here to see what is best: - Developers I think it is a very bad idea if the 3rdparty libraries change automatically or semi automatically. As we all know the libraries are not as stable between versions as they should be so this makes development very unpredictable. This would even mean that different developers have different versions because they update their git externals or composer at different point in times. This makes development super difficult. We should update the 3rdparty libraries at the beginning of every development cycle centrally and then stick to them unless there is an important bugfix/security update and all developers are fine with updating and everything is working with the new version. Developers can code agains a defined stable version and can be sure that things doesn't break behind their back. - Testing I think we always should test against the same version of a library for our continuos integration testing. Otherwise we never know why something breaks. We already have a lot of challenges with testing against different browsers, different webservers and configurations, different PHP versions, installed modules and operating systems. If the 3rdparty libraries are also in flux then it gets very difficult IMHO. - Release of the tar release. We want to have a single defined release tar file that contains a tested set of 3rdparty libraries that went through a proper CI and beta testing phase. For this release we need a tested and verified set of 3rd party libraries anyways so we should stick with the current approach here. - Release of Linux Packages This is a different thing. Some Linux packagers complain that it is currently difficult to make ownCloud use the system 3rd party libraries. For example someone has the jquery package installed in Debian and ownCloud should use it. I'm personally not 100% sure is this is a good idea because of the QA and Testing issues but packagers seems to want it. :-) What we can do here is: We create a packaging-config.php in the config directory. Every time we include a 3rdparty library in ownCloud we check if there is a config variable set that defines a filesystem path to the library (for php) or a url (for js, css and images) and use them instead. So if let's say the Debian packager want's to ship an ownCloud that is using the system jquery than they can: 1. remove jquery from the 3rdparty folder, 2. add a packaging-config.php to the config folder that defines something like $jquerypath='/debianlibs/jquery/jquery.js'; 3. make the jquery package a dependency for the ownCloud package. Not sure if someone is motivated to implement that but this is probably the easiest and most flexible solution to make our packagers more happy. Because of that I´m not a fan of using composer or git sub modules because this would make development, testing, releasing and bugfixing significantly more difficult and we already have enough challenges ;-) So what should we do: - Update all the 3rdpary libaries to the newest versions at the beginning of a development cycle and port all the ownCloud code to it. There is still time to do this now for ownCloud 5. - We should put every 3rd party library into its own folder together with a proper licensing file. Currently it´s a mess. - If someone is motivated then we create a packaging-config.php system as described. Frank On 07.12.2012, at 20:13, Bart Visscher ba...@thisnet.nl wrote: On Tue, Nov 27, 2012 at 10:32:27PM +0100, eMerzh wrote: I'm not a git expert but we also have the possibility to use git sub trees ( it have the advantage of the git tool like submodules and it doesn't require an extra command unlike the submodule) ... aside from that, does every projects use git or have a composer file ? because it can push us to use one or another technique .. Most of the projects that use composer also use git for the repository. The git subtree command looks interesting, but is not available in all distribution and on all platforms. It is now possible to test composer and/or submodules by using the composer-and-submodules branch in the 3rdparty repository. Regards, Bart Regards, eMerzh On Tue, Nov 27, 2012 at 9:50 PM, Bart Visscher ba...@thisnet.nl wrote: Hello all, At the sprint in Berlin I merged the routing branch. After the merge there where some problems with having to download extra repositories. We need to solve this problem in a structural way. The quick fix for the routing was to include the code in our 3rdparty repository. I would like to discuss how to have the external code in 3rdparty. At the moment I suggest to limit this to the large projects: Sabre, Symfony routing component and Doctrine. For these projects we have 2 ways of doing this. The first is to use git submodules, and the second one is to use composer. The positive about git submodules is
Re: [Owncloud] External (git) code repositories
On Thu, Dec 13, 2012 at 12:24:17PM +0100, Frank Karlitschek wrote: Hi Bart, I think we have to look at different scenarios here to see what is best: - Developers I think it is a very bad idea if the 3rdparty libraries change automatically or semi automatically. As we all know the libraries are not as stable between versions as they should be so this makes development very unpredictable. This would even mean that different developers have different versions because they update their git externals or composer at different point in times. This makes development super difficult. We should update the 3rdparty libraries at the beginning of every development cycle centrally and then stick to them unless there is an important bugfix/security update and all developers are fine with updating and everything is working with the new version. Developers can code agains a defined stable version and can be sure that things doesn't break behind their back. - Testing I think we always should test against the same version of a library for our continuos integration testing. Otherwise we never know why something breaks. We already have a lot of challenges with testing against different browsers, different webservers and configurations, different PHP versions, installed modules and operating systems. If the 3rdparty libraries are also in flux then it gets very difficult IMHO. - Release of the tar release. We want to have a single defined release tar file that contains a tested set of 3rdparty libraries that went through a proper CI and beta testing phase. For this release we need a tested and verified set of 3rd party libraries anyways so we should stick with the current approach here. With composer and submodules you can specify the version and the commit you want to use. So we don't need to worry that everyone is using different versions. - Release of Linux Packages This is a different thing. Some Linux packagers complain that it is currently difficult to make ownCloud use the system 3rd party libraries. For example someone has the jquery package installed in Debian and ownCloud should use it. Using composer and/or submodules also helps the packagers, this way they know that we are using the version from upstream without any changes of our own. I'm personally not 100% sure is this is a good idea because of the QA and Testing issues but packagers seems to want it. :-) What we can do here is: We create a packaging-config.php in the config directory. Every time we include a 3rdparty library in ownCloud we check if there is a config variable set that defines a filesystem path to the library (for php) or a url (for js, css and images) and use them instead. So if let's say the Debian packager want's to ship an ownCloud that is using the system jquery than they can: 1. remove jquery from the 3rdparty folder, 2. add a packaging-config.php to the config folder that defines something like $jquerypath='/debianlibs/jquery/jquery.js'; 3. make the jquery package a dependency for the ownCloud package. Not sure if someone is motivated to implement that but this is probably the easiest and most flexible solution to make our packagers more happy. No comment, yet. Because of that I?m not a fan of using composer or git sub modules because this would make development, testing, releasing and bugfixing significantly more difficult and we already have enough challenges ;-) I hope i addressed your worries, if not, let me know. Bart So what should we do: - Update all the 3rdpary libaries to the newest versions at the beginning of a development cycle and port all the ownCloud code to it. There is still time to do this now for ownCloud 5. - We should put every 3rd party library into its own folder together with a proper licensing file. Currently it?s a mess. - If someone is motivated then we create a packaging-config.php system as described. Frank On 07.12.2012, at 20:13, Bart Visscher ba...@thisnet.nl wrote: On Tue, Nov 27, 2012 at 10:32:27PM +0100, eMerzh wrote: I'm not a git expert but we also have the possibility to use git sub trees ( it have the advantage of the git tool like submodules and it doesn't require an extra command unlike the submodule) ... aside from that, does every projects use git or have a composer file ? because it can push us to use one or another technique .. Most of the projects that use composer also use git for the repository. The git subtree command looks interesting, but is not available in all distribution and on all platforms. It is now possible to test composer and/or submodules by using the composer-and-submodules branch in the 3rdparty repository. Regards, Bart Regards, eMerzh On Tue, Nov 27, 2012 at 9:50 PM, Bart Visscher ba...@thisnet.nl wrote: Hello all, At the sprint in Berlin I merged the
Re: [Owncloud] External (git) code repositories
On Tue, Nov 27, 2012 at 10:32:27PM +0100, eMerzh wrote: I'm not a git expert but we also have the possibility to use git sub trees ( it have the advantage of the git tool like submodules and it doesn't require an extra command unlike the submodule) ... aside from that, does every projects use git or have a composer file ? because it can push us to use one or another technique .. Most of the projects that use composer also use git for the repository. The git subtree command looks interesting, but is not available in all distribution and on all platforms. It is now possible to test composer and/or submodules by using the composer-and-submodules branch in the 3rdparty repository. Regards, Bart Regards, eMerzh On Tue, Nov 27, 2012 at 9:50 PM, Bart Visscher ba...@thisnet.nl wrote: Hello all, At the sprint in Berlin I merged the routing branch. After the merge there where some problems with having to download extra repositories. We need to solve this problem in a structural way. The quick fix for the routing was to include the code in our 3rdparty repository. I would like to discuss how to have the external code in 3rdparty. At the moment I suggest to limit this to the large projects: Sabre, Symfony routing component and Doctrine. For these projects we have 2 ways of doing this. The first is to use git submodules, and the second one is to use composer. The positive about git submodules is that you don't need extra programs to download the code, every thing you need is included with the git installation. There are some posts on the internet that this doesn't work very well with switching branches, but I'm not sure how relevant that is for us. With composer you don't need git to get the external code, but can download a php phar and use that to download the other code. With composer we can also specify the version in a flexible way. We can also do a combination of these, then everyone can chose which works better for them. Haven't really tested this, but i expect that this works. The commands to get the external code are: git submodules: git submodule init git submodule update composer: curl -s https://getcomposer.org/installer | php composer update Bart ___ Owncloud mailing list Owncloud@kde.org https://mail.kde.org/mailman/listinfo/owncloud ___ Owncloud mailing list Owncloud@kde.org https://mail.kde.org/mailman/listinfo/owncloud ___ Owncloud mailing list Owncloud@kde.org https://mail.kde.org/mailman/listinfo/owncloud
Re: [Owncloud] External (git) code repositories
I'm not a git expert but we also have the possibility to use git sub trees ( it have the advantage of the git tool like submodules and it doesn't require an extra command unlike the submodule) ... aside from that, does every projects use git or have a composer file ? because it can push us to use one or another technique .. Regards, eMerzh On Tue, Nov 27, 2012 at 9:50 PM, Bart Visscher ba...@thisnet.nl wrote: Hello all, At the sprint in Berlin I merged the routing branch. After the merge there where some problems with having to download extra repositories. We need to solve this problem in a structural way. The quick fix for the routing was to include the code in our 3rdparty repository. I would like to discuss how to have the external code in 3rdparty. At the moment I suggest to limit this to the large projects: Sabre, Symfony routing component and Doctrine. For these projects we have 2 ways of doing this. The first is to use git submodules, and the second one is to use composer. The positive about git submodules is that you don't need extra programs to download the code, every thing you need is included with the git installation. There are some posts on the internet that this doesn't work very well with switching branches, but I'm not sure how relevant that is for us. With composer you don't need git to get the external code, but can download a php phar and use that to download the other code. With composer we can also specify the version in a flexible way. We can also do a combination of these, then everyone can chose which works better for them. Haven't really tested this, but i expect that this works. The commands to get the external code are: git submodules: git submodule init git submodule update composer: curl -s https://getcomposer.org/installer | php composer update Bart ___ Owncloud mailing list Owncloud@kde.org https://mail.kde.org/mailman/listinfo/owncloud ___ Owncloud mailing list Owncloud@kde.org https://mail.kde.org/mailman/listinfo/owncloud
