Re: [Flightgear-devel] fgdata trouble
> So you are calling for git monkeys that take care of the "tedious" > process of getting changes into the tree? If it is as critical to do this right as you say, everyone might be better off if only people who know what they're doing handle commits and every other commit goes through them, rather than everyone commits as he thinks and the specialists sort out the trouble later. What is tedious for me isn't necessarily for someone else with more knowledge. But I guess since you are talking about users with commit rights, there is no major point of disagreement here - I do agree that whoever gets direct access should be held to standards accordingly. * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Hi Thorsten, Renk Thorsten wrote: >> Every fgdata contributor who creates complicated xml/shader files should >> be able to understand basic git workflow as well... > > I'm not sure if you really mean every contributor, or every contributor > with commit rights to FGData. In the second case I'd agree with you, but > in case it's the first: I meant the second case indeed. > I don't think GIT is particularly simple, nor have I found a good > documentation. The basic tutorials / references which are human-readable > are nice, but then all sorts of problems not covered in the tutorial crop > up in reality. For instance, for some reason I can't push updates to my > devel branch to my repo clone because of some timestamp issue, but > remotely deleting and pushing new works fine. A rebase where the file a > patch applies to has been deleted on master is a really good puzzle. And > so on. On the other hand, the advanced manuals which would presumably > treat these problems get into specialized nomenclature like alternative > histories, octopus merging and what not and I can't find any > understandable answers there. If you need help in a special case, there are always people here who are glad to help. In case of deleted files upstream that you want to rebase upon, yes that can be a bit more difficult, but generally , if the file has been deleted it was deleted for a reason. > > So in order to understand it on the level you seem to be expecting, I > would really need to reserve a week and work through a long GIT reference > book. No need for that IMHO. You need to understand essentially 3 commands: "git pull --rebase", "git rebase", "git stash" to do 90% of the work that you will ever come along (not counting in commands like "git log", "git diff" and "git status" here, that are mainly for inspection purpose). > It's called specialization. In the physics department we work in, we have > for instance administrative secretaries. So, whenever I spend money from > my research grant, I don't know all the accounting codes for the various > items, nor the procedures, they do. Of course the system could in theory > be set up such as to require 60 physicists to learn accounting procedures > and follow all the accounting rule changes, but it's been generally > acknowledged that it's more efficient if the 4 secretaries do so, and the > physicists focus on their business. So you are calling for git monkeys that take care of the "tedious" process of getting changes into the tree? > > Of course, you can be of the opionion 'Hey, if you want to contribute > here, we require you to learn 'proper' GIT procedures' (whatever 'proper' > is...). To which an alternative scenario would be 'If you want my > contribution on your GIT server, you make it easy for me to get it there > and don't make my jump through 10 hoops.' Everyone is welcome to contribute, but yes, I request those people with commit rights to have a good knowledge of what they do when pushing to the repos. I don't mess with GLSL or Nasal code either if I have no clue what I'm doing. > I think asking every contributor to properly work through a GIT manual > before he can contribute is about as useful as to ask every contributor to > learn the effects and GLSL framework before he can contribute anything - > you're just reducing the not so large to begin with pool of contributors. > In case of Nasal or shader problems, I usually try to step up and help > with a fix if I can, because that's my speciality, I don't argue that > everyone must know all Nasal tricks before he can contribute. I would hope > that in case of GIT trouble, the GIT specialists step up. The specialists would love to have the possibility to step up, but that's only possible if they are asked *prior* to the push. Once the damage is done in the repo, fixing it is possible, but would include a rewrite of the history and that is not very much what anyone would like to do. Chris -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
> Every fgdata contributor who creates complicated xml/shader files should > be able to understand basic git workflow as well... I'm not sure if you really mean every contributor, or every contributor with commit rights to FGData. In the second case I'd agree with you, but in case it's the first: I don't think GIT is particularly simple, nor have I found a good documentation. The basic tutorials / references which are human-readable are nice, but then all sorts of problems not covered in the tutorial crop up in reality. For instance, for some reason I can't push updates to my devel branch to my repo clone because of some timestamp issue, but remotely deleting and pushing new works fine. A rebase where the file a patch applies to has been deleted on master is a really good puzzle. And so on. On the other hand, the advanced manuals which would presumably treat these problems get into specialized nomenclature like alternative histories, octopus merging and what not and I can't find any understandable answers there. So in order to understand it on the level you seem to be expecting, I would really need to reserve a week and work through a long GIT reference book. Now, I'm certainly intellectually capable of doing so, it just doesn't make any sense to me. I can spend time only once, so I can either read 400 pages of real-time 3d rendering techniques and GLSL (which I find interesting and helps me to do what I want) or GIT (which I find boring and doesn't help me develop what I want). It's a no-brainer what I do. It's called specialization. In the physics department we work in, we have for instance administrative secretaries. So, whenever I spend money from my research grant, I don't know all the accounting codes for the various items, nor the procedures, they do. Of course the system could in theory be set up such as to require 60 physicists to learn accounting procedures and follow all the accounting rule changes, but it's been generally acknowledged that it's more efficient if the 4 secretaries do so, and the physicists focus on their business. Of course, you can be of the opionion 'Hey, if you want to contribute here, we require you to learn 'proper' GIT procedures' (whatever 'proper' is...). To which an alternative scenario would be 'If you want my contribution on your GIT server, you make it easy for me to get it there and don't make my jump through 10 hoops.' I think asking every contributor to properly work through a GIT manual before he can contribute is about as useful as to ask every contributor to learn the effects and GLSL framework before he can contribute anything - you're just reducing the not so large to begin with pool of contributors. In case of Nasal or shader problems, I usually try to step up and help with a fix if I can, because that's my speciality, I don't argue that everyone must know all Nasal tricks before he can contribute. I would hope that in case of GIT trouble, the GIT specialists step up. * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
> Hi Pete, > > What about the idea I thought you were working > on of zipping each aircraft - presumably regularly > updated as more git changes happen - and having > like the FGx GUI downloading and installing these > at the users choice/request? > > Thus an initial install of fgdata would only have a > dozen or so a/c, like the releases, but additional > a/c could be easily installed on selection... > > Or did this not work out... Indeed I am working on an installer (time problems as usual) https://gitorious.org/fgx-xtras/fgx-installer The way it works is that 1) the server would make the zip, calculate the md5 to detect changes and present the "list" via an ajax feed http://fgx.ch/projects/fgx-server/repository/revisions/master/entry/fgx_shell/aircraft/import_update_zip_aircraft.py 2) the client would then read this list, and make a list of aircraft and/or updates available upon client. https://gitorious.org/fgx-xtras/fgx-installer However I kinda thought this is a naff idea, as it still means all a "zip" download of size. I also tested the possibility of calculating the md5 of each file, and somehow downloading only the changed files.. which is kinda a naff idea. This led me back to using svn to do the work and is the current idea... ie svn on server and an embedded svn client in the fgx-installer.. The svn server is there already, however I need to chat with Yves (FGx's BDFL) if its a good idea to make this public available, what with bandwidth etc... All the stuff is still experimental, but if there is interest then I;d be prepared to dedicate some more time to it. pete -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
On Sun, 2012-09-23 at 17:43 +0100, Peter Morgan wrote: > I got a really really slow connection > > Got around the git problem by using a script on a dedicated to pull > from git, and "push" to a subversion, which runs twice a day > > I then checkout from subversion read only of course, it works a treat > and get all updates very quickly.. > > However this approach is not for development, but as an user its a > very efficient way to keep up to data with master and 2.8 etc > > Pete > Hi Pete, What about the idea I thought you were working on of zipping each aircraft - presumably regularly updated as more git changes happen - and having like the FGx GUI downloading and installing these at the users choice/request? Thus an initial install of fgdata would only have a dozen or so a/c, like the releases, but additional a/c could be easily installed on selection... Or did this not work out... Regards, Geoff. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
I got a really really slow connection Got around the git problem by using a script on a dedicated to pull from git, and "push" to a subversion, which runs twice a day I then checkout from subversion read only of course, it works a treat and get all updates very quickly.. However this approach is not for development, but as an user its a very efficient way to keep up to data with master and 2.8 etc Pete -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
-Original Message- From: Torsten Dreyer Sent: Sunday, September 23, 2012 3:36 PM To: FlightGear developers discussions Subject: Re: [Flightgear-devel] fgdata trouble Hi all, there is a WIKI page for this topic: http://wiki.flightgear.org/FlightGear_Git:_splitting_fgdata Many points have been discussed over and over some time ago. If there is something new that has developed over time, please add it to the wiki page before it gets lost on the mailing list. Torsten -- Sadly http://wiki.flightgear.org/FlightGear_Git:_splitting_fgdata has been silent all of this year, and only has ideas and comments from 3 or 4 individuals. At the moment the wiki is not being used for this purpose. For a start could we draw up a spec for what is needed? As a user my initial input to this would be. 1. Ability to divide aircraft into categories. The obvious ones include military, light aircraft, civil transport, helicopters, spaceships, comic-book etc. This should be quite flexible. The ability to add categories should be available. Some aircraft will belong in more than one category, so a cross-reference is desirable. Assuming cross-referring is possible, a possible category is "author", which would allow authors to have their own easily identifiable collections to showcase. Boolean selection of the categories would also be a plus. 2. A central index, accessible both by Flightgear and by utilities such as Fgrun. 3. Download individual aircraft manually (e.g. by git , http, svn , torrent whatever). 4. Automated download of further aircraft and updates as already managed by the internal and external versions of Terrasync . 5. Aircraft currently in the release version should remain within the existing Fgdata. IMHO including just the Cessna is perhaps going too far. 6. Common/shared instruments, nasal libraries and other utilities should also remain within Fgdata. I have no doubt that core developers and others will have different requirements, but sorting issues like this that is what this forum is meant to be for. The implementation is something which I do not feel qualified to say much about, other than not making the system unmanageable by having too few repositories which are then too large, or having too many repositories making things too fragmented. Assuming cross-referenced categories are available as described above, the sub-division into separate repositories could be done on an arbitrary basis. It could be alphabetical, or simply in batches of 50 or so aircraft at a time. Such a structure would then be invisible to the end user. A central common index, perhaps duplicated by each repo, might be needed to manage this. Alan -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
"Alan Teeder" wrote: > Sorting out the aircraft is then a completely separate problem, and several > solutions to this have already been proposed. As we all remember one was > actually implemented last year, but for various reasons not made fully > public , was promptly abandoned. You might try searching the list archives for a posting by Durk in the first half of November last year. I've carefully avoided to declare my "favourite" as a "suggestion" or even a "recommendation" because I know it has several drawbacks ;-) Keeping just the C172 in the Base Package has downsides, keeping the release-hangar has downsides, installing various hangars by aircraft- type or level of fidelity has downsides, installing one repo per aircraft has downsides as well I'm pretty much convinced that the most beneficial and most appropriate solution has not yet been posted to this list. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Hi all, there is a WIKI page for this topic: http://wiki.flightgear.org/FlightGear_Git:_splitting_fgdata Many points have been discussed over and over some time ago. If there is something new that has developed over time, please add it to the wiki page before it gets lost on the mailing list. Torsten -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Martin Your suggestion of just having the Cessna is a "base package" version of fgdata is just what is required. I would go further and extend it to including all those aircraft which are in the regular release versions. These particular aircraft are on the whole well maintained and showcase the capabilities of Flightgear. Sorting out the aircraft is then a completely separate problem, and several solutions to this have already been proposed. As we all remember one was actually implemented last year, but for various reasons not made fully public , was promptly abandoned. Flightgear already supports the use of multiple aircraft directories, and I suggest that this should form the basis for dividing up and managing the existing collection, as well as making a structure for adding 3rd party hangar collections. Alan -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
On Sun, 2012-09-23 at 13:33 +, Martin Spott wrote: > "Alan Teeder" wrote: > > > Brisa script has this line > >git clone git://gitorious.org/fg/fgdata.git fgdata > > > > This is also the default to which all new users are most likely to come > > across.. > > > Q) We need to use smaller bolts for the railway bridge ! Every time I >try to mount a bolt with my hammer, I'm getting stuck in the >middle. > A) Would you consider using one of the big sledgehammers for these >bolts ? > Q) I picked up one of the toolbooxes they have as a gift at the local >homebuilders store and it contains just this single 250 g hammer. >I think it's the default to assemble bridges with hammers of this >size. > > > I agree, the Base Package is really big, but, as Vivian already pointed > out, the problem is way too complicated for solutions you can buy in a > box. > > Cheers, > Martin. The main problem is of distribution of aircraft, this is the largest and most volatile source of data in fgdata, if we had some way to distribute aircraft "on demand" that also auto-update when versions are released that would cut out a large portion of the problem. So what if we a mechanism that would on-demand download an aircraft package when it is first referenced (fgrun, or on the command line). The downloading could pre-cached by downloading tar files. Then we can remove (in theory) all the aircraft from fgdata, this would also remove the need for juggling the "base package" aircraft if that was desired. Since moving to gitorious, there is no good reason for any aircraft developer to be developing in fgdata, they can have their own development "hangars" then release a packaged aircraft to the on-demand download server. The download package could use the -set.xml file (or a new manifest file) that contains versioning (package version and matching FG version) and any license information or dependencies. The downloading of these aircraft packages only needs to be done by HTTP or FTP, it would just be a single packaged tar file, these could be gzip/bziped to make the transfer smaller. The concept could be further developed to provide a single "aircraft" store similar in concept to android, iOS, Windows etc except all of our packages would be free. Just a thought S. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
"Alan Teeder" wrote: > Brisa script has this line >git clone git://gitorious.org/fg/fgdata.git fgdata > > This is also the default to which all new users are most likely to come > across.. Q) We need to use smaller bolts for the railway bridge ! Every time I try to mount a bolt with my hammer, I'm getting stuck in the middle. A) Would you consider using one of the big sledgehammers for these bolts ? Q) I picked up one of the toolbooxes they have as a gift at the local homebuilders store and it contains just this single 250 g hammer. I think it's the default to assemble bridges with hammers of this size. I agree, the Base Package is really big, but, as Vivian already pointed out, the problem is way too complicated for solutions you can buy in a box. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Martin wrote: > "Alan Teeder" wrote: > > > Twice I left it running overnight, but it failed both times after > > several hours during the fgdata clone. > > Which server do you clone from ? If you don't already do so, then you should > consider fetching the initial clone from mapserver and to change the remote > origin afterwards. > > > The fact remains that I believe that the current fgdata is too large > > and is not fit for purpose. The need to re-organise it exists now, as > > it did last year. Perhaps you, or someone else, could comment on that. > > Indeed, the current evil is asking for a *clever* solution but every of those > I've seen so far (at least most of them) had been ignoring one or another > quite relevant context and I think we're not well-advised to embark on yet > another significant evil to get rid of the current one. > > I don't know the 'perfect' recipe either. My favourite would be to keep just > the default Cessna in the base package and to move all the other aircraft into > one single, separate repo but as far as I remember, this plan doesn't have > much support (for various reasons, I guess). > Yes, that would be an obvious plan, but the elephant in the room is that it is the Aircraft folder which contains the bulk of the problem, so as you said, we swap one evil for another. I suppose if there were an answer to this problem we would have implemented it by now. Vivian -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Stefan Seifert wrote: > Since really only the initial clone is a problem, we could just offer a > weekly > updated tar ball of a bare clone for download. This download would just be ~ > 5 > GiB and would be resumable. [...] > Any thoughts? Good idea. We actually had this for a while, but I don't know wether it's still available, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
On Sunday 23 September 2012 10:19:52 Alan Teeder wrote: > The reason I quoted 10 Gb is that my fgdate/.git/objects directory is > currently 8.9Gb, and I assumed that is what gets downloaded during a clone. > I bow to your wisdom if you say that it is "only" 4.9Gb. Since really only the initial clone is a problem, we could just offer a weekly updated tar ball of a bare clone for download. This download would just be ~ 5 GiB and would be resumable. Getting an up to date fgdata for the first time would then just be something like the following sequence: wget -c http://.../fgdata.git.tar.xz tar xJf fgdata.git.tar.xz cd fgdata git checkout master git pull This would keep all the advantages of having everything in the same repo while offering a reliable solution for people who download for the first time. Keeping the tar ball up to date would be a simple cron job. Any thoughts? Stefan -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
-Original Message- From: Martin Spott Sent: Sunday, September 23, 2012 11:42 AM Newsgroups: list.flightgear-devel To: flightgear-devel@lists.sourceforge.net Subject: Re: [Flightgear-devel] fgdata trouble "Alan Teeder" wrote: > Twice I left it running overnight, but it failed both times after several > hours during the fgdata clone. Which server do you clone from ? If you don't already do so, then you should consider fetching the initial clone from mapserver and to change the remote origin afterwards. --- Martin Brisa script has this line git clone git://gitorious.org/fg/fgdata.git fgdata This is also the default to which all new users are most likely to come across.. Alan -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Bertrand Coconnier wrote: > git repack > git gc > git prune Same here, I'm running a similar set as hooks on the mapserver GIT mirror, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
"Alan Teeder" wrote: > Twice I left it running overnight, but it failed both times after several > hours during the fgdata clone. Which server do you clone from ? If you don't already do so, then you should consider fetching the initial clone from mapserver and to change the remote origin afterwards. > The fact remains that I believe that the current fgdata is too large and is > not fit for purpose. The need to re-organise it exists now, as it did last > year. Perhaps you, or someone else, could comment on that. Indeed, the current evil is asking for a *clever* solution but every of those I've seen so far (at least most of them) had been ignoring one or another quite relevant context and I think we're not well-advised to embark on yet another significant evil to get rid of the current one. I don't know the 'perfect' recipe either. My favourite would be to keep just the default Cessna in the base package and to move all the other aircraft into one single, separate repo but as far as I remember, this plan doesn't have much support (for various reasons, I guess). Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
-Original Message- From: Bertrand Coconnier Sent: Sunday, September 23, 2012 10:30 AM To: FlightGear developers discussions Subject: Re: [Flightgear-devel] fgdata trouble I have the same size than Martin. For that I execute on a regular basis the following commands git repack git gc git prune --- Betrand Many thanks for the "gitspertise" now I have just 4.87 Gb in .git. It took less than 15 minutes. Alan -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
On Sat, 2012-09-22 at 19:44 +, Martin Spott wrote: > "Alan Teeder" wrote: > > > New flightgear git users are faced with an initial download of about 10gb > > just to get started. > > Currently the "fgdata" GIT repo has approx. 4.9 GByte, > > Martin. Hi Martin, Alan, Bertrand, et al... I guess we each have a 'different' way of measuring things ;=)) This is back on Sep 1, and it grows every day... /media/Disk2/FG/fg20$ dirdate fgdata Processed 9130 directories, 59899 files, 11,281,006,933 bytes. Latest : fgdata/.git/index 2012/08/01 16:08:21, of 59899 files. /media/Disk2/FG/fg20$ dirsize fgdata Doing du -b -s fgdata... moment... Total of [fgdata] = 11,298,718,037 bytes (10.52GB) Ok, now I know some MORE git commands that reduce this on disk size, thanks Bertrand, $ git repack $ git gc $ git prune but that does not stop the initial download into fgdata/.git /media/Disk2/FG/fg20/fgdata$ dirsize .git Doing du -b -s .git... moment... Total of [.git] = 4,419,875,575 bytes (4.12GB) It seems a shame all the discussion I saw about somehow splitting this 4-5GB download came to nothing ;=(( It seems 'some' absolutely wanted to keep a single git command to get it ALL... while I would see no trouble in a simple clean split... like - $ git clone fgdata $ cd fgdata $ git clone Aircraft /media/Disk2/FG/fg20/fgdata$ dirsize Aircraft Doing du -b -s Aircraft... moment... Total of [Aircraft] = 5,879,186,664 bytes (5.48GB) so assume the .git portion of this would be similarly about half... But that does not stop the BIG initial downloads, just splits it into 2, so perhaps better - $ git clone fgdata also includes a dozen or so Aircraft, like the windows release packages... Then I go somewhere else to ADD Aircraft, when, if I want... Perhaps a nice little GUI app, that lets me choose which aircraft I want to add... maybe does a download of a zip file, and continues to unzip it into - /fgdata/Aircraft Something I think Pete was working on... Or into a 'separate' folder since I think we do now support multiple 'aircraft' directories, or should if we don't... And it would be nice if that app could also download from other people's 'hangars'... So you can see I am strongly with Alan on this... this topic needs to be continued, and resolved... BTW, my makefg script also frequently FAILS on fgdata - clone or pull... So much so that I usually now always do that manually... seems git does not like to be 'managed' in a script... Regards, Geoff. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
2012/9/23 Alan Teeder : > > The reason I quoted 10 Gb is that my fgdate/.git/objects directory is > currently 8.9Gb, and I assumed that is what gets downloaded during a clone. > I bow to your wisdom if you say that it is "only" 4.9Gb. I have the same size than Martin. For that I execute on a regular basis the following commands git repack git gc git prune Given the size of your objects, expect these commands to take many minutes (hopefully less than 1 hour) to complete. My rule of thumb is to execute these commands when git count-objects reports several objects and kilobytes. Bertrand. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Martin Last week I set up a Linux partition and ran the Brisa script. Twice I left it running overnight, but it failed both times after several hours during the fgdata clone. Eventually I just copied over all 15gb from my current windows fgdata directory and did a git pull on this. That seems to have solved the problem for me (although there may well be some Windows-Linux CRLF issues), but this would of course not be a posible solution for a new user. My Internet connection is ADSL2+ over copper, and the router reports the current data rate as 6583 kbps, which is all I can get out here in the middle of nowhere. Although not lightning fast, it is quite stable. The reason I quoted 10 Gb is that my fgdate/.git/objects directory is currently 8.9Gb, and I assumed that is what gets downloaded during a clone. I bow to your wisdom if you say that it is "only" 4.9Gb. The fact remains that I believe that the current fgdata is too large and is not fit for purpose. The need to re-organise it exists now, as it did last year. Perhaps you, or someone else, could comment on that. Regards Alan -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
Thanks Thorsten for your clear words, yes, it sucks to see people messing up the history, merging local branches and then pushing them to gitorious. I personally see another problem in the way changes that are merged appear in the history: The merge date is there, but the commits associated to it can be hidden somewhere down in the history. This is a very bad state when it comes to bughunting (bisecting). Every fgdata contributor who creates complicated xml/shader files should be able to understand basic git workflow as well... Chris ThorstenB wrote: > On 21 Sep 2012, at 13:03, Anders Gidenstam wrote: >> The master branch of fgdata has become messed up. A number of commits >> have become duplicated and some undone (they exist in the history but >> their effect is gone from HEAD). > > It has happened again, fgdata history is messed up. It looks as if my > last commits (6d46fe7, f722671) were applied to fgdata multiple times > now (duplicates are 818b92f and e7452f7). With gitk itself, I can't even > see how where these originated (looks as if I had pushed them multiple > times). Only the gitorious.org history shows that these were indeed not > pushed by me: > > https://www.gitorious.org/fg/fgdata/commit/818b92fa07a49c333f80ac21f483b33fd5e2b7c7 > > https://www.gitorious.org/fg/fgdata/commit/e7452f70358e67824e499294fd32d6d6f8d3dd93 > > Can we please make it a requirement that _local_ merge operations must > not become visible on our public repository, so _everyone_ has to > "rebase" before pushing anything? > > The only merge/branch operations that should be visible in our public > repo should be those that affect public branches (release branch > creation/merges etc). > > There's really no reason why other people should see that user XYZ has > merged his local branch 1-15 times with gitorious, before pushing back. > It only results in the git history becoming unreadable. And apparently > it results in more issues. > > If you compare simgear's or flightgear's history with fgdata's history, > you'll see how nice and readable a git history can be - since obviously > (almost) everyone pushing to sg/fg knows hows how to rebase. > > Resolving merge operations locally, to reorder and cleanup the history > is really simple - and only requires a few seconds. If you have > uncommited changes in your local directory, you can temporarily stash > them - so that "rebase" won't complain. > > For fgdata: > git stash > git rebase origin/master > git stash pop > > (And for simgear/flightgear: > git stash > git rebase origin/next > git stash pop > ) > > It is also a good idea to check the git history (gitk) before pushing to > gitorious: you can read and double check your own changes a final time - > and also check the history for local merge or funny duplication issues. > If you're having local Git trouble (like duplications), *please* ask > before pushing to gitorious. > > cheers, > Thorsten > > -- > Got visibility? > Most devs has no idea what their production app looks like. > Find out how fast your code is with AppDynamics Lite. > http://ad.doubleclick.net/clk;262219671;13503038;y? > http://info.appdynamics.com/FreeJavaPerformanceDownload.html -- How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
"Alan Teeder" wrote: > New flightgear git users are faced with an initial download of about 10gb > just to get started. Currently the "fgdata" GIT repo has approx. 4.9 GByte, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] fgdata trouble
What happened to the work that went into breaking up fgdata into manageable chunks? New flightgear git users are faced with an initial download of about 10gb just to get started. It seems to me that this current problem is another good reason to re-arrange fgdata. So much work went into it last year, all to be scuppered by what seems to have been a unwillingness on the part of a few individuals to agree on the implementation. Alan -Original Message- From: Tim Moore Sent: Saturday, September 22, 2012 8:34 AM To: FlightGear developers discussions Subject: Re: [Flightgear-devel] fgdata trouble On Fri, Sep 21, 2012 at 11:51 PM, ThorstenB wrote: > On 21 Sep 2012, at 13:03, Anders Gidenstam wrote: >> The master branch of fgdata has become messed up. A number of commits ... > It has happened again, fgdata history is messed up. It looks as if my > last commits (6d46fe7, f722671) were applied to fgdata multiple times > now (duplicates are 818b92f and e7452f7). With gitk itself, I can't even > see how where these originated (looks as if I had pushed them multiple > times). Only the gitorious.org history shows that these were indeed not > pushed by me: > > https://www.gitorious.org/fg/fgdata/commit/818b92fa07a49c333f80ac21f483b33fd5e2b7c7 > > https://www.gitorious.org/fg/fgdata/commit/e7452f70358e67824e499294fd32d6d6f8d3dd93 > > Can we please make it a requirement that _local_ merge operations must > not become visible on our public repository, so _everyone_ has to > "rebase" before pushing anything? > > The only merge/branch operations that should be visible in our public > repo should be those that affect public branches (release branch > creation/merges etc). > I haven't looked at this latest problem in detail, but you can do as much damage rebasing as merging. I suspect the real problem here is rampant cherry-picking. I happen to think merging, at least when pushing local changes to the public tree, is a lot better than rebasing, because then a given set of changes only appears in a single commit in the repository. > There's really no reason why other people should see that user XYZ has > merged his local branch 1-15 times with gitorious, before pushing back. > It only results in the git history becoming unreadable. And apparently > it results in more issues. Yes, pushing a branch that has numerous merges from master/next is unpleasant. There are many pages on the Web describing git workflows that avoid rebasing and keep a clean history. > > If you compare simgear's or flightgear's history with fgdata's history, > you'll see how nice and readable a git history can be - since obviously > (almost) everyone pushing to sg/fg knows hows how to rebase. Except that I never rebase before pushing to sg/fg :) I should qualify that: I do interactively rebase before pushing, often rearranging commits and divying them up to make more sense. But I *never* rebase onto a different head than the one on which I started the branch. > > Resolving merge operations locally, to reorder and cleanup the history > is really simple - and only requires a few seconds. If you have > uncommited changes in your local directory, you can temporarily stash > them - so that "rebase" won't complain. > > For fgdata: > git stash > git rebase origin/master > git stash pop > > (And for simgear/flightgear: > git stash > git rebase origin/next > git stash pop > ) > > It is also a good idea to check the git history (gitk) before pushing to > gitorious: you can read and double check your own changes a final time - > and also check the history for local merge or funny duplication issues. > If you're having local Git trouble (like duplications), *please* ask > before pushing to gitorious. Can't argue with that. > > cheers, > Thorsten > > -- > Got visibility? > Most devs has no idea what their production app looks like. > Find out how fast your code is with AppDynamics Lite. > http://ad.doubleclick.net/clk;262219671;13503038;y? > http://info.appdynamics.com/FreeJavaPerformanceDownload.html > ___ > Flightgear-devel mailing list > Flightgear-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/flightgear-devel -- How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html
Re: [Flightgear-devel] fgdata trouble
On Fri, Sep 21, 2012 at 11:51 PM, ThorstenB wrote: > On 21 Sep 2012, at 13:03, Anders Gidenstam wrote: >> The master branch of fgdata has become messed up. A number of commits ... > It has happened again, fgdata history is messed up. It looks as if my > last commits (6d46fe7, f722671) were applied to fgdata multiple times > now (duplicates are 818b92f and e7452f7). With gitk itself, I can't even > see how where these originated (looks as if I had pushed them multiple > times). Only the gitorious.org history shows that these were indeed not > pushed by me: > > https://www.gitorious.org/fg/fgdata/commit/818b92fa07a49c333f80ac21f483b33fd5e2b7c7 > > https://www.gitorious.org/fg/fgdata/commit/e7452f70358e67824e499294fd32d6d6f8d3dd93 > > Can we please make it a requirement that _local_ merge operations must > not become visible on our public repository, so _everyone_ has to > "rebase" before pushing anything? > > The only merge/branch operations that should be visible in our public > repo should be those that affect public branches (release branch > creation/merges etc). > I haven't looked at this latest problem in detail, but you can do as much damage rebasing as merging. I suspect the real problem here is rampant cherry-picking. I happen to think merging, at least when pushing local changes to the public tree, is a lot better than rebasing, because then a given set of changes only appears in a single commit in the repository. > There's really no reason why other people should see that user XYZ has > merged his local branch 1-15 times with gitorious, before pushing back. > It only results in the git history becoming unreadable. And apparently > it results in more issues. Yes, pushing a branch that has numerous merges from master/next is unpleasant. There are many pages on the Web describing git workflows that avoid rebasing and keep a clean history. > > If you compare simgear's or flightgear's history with fgdata's history, > you'll see how nice and readable a git history can be - since obviously > (almost) everyone pushing to sg/fg knows hows how to rebase. Except that I never rebase before pushing to sg/fg :) I should qualify that: I do interactively rebase before pushing, often rearranging commits and divying them up to make more sense. But I *never* rebase onto a different head than the one on which I started the branch. > > Resolving merge operations locally, to reorder and cleanup the history > is really simple - and only requires a few seconds. If you have > uncommited changes in your local directory, you can temporarily stash > them - so that "rebase" won't complain. > > For fgdata: > git stash > git rebase origin/master > git stash pop > > (And for simgear/flightgear: > git stash > git rebase origin/next > git stash pop > ) > > It is also a good idea to check the git history (gitk) before pushing to > gitorious: you can read and double check your own changes a final time - > and also check the history for local merge or funny duplication issues. > If you're having local Git trouble (like duplications), *please* ask > before pushing to gitorious. Can't argue with that. > > cheers, > Thorsten > > -- > Got visibility? > Most devs has no idea what their production app looks like. > Find out how fast your code is with AppDynamics Lite. > http://ad.doubleclick.net/clk;262219671;13503038;y? > http://info.appdynamics.com/FreeJavaPerformanceDownload.html > ___ > Flightgear-devel mailing list > Flightgear-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/flightgear-devel -- How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
[Flightgear-devel] fgdata trouble
On 21 Sep 2012, at 13:03, Anders Gidenstam wrote: > The master branch of fgdata has become messed up. A number of commits > have become duplicated and some undone (they exist in the history but > their effect is gone from HEAD). It has happened again, fgdata history is messed up. It looks as if my last commits (6d46fe7, f722671) were applied to fgdata multiple times now (duplicates are 818b92f and e7452f7). With gitk itself, I can't even see how where these originated (looks as if I had pushed them multiple times). Only the gitorious.org history shows that these were indeed not pushed by me: https://www.gitorious.org/fg/fgdata/commit/818b92fa07a49c333f80ac21f483b33fd5e2b7c7 https://www.gitorious.org/fg/fgdata/commit/e7452f70358e67824e499294fd32d6d6f8d3dd93 Can we please make it a requirement that _local_ merge operations must not become visible on our public repository, so _everyone_ has to "rebase" before pushing anything? The only merge/branch operations that should be visible in our public repo should be those that affect public branches (release branch creation/merges etc). There's really no reason why other people should see that user XYZ has merged his local branch 1-15 times with gitorious, before pushing back. It only results in the git history becoming unreadable. And apparently it results in more issues. If you compare simgear's or flightgear's history with fgdata's history, you'll see how nice and readable a git history can be - since obviously (almost) everyone pushing to sg/fg knows hows how to rebase. Resolving merge operations locally, to reorder and cleanup the history is really simple - and only requires a few seconds. If you have uncommited changes in your local directory, you can temporarily stash them - so that "rebase" won't complain. For fgdata: git stash git rebase origin/master git stash pop (And for simgear/flightgear: git stash git rebase origin/next git stash pop ) It is also a good idea to check the git history (gitk) before pushing to gitorious: you can read and double check your own changes a final time - and also check the history for local merge or funny duplication issues. If you're having local Git trouble (like duplications), *please* ask before pushing to gitorious. cheers, Thorsten -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
[Flightgear-devel] fgdata trouble
Hi all, The master branch of fgdata has become messed up. A number of commits have become duplicated and some undone (they exist in the history but their effect is gone from HEAD). I'm not sure how to sort this out or if it is worth the effort to do so (I can easily push yet another copy of my commits that got undone to reinstate their effect - but I don't know how many more commits have been undone). /Please be careful/ and if you are not sure that the branch you are merging into master is ok do not push the result to the master repository. Instead, ask somebody with more git experience to have a look first. Example of undone commits - they exist in the history yet their effects are not in the HEAD revision (despite being the most recent commits to the respective files - excepting merges): 7715893d774fe966cb332ef37206c5198cbff690 c973fc9dc4895f3366204dffdc013d83aabcb533 Cheers, Anders -- --- Anders Gidenstam WWW: http://gitorious.org/anders-hangar http://www.gidenstam.org/FlightGear/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
