Re: [fossil-users] Handling renames on import
Replying Richard Hipp: On Mon, May 27, 2013 at 3:35 PM, Isaac Jurado dipto...@gmail.com wrote: Hello, I'm testing Fossil by migrating a bunch of Mercurial repositories. Unlike Git, Mercurial tracks file renames. However, in order to create a Fossil repository I have to follow the Mercurial - Git - Fossil path. On the first conversion I lose file copying/renaming information. But the git fast-export command has the -M and -C switches to enable copy/rename detection heuristics. Therefore, the last conversion step looks like: git fast-export -M -C --all | fossil import --git ../myrepo.fossil The good news is that the heuristics work, as Fossil fails with the following message: cannot handle R records, use --full-tree Do you have a git-fast-export output file that you can send me for testing purposes (perhaps via private email)? Well, I've been digging around a bit. After reading the git-fast-import man page and doing some experiments, I've discovered the following: The R record has exactly the same syntax as the C record. Therefore, for each file that has been renamed there is a commit record like: R old_name new_name If, apart from a rename, there were changes in the file contents, an additional M record is appended just below the rename. I'm not really sure about the value and support of file rename in Fossil for three reasons: 1. There is no copy command 2. The mv/rename command does not actually operate on the filesystem, but after the user has performed the operation 3. This mail: http://lists.fossil-scm.org:8080/pipermail/fossil-users/2008-July/000325.html I'm willing to carry on and try to fix this missing import piece, but only if it is worth the trouble. Does mv/rename improve the delta algorithm and/or file tracking in any way? Regards. -- Isaac Jurado The noblest pleasure is the joy of understanding. Leonardo da Vinci ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Handling renames on import
On Wed, May 29, 2013 at 4:22 PM, Isaac Jurado dipto...@gmail.com wrote: Does mv/rename improve the delta algorithm and/or file tracking in any way? Unclear. I think it does, but I'm not 100% sure. Deltas in Fossil are decoupled from the rest of the system. There is no fixed delta schedule. You can create new, completely compatible versions of Fossil that delta differently. So it is certainly possible to create a version of Fossil that takes advantage of mv/rename to improve delta encoding performance. But whether or not the current implementation does so is unclear. Probably best to determine that by experimentation. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Handling renames on import
On Mon, May 27, 2013 at 3:35 PM, Isaac Jurado dipto...@gmail.com wrote:
Hello,
I'm testing Fossil by migrating a bunch of Mercurial repositories.
Unlike Git, Mercurial tracks file renames. However, in order to create
a Fossil repository I have to follow the Mercurial - Git - Fossil
path.
Is there a better way? Can we create a direct Hg-Fossil importer?
On the first conversion I lose file copying/renaming information. But
the git fast-export command has the -M and -C switches to enable
copy/rename detection heuristics. Therefore, the last conversion step
looks like:
git fast-export -M -C --all | fossil import --git ../myrepo.fossil
The good news is that the heuristics work, as Fossil fails with the
following message:
cannot handle R records, use --full-tree
Which would be understandable if it was a completely unsupported caes.
Nevertheless, I searched for the place where the error originates and I
found this:
I apparently wrote that particular code. But if I ever knew what an R
record was, I have now forgotten. You might try commenting out the
fossil_fatal() call at the end of that code block to see what happens. If
that doesn't work, somebody is going to need to spend a day or say
reverse-engineering the rename-support in git-fast-export.
}else
if( memcmp(zLine, R , 2)==0 ){
int nFrom;
import_prior_files();
z = zLine[2];
zFrom = next_token(z);
zTo = rest_of_line(z);
i = 0;
nFrom = strlen(zFrom);
while( (pFile = import_find_file(zFrom, i, gg.nFile))!=0 ){
if( pFile-isFrom==0 ) continue;
pNew = import_add_file();
pFile = gg.aFile[i-1];
if( strlen(pFile-zName)nFrom ){
pNew-zName = mprintf(%s%s, zTo, pFile-zName[nFrom]);
}else{
pNew-zName = fossil_strdup(pFile-zName);
}
pNew-zPrior = pFile-zName;
pNew-isExe = pFile-isExe;
pNew-isLink = pFile-isLink;
pNew-zUuid = pFile-zUuid;
pNew-isFrom = 0;
gg.nFile--;
*pFile = *pNew;
memset(pNew, 0, sizeof(*pNew));
}
fossil_fatal(cannot handle R records, use --full-tree);
}else
Then I tried commenting the call to fossil_fatal() and the import
completed, but the old files were not removed.
So, finally, my question is. What is that code lacking in order to
correctly import copy and rename information?
Regards.
--
Isaac Jurado
The noblest pleasure is the joy of understanding.
Leonardo da Vinci
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Handling renames on import
On Mon, May 27, 2013 at 3:45 PM, Richard Hipp d...@sqlite.org wrote: On Mon, May 27, 2013 at 3:35 PM, Isaac Jurado dipto...@gmail.com wrote: Hello, I'm testing Fossil by migrating a bunch of Mercurial repositories. Unlike Git, Mercurial tracks file renames. However, in order to create a Fossil repository I have to follow the Mercurial - Git - Fossil path. Is there a better way? Can we create a direct Hg-Fossil importer? Seriously - is there a description of the Hg format someplace that is sufficient detailed to be able export the repo file? Note that there is no such description of git. There is a format description for git-fast-export which is ok but not great. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Handling renames on import
On Mon, May 27, 2013 at 3:35 PM, Isaac Jurado dipto...@gmail.com wrote:
Hello,
I'm testing Fossil by migrating a bunch of Mercurial repositories.
Unlike Git, Mercurial tracks file renames. However, in order to create
a Fossil repository I have to follow the Mercurial - Git - Fossil
path.
On the first conversion I lose file copying/renaming information. But
the git fast-export command has the -M and -C switches to enable
copy/rename detection heuristics. Therefore, the last conversion step
looks like:
git fast-export -M -C --all | fossil import --git ../myrepo.fossil
The good news is that the heuristics work, as Fossil fails with the
following message:
cannot handle R records, use --full-tree
Do you have a git-fast-export output file that you can send me for testing
purposes (perhaps via private email)?
Which would be understandable if it was a completely unsupported caes.
Nevertheless, I searched for the place where the error originates and I
found this:
}else
if( memcmp(zLine, R , 2)==0 ){
int nFrom;
import_prior_files();
z = zLine[2];
zFrom = next_token(z);
zTo = rest_of_line(z);
i = 0;
nFrom = strlen(zFrom);
while( (pFile = import_find_file(zFrom, i, gg.nFile))!=0 ){
if( pFile-isFrom==0 ) continue;
pNew = import_add_file();
pFile = gg.aFile[i-1];
if( strlen(pFile-zName)nFrom ){
pNew-zName = mprintf(%s%s, zTo, pFile-zName[nFrom]);
}else{
pNew-zName = fossil_strdup(pFile-zName);
}
pNew-zPrior = pFile-zName;
pNew-isExe = pFile-isExe;
pNew-isLink = pFile-isLink;
pNew-zUuid = pFile-zUuid;
pNew-isFrom = 0;
gg.nFile--;
*pFile = *pNew;
memset(pNew, 0, sizeof(*pNew));
}
fossil_fatal(cannot handle R records, use --full-tree);
}else
Then I tried commenting the call to fossil_fatal() and the import
completed, but the old files were not removed.
So, finally, my question is. What is that code lacking in order to
correctly import copy and rename information?
Regards.
--
Isaac Jurado
The noblest pleasure is the joy of understanding.
Leonardo da Vinci
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Handling renames on import
Wow, I didn't expect this attention. Let me reply in detail. I'm testing Fossil by migrating a bunch of Mercurial repositories. Unlike Git, Mercurial tracks file renames. However, in order to create a Fossil repository I have to follow the Mercurial - Git - Fossil path. Is there a better way? Can we create a direct Hg-Fossil importer? Seriously - is there a description of the Hg format someplace that is sufficient detailed to be able export the repo file? Note that there is no such description of git. There is a format description for git-fast-export which is ok but not great. For now it's enough. Besides, there are only two ways I know of exporting changesets (i.e. commits in Mercurial jargon): 1. An exported changeset, which looks like a patch with a header. 2. The wire protocol (or bundle), which I believe has a durable format, for backwards compatibility reasons. However, I still need to have a look at it. On the other hand, there are also two ways of converting from Mercurial to Git: 1. Via the fast-export scripts: git://repo.or.cz/fast-export.git 2. By pushing from a Mercurial repo to a Git repository with the aid of the hg-git extension: http://hg-git.github.io/ Whether we like it or not, Mercurial users have to deail with Git's network effect so the situation has to be accepted as it is. Therefore, I don't think that developing a straight importer from Mercurial is worth the effort. On the first conversion I lose file copying/renaming information. But the git fast-export command has the -M and -C switches to enable copy/rename detection heuristics. Therefore, the last conversion step looks like: git fast-export -M -C --all | fossil import --git ../myrepo.fossil The good news is that the heuristics work, as Fossil fails with the following message: cannot handle R records, use --full-tree Which would be understandable if it was a completely unsupported caes. Nevertheless, I searched for the place where the error originates and I found this: I apparently wrote that particular code. But if I ever knew what an R record was, I have now forgotten. You might try commenting out the fossil_fatal() call at the end of that code block to see what happens. If that doesn't work, somebody is going to need to spend a day or say reverse-engineering the rename-support in git-fast-export. Sounds like an interesting challenge. I'll try to make time this weekend to take a look. Do you have a git-fast-export output file that you can send me for testing purposes (perhaps via private email)? Right now I'm using private code for testing. But in a day or two I can look for my published repositories from which I can sure prepare a similar scenario. My initial motivation of all this copy and rename issue was to see if having the rename information would guide or help delta generation and provide better repository compression. I have special interest, even a bit irrational I must admit, in testing how Fossil's repository size responds to renames as I tend to reorganize code quite often. Specially since Mercurial suffers a bit dramatically in most cases. Thanks for your time. Best regards. -- Isaac Jurado The noblest pleasure is the joy of understanding. Leonardo da Vinci ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
