Re: Changing encoding of a file : What should happen to CRLF in file ?
Great work !!! Thanks On Fri, Nov 24, 2017 at 1:55 AM, Torsten Bögershausen wrote: > On Thu, Nov 23, 2017 at 10:01:40PM +0530, Ashish Negi wrote: >> Thanks for confirming. >> Is it possible to track this via a bug number ? >> It will help me to try out the fix when its available. >> > > No worry, the fix is nearly complete and will come out in a couple of days.
Re: Changing encoding of a file : What should happen to CRLF in file ?
On Thu, Nov 23, 2017 at 10:01:40PM +0530, Ashish Negi wrote: > Thanks for confirming. > Is it possible to track this via a bug number ? > It will help me to try out the fix when its available. > No worry, the fix is nearly complete and will come out in a couple of days.
Re: Changing encoding of a file : What should happen to CRLF in file ?
Thanks for confirming. Is it possible to track this via a bug number ? It will help me to try out the fix when its available. On Thu, Nov 16, 2017 at 9:45 PM, Torsten Bögershausen wrote: > On Thu, Nov 16, 2017 at 12:35:33AM +0530, Ashish Negi wrote: >> On windows : >> > git --version >> git version 2.14.2.windows.2 >> >> On linux : >> > git --version >> git version 2.7.4 >> >> I would like to understand the solution : >> If i understood it correctly : it removes file_name.txt from index, so >> git forgets about it. >> we then add the file again after changing encoding. This time, git >> takes it as utf-8 file and converts crlf to lf when storing it >> internally. >> Right ? > > Yes, exactly. > (In a coming release of Git there will be a "git add --renormalize > " ) > >> >> Thank you for the support. >> > > Thanks for a clean bug report. > Actually it is a bug, I put it on my to do list > >
Re: Changing encoding of a file : What should happen to CRLF in file ?
On Thu, Nov 16, 2017 at 12:35:33AM +0530, Ashish Negi wrote: > On windows : > > git --version > git version 2.14.2.windows.2 > > On linux : > > git --version > git version 2.7.4 > > I would like to understand the solution : > If i understood it correctly : it removes file_name.txt from index, so > git forgets about it. > we then add the file again after changing encoding. This time, git > takes it as utf-8 file and converts crlf to lf when storing it > internally. > Right ? Yes, exactly. (In a coming release of Git there will be a "git add --renormalize " ) > > Thank you for the support. > Thanks for a clean bug report. Actually it is a bug, I put it on my to do list
Re: Changing encoding of a file : What should happen to CRLF in file ?
On windows : > git --version git version 2.14.2.windows.2 On linux : > git --version git version 2.7.4 I would like to understand the solution : If i understood it correctly : it removes file_name.txt from index, so git forgets about it. we then add the file again after changing encoding. This time, git takes it as utf-8 file and converts crlf to lf when storing it internally. Right ? Thank you for the support. On Wed, Nov 15, 2017 at 10:42 PM, Torsten Bögershausen wrote: > On Wed, Nov 15, 2017 at 01:41:42PM +0530, Ashish Negi wrote: >> > If you commit the file, it will be stored with LF in the index, >> This is what i believe is not happening. >> >> Lets do this with a public repository and steps which are reproducible. >> I have created a repo : https://github.com/ashishnegi/git_encoding >> >> If you clone this repo in linux and run `git status`, you will find >> that file is modified. > > This is what what I get: > > git ls-files --eol > i/lfw/lfattr/text=auto .gitattributes > i/crlf w/crlf attr/text=auto file_name.txt > > (And you get the same, I think) > Newer versions of Git (>=2.10) will -not- treat file_name.txt as changed, > older versions do. > What does "git --version" say on your Linux machine ? > > However, if you want to fix it, you want to either end up with > > A) > git ls-files --eol > i/lfw/lfattr/text=auto .gitattributes > i/lfw/crlf attr/text=auto file_name.txt > > or > B) > git ls-files --eol > i/lfw/lfattr/text=auto .gitattributes > i/crlf w/crlf attr/-text file_name.txt > > (The "attr/-text" means "don't change the line endings") > > Both A) or B) will work, typically A) is preferred. > > You should be able to achive A) : > git rm --cached file_name.txt && git add file_name.txt > rm 'file_name.txt' > warning: CRLF will be replaced by LF in file_name.txt. > The file will have its original line endings in your working directory. > > git ls-files --eol > i/lfw/lfattr/text=auto .gitattributes > i/lfw/crlf attr/text=auto file_name.txt > > [snip the rest]
Re: Changing encoding of a file : What should happen to CRLF in file ?
On Wed, Nov 15, 2017 at 01:41:42PM +0530, Ashish Negi wrote: > > If you commit the file, it will be stored with LF in the index, > This is what i believe is not happening. > > Lets do this with a public repository and steps which are reproducible. > I have created a repo : https://github.com/ashishnegi/git_encoding > > If you clone this repo in linux and run `git status`, you will find > that file is modified. This is what what I get: git ls-files --eol i/lfw/lfattr/text=auto .gitattributes i/crlf w/crlf attr/text=auto file_name.txt (And you get the same, I think) Newer versions of Git (>=2.10) will -not- treat file_name.txt as changed, older versions do. What does "git --version" say on your Linux machine ? However, if you want to fix it, you want to either end up with A) git ls-files --eol i/lfw/lfattr/text=auto .gitattributes i/lfw/crlf attr/text=auto file_name.txt or B) git ls-files --eol i/lfw/lfattr/text=auto .gitattributes i/crlf w/crlf attr/-text file_name.txt (The "attr/-text" means "don't change the line endings") Both A) or B) will work, typically A) is preferred. You should be able to achive A) : git rm --cached file_name.txt && git add file_name.txt rm 'file_name.txt' warning: CRLF will be replaced by LF in file_name.txt. The file will have its original line endings in your working directory. git ls-files --eol i/lfw/lfattr/text=auto .gitattributes i/lfw/crlf attr/text=auto file_name.txt [snip the rest]
Re: Changing encoding of a file : What should happen to CRLF in file ?
> If you commit the file, it will be stored with LF in the index, This is what i believe is not happening. Lets do this with a public repository and steps which are reproducible. I have created a repo : https://github.com/ashishnegi/git_encoding If you clone this repo in linux and run `git status`, you will find that file is modified. About repo : Repo have 2 commits, done on windows machine. First one check in a utf-16le encoded file which has crlf. crlf will not be converted to lf in index as git treats it as binary file. 2nd commit changes encoding to utf-8 and commits. This commit does not change crlf to lf in index, even though new format is utf-8 which is text based for git. This is the crux of problem. I have attached all commands i ran on windows while creating the repo. I tried to capture all information that i could give. Please have a look. It might be useful. Finally, thank you Torsten for giving your time to the problem. Really appreciate it. On Tue, Nov 14, 2017 at 10:39 PM, Torsten Bögershausen wrote: > (Back to the beginning) > > You have a file ApplicationManifest.xml > It is encoded in UTF-16 (and has CRLF) > > You convert it into UTF-8 > The file has still CRLF (in the worktree) > > Now you add it and make a commit. > Under both Linux and Windows you have "text=auto". > > I assume that you have efficiently core.eol=lf under Linux > and core.eol=crlf on Windows. > > (That is the default, when you don't change anything) > > Now, what happens to the CRLF? > If you commit the file, it will be stored with LF in the index, > on both systems. > On checkout, Windows will convert them into CRLF, but Linux will not. > > That why you see >>On linux, during committing i get warning : warning: CRLF will be >>replaced by LF in …file_name.. > > All in all there is nothing wrong, at least as I see it. > > The question remains: > Do you need CRLF in Linux ? > Probably not, but if yes, plase add a line > > *.xml text eol=crlf > > to your > .gitattributes > > Otherwise your .gitconfig looks good to me. > > > > > > git_encoding_repo>git config --global core.safecrlf true git_encoding_repo>git config core.autocrlf false git_encoding_repo>git config --get core.autocrlf false git_encoding_repo>cat .gitattributes # Set the default behavior, in case people don't have core.autocrlf set. * text=auto *.vcxproj eol=crlf *.sh eol=lf # Denote all files that are truly binary and should not be modified. *.exe binary *.dll binary *.pdb binary *.ico binary *.png binary *.jpg binary git_encoding_repo>git status On branch master No commits yet Untracked files: (use "git add ..." to include in what will be committed) .gitattributes file_name.txt nothing added to commit but untracked files present (use "git add" to track) git_encoding_repo>git ls-files --eol file_name.txt git_encoding_repo>git add . git_encoding_repo>git status On branch master No commits yet Changes to be committed: (use "git rm --cached ..." to unstage) new file: .gitattributes new file: file_name.txt git_encoding_repo>git ls-files --eol file_name.txt i/-text w/-text attr/text=auto file_name.txt git_encoding_repo>git commit -m "Commit utf-16le encoded file which has crlf." [master (root-commit) 91fe3bd] Commit utf-16le encoded file which has crlf. 2 files changed, 13 insertions(+) create mode 100644 .gitattributes create mode 100644 file_name.txt ## At this time, i changed file encoding to utf-8. git_encoding_repo>git status On branch master nothing to commit, working tree clean git_encoding_repo>git add -p Only binary files changed. git_encoding_repo>git status On branch master Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: file_name.txt no changes added to commit (use "git add" and/or "git commit -a") git_encoding_repo>git add file_name.txt git_encoding_repo>git status On branch master Changes to be committed: (use "git reset HEAD ..." to unstage) modified: file_name.txt git_encoding_repo>git commit -m "Change encoding of file to utf-8" [master 179c27b] Change encoding of file to utf-8 1 file changed, 0 insertions(+), 0 deletions(-) rewrite file_name.txt (100%) git_encoding_repo>git remote add origin https://github.com/ashishnegi/git_encoding.git git_encoding_repo>git push -u origin master Counting objects: 7, done. Delta compression using up to 12 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (7/7), 837 bytes | 837.00 KiB/s, done. Total 7 (delta 0), reused 0 (delta 0) To https://github.com/ashishnegi/git_encoding.git * [new branch] master -> master Branch master set up to track remote branch master from origin. git_encoding_repo>git ls-files --eol file_name.txt i/crlf w/crlf attr/text=auto file_name.txt git_encoding_repo>
Re: Changing encoding of a file : What should happen to CRLF in file ?
(Back to the beginning) You have a file ApplicationManifest.xml It is encoded in UTF-16 (and has CRLF) You convert it into UTF-8 The file has still CRLF (in the worktree) Now you add it and make a commit. Under both Linux and Windows you have "text=auto". I assume that you have efficiently core.eol=lf under Linux and core.eol=crlf on Windows. (That is the default, when you don't change anything) Now, what happens to the CRLF? If you commit the file, it will be stored with LF in the index, on both systems. On checkout, Windows will convert them into CRLF, but Linux will not. That why you see >On linux, during committing i get warning : warning: CRLF will be >replaced by LF in …file_name.. All in all there is nothing wrong, at least as I see it. The question remains: Do you need CRLF in Linux ? Probably not, but if yes, plase add a line *.xml text eol=crlf to your .gitattributes Otherwise your .gitconfig looks good to me.
Re: Changing encoding of a file : What should happen to CRLF in file ?
On 2017-11-14 17:13, Ashish Negi wrote: > Running the command gives me : > > git ls-files --eol file_name > i/-text w/-text attr/text=auto file_name > That is strange to me: According to that, Git would treat the file as text=auto. And the content is "not next", so there is no need to convert. Do you have configured any filters ? Is this a public repo ? if not: Is there any chance that you provide a public example, so that we can have a look ?
Re: Changing encoding of a file : What should happen to CRLF in file ?
After changing the encoding of file to utf-8, same command gives: git ls-files --eol file_name i/lfw/crlf attr/text=auto ApplicationManifest.xml On Tue, Nov 14, 2017 at 9:43 PM, Ashish Negi wrote: > Running the command gives me : > > git ls-files --eol file_name > i/-text w/-text attr/text=auto file_name
Re: Changing encoding of a file : What should happen to CRLF in file ?
Running the command gives me : git ls-files --eol file_name i/-text w/-text attr/text=auto file_name
Re: Changing encoding of a file : What should happen to CRLF in file ?
On 2017-11-14 13:31, Ashish Negi wrote: > Hello > > I have a cross platform project. I have a utf-16 file in it. > I changed its encoding to urf-8 and committed. When i pulled the file > in Linux, it shows that file is modified. This means that the commit > which changed the encoding does not convert crlf to lf, when new > format is text based (utf-8). > > Steps to reproduce: > >In windows : > > Change encoding of file from utf-16 to utf-8. > Commit the change. > >In linux: > > Pull your branch. > You will see the issue of file being shown as modified even though > you have not used it. > > > If i change the file encoding in linux and commit. Then if i do git > pull in windows, i don't see the issue. > In linux, during committing i get warning : warning: CRLF will be > replaced by LF in …file_name.. > > Here are my configuration : > > >> git config --global --get core.autocrlf > > false > > >> git config --get core.autocrlf > > false > > > >> cat E:\work\WindowsFabric\.gitattributes > > > # Set the default behavior, in case people don't have core.autocrlf set. > > * text=auto > *.vcxproj eol=crlf > *.sh eol=lf > > # Denote all files that are truly binary and should not be modified. > *.exe binary > *.dll binary > *.pdb binary > *.ico binary > *.png binary > *.jpg binary > > >> git --version > git version 2.14.2.windows.2 > > > I played around with core.autocrlf values like true and native, but > that didn't help. > > The behavior is inconsistent across platforms, and windows version is > giving me problem. > > Can someone suggest right settings for this ? Or is this a bug in Git. > I don't think it is a bug :-) What does git ls-files --eol …file_name give you under Windows ?
Changing encoding of a file : What should happen to CRLF in file ?
Hello I have a cross platform project. I have a utf-16 file in it. I changed its encoding to urf-8 and committed. When i pulled the file in Linux, it shows that file is modified. This means that the commit which changed the encoding does not convert crlf to lf, when new format is text based (utf-8). Steps to reproduce: In windows : Change encoding of file from utf-16 to utf-8. Commit the change. In linux: Pull your branch. You will see the issue of file being shown as modified even though you have not used it. If i change the file encoding in linux and commit. Then if i do git pull in windows, i don't see the issue. In linux, during committing i get warning : warning: CRLF will be replaced by LF in …file_name.. Here are my configuration : > git config --global --get core.autocrlf false > git config --get core.autocrlf false > cat E:\work\WindowsFabric\.gitattributes # Set the default behavior, in case people don't have core.autocrlf set. * text=auto *.vcxproj eol=crlf *.sh eol=lf # Denote all files that are truly binary and should not be modified. *.exe binary *.dll binary *.pdb binary *.ico binary *.png binary *.jpg binary > git --version git version 2.14.2.windows.2 I played around with core.autocrlf values like true and native, but that didn't help. The behavior is inconsistent across platforms, and windows version is giving me problem. Can someone suggest right settings for this ? Or is this a bug in Git. Thanks