Re: [Bug-tar] Regarding untaring tar files containing symlinks
On Sun, Jul 6, 2014 at 7:56 PM, Linda A. Walsh g...@tlinx.org wrote: Ah I have my cygwin cygdrive path prefix set to / mount -p Prefix Type Flags / user binmode You should be able to set it once: mount -c / I also have my cygwin installed in C:\ not C:\cygwin. With those two changes, my cygwin and windows paths are identical (except for backslash or forward slash). Forgot about that -- have had it in root for over 10 years...since I first installed cygwin. FWIW -- I **might** call this a minor bug (or feature deficit). If you ahve native mode set for the symlinks, then, I would argue that it shouldn't insert a cygwin/c in the path... Windows won't let you create a symlink to a non-existent path (you can on unix or linux), so when it fails for any reason -- cygwin falls back to storing the link in a file. Thanks for all your suggestions. With Regards, Amit Kapila.
Re: [Bug-tar] Regarding untaring tar files containing symlinks
Ah I have my cygwin cygdrive path prefix set to / mount -p Prefix Type Flags / user binmode You should be able to set it once: mount -c / I also have my cygwin installed in C:\ not C:\cygwin. With those two changes, my cygwin and windows paths are identical (except for backslash or forward slash). Forgot about that -- have had it in root for over 10 years...since I first installed cygwin. FWIW -- I **might** call this a minor bug (or feature deficit). If you ahve native mode set for the symlinks, then, I would argue that it shouldn't insert a cygwin/c in the path... Windows won't let you create a symlink to a non-existent path (you can on unix or linux), so when it fails for any reason -- cygwin falls back to storing the link in a file.
Re: [Bug-tar] Regarding untaring tar files containing symlinks
Amit Kapila wrote Please extract in order as mentioned to ensure that symlink folder gets extracted first. Please check contents of C:\Data\pg_tblspc, ideally it should contain symlink to C:\tbs. --- It appears to work as you are wanting it to (console output below). But I also note that if they are created in the wrong order (something that would work on linux). -- then I get the behavior you mention: instead of a symlink, I get a file containing text: C:\data\pg_tblspchexdump -C 16386 21 3c 73 79 6d 6c 69 6e 6b 3e ff fe 63 00 3a 00 |!symlink..c.:.| 0010 5c 00 74 00 62 00 73 00 00 00|\.t.b.s...| 001a Working case follows: mkdir tbs / cd tbs /tbs tar xvaf /tmp/tars/16386.tar PG_9.5_201406292/ PG_9.5_201406292/12135/ PG_9.5_201406292/12135/16387 /tbs ll -R .: total 0 drwxrwxr-x+ 1 0 Jul 4 21:47 PG_9.5_201406292/ ./PG_9.5_201406292: total 0 drwxrwxr-x+ 1 0 Jul 4 21:47 12135/ ./PG_9.5_201406292/12135: total 0 -rw-rw-r-- 1 0 Jul 4 21:47 16387 /tbs cd .. / mkdir data / cd data /data tar xaf /tmp/tars/base.tar /data ll total 47 -rw-rw-r--+ 1 4 Jul 4 00:01 PG_VERSION -rw---+ 1 206 Jul 4 21:47 backup_label drwxrwxr-x+ 1 0 Jul 4 00:02 base/ drwxrwxr-x+ 1 0 Jul 5 00:18 global/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_clog/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_dynshmem/ -rw-rw-r--+ 1 4352 Jul 4 00:03 pg_hba.conf -rw-rw-r--+ 1 1678 Jul 4 00:01 pg_ident.conf drwxrwxr-x+ 1 0 Jul 4 00:01 pg_llog/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_multixact/ drwxrwxr-x+ 1 0 Jul 4 21:45 pg_notify/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_replslot/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_serial/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_snapshots/ drwxrwxr-x+ 1 0 Jul 4 21:45 pg_stat/ drwxrwxr-x+ 1 0 Jul 4 21:47 pg_stat_tmp/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_subtrans/ drwxrwxr-x+ 1 0 Jul 4 21:46 pg_tblspc/ drwxrwxr-x+ 1 0 Jul 4 00:01 pg_twophase/ drwxrwxr-x+ 1 0 Jul 4 00:17 pg_xlog/ -rw-rw-r--+ 190 Jul 4 00:01 postgresql.auto.conf -rw-rw-r--+ 1 21994 Jul 4 00:05 postgresql.conf /data ll pg_tblspc/ total 0 lrwxrwxrwx 1 4 Jul 4 21:46 16386 - /tbs/ /data cd pg_tblspc/ /data/pg_tblspc ls 16386@ /data/pg_tblspc cd 16386/ /data/pg_tblspc/16386 ls PG_9.5_201406292/ /data/pg_tblspc/16386 cd .. /data/pg_tblspc cmd Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\data\pg_tblspcdir Volume in drive C is System Disk Volume Serial Number is E889-68E4 Directory of C:\data\pg_tblspc 07/04/2014 09:46 PMDIR . 07/04/2014 09:46 PMDIR .. 07/04/2014 09:46 PMSYMLINKD 16386 [C:\tbs] 0 File(s) 0 bytes 3 Dir(s) 218,077,290,496 bytes free C:\data\pg_tblspc - This might have to do with the version of cygwin you have. You might make sure your cygwin is up to date -- as I vaguely remember somewhat older versions of 'cygwin' didn't create actual windows symlinks and hardlinks but created dummy files with the information in them (like you are seeing). But that changed... maybe 2-4 years ago? If you are running on a 64-bit windows, there is also a 64-bit cygwin that runs in native mode that gives some advantages over 32-bit cygwin on a 64-bit machine (like 32-bit processes can't see the real C:\windows\system32, but are redirected to C:\windows\syswow64 syswow64 = windows[32] on windows64) you can tell which version you are running by running uname -a: CYGWIN_NT-6.1 Athenae 1.7.30(0.272/5/3) 2014-05-23 10:36 x86_64 Cygwin 32-bit cygwin would say: CYGWIN_NT-6.1-WOW64 Athenae 1.7.28(0.271/5/3) 2014-02-09 21:06 i686 Cygwin
Re: [Bug-tar] Regarding untaring tar files containing symlinks
On Sat, Jul 5, 2014 at 1:14 PM, Linda A. Walsh g...@tlinx.org wrote: Amit Kapila wrote Please extract in order as mentioned to ensure that symlink folder gets extracted first. Please check contents of C:\Data\pg_tblspc, ideally it should contain symlink to C:\tbs. --- It appears to work as you are wanting it to (console output below). Thats great. But I also note that if they are created in the wrong order (something that would work on linux). -- then I get the behavior you mention: instead of a symlink, I get a file containing text: C:\data\pg_tblspchexdump -C 16386 21 3c 73 79 6d 6c 69 6e 6b 3e ff fe 63 00 3a 00 |!symlink..c.:.| 0010 5c 00 74 00 62 00 73 00 00 00|\.t.b.s...| 001a I am always getting file having contents as above. This might have to do with the version of cygwin you have. You might make sure your cygwin is up to date -- as I vaguely remember somewhat older versions of 'cygwin' didn't create actual windows symlinks and hardlinks but created dummy files with the information in them (like you are seeing). But that changed... maybe 2-4 years ago? If you are running on a 64-bit windows, there is also a 64-bit cygwin that runs in native mode that gives some advantages over 32-bit cygwin on a 64-bit machine (like 32-bit processes can't see the real C:\windows\system32, but are redirected to C:\windows\syswow64 syswow64 = windows[32] on windows64) you can tell which version you are running by running uname -a: CYGWIN_NT-6.1 Athenae 1.7.30(0.272/5/3) 2014-05-23 10:36 x86_64 Cygwin 32-bit cygwin would say: CYGWIN_NT-6.1-WOW64 Athenae 1.7.28(0.271/5/3) 2014-02-09 21:06 i686 Cygwin I am working Windows 64-bit and uname -a gives below info on my m/c. CYGWIN_NT-6.2 WIN-J9UR5I3UTSM 1.7.30(0.272/5/3) 2014-05-23 10:36 x86_64 Cygwin I have downloaded cygwin for windows64-bit from below link: https://www.cygwin.com/install.html Do I need to update my cygwin installation, any recommendation for same? With Regards, Amit Kapila.
Re: [Bug-tar] Regarding untaring tar files containing symlinks
On Sat, Jul 5, 2014 at 11:24 PM, Linda A. Walsh g...@tlinx.org wrote:
How are you setting your cygwin variable?
I was using export command:
export CYGWIN=system nodosfilewarning winsymlinks:native export
It needs to be set before
any cygwin processes start.
I.e. in controlpanel-system-, advanced, environment variables.
Best to set in System variables so all users will be sure to have it set.
Okay, I have tried by this way as well, but results are same.
Please see results:
edb@WIN-J9UR5I3UTSM /data
$ ls -l pg_tblspc/
total 1
lrwxrwxrwx 1 edb None 15 Jul 5 10:16 16386 - /cygdrive/c/tbs
Now here point to note is that it is showing the link, even
though it is just a symbol file.
edb@WIN-J9UR5I3UTSM /data
$ cd pg_tblspc/
edb@WIN-J9UR5I3UTSM /data/pg_tblspc
$ ls -l
total 1
lrwxrwxrwx 1 edb None 15 Jul 5 10:16 16386 - /cygdrive/c/tbs
edb@WIN-J9UR5I3UTSM /data/pg_tblspc
$ cd 16386
-bash: cd: 16386: No such file or directory
edb@WIN-J9UR5I3UTSM /data/pg_tblspc
$ cmd
Microsoft Windows [Version 6.2.9200]
(c) 2012 Microsoft Corporation. All rights reserved.
C:\cygwin64\data\pg_tblspcls -l
ls -l
total 1
lrwxrwxrwx 1 edb None 15 Jul 5 10:16 16386 - /cygdrive/c/tbs
C:\cygwin64\data\pg_tblspcdir
dir
Volume in drive C has no label.
Volume Serial Number is F049-7E4C
Directory of C:\cygwin64\data\pg_tblspc
07/05/2014 10:16 AMDIR .
07/05/2014 10:16 AMDIR ..
0 File(s) 0 bytes
2 Dir(s) 1,508,360,192 bytes free
C:\cygwin64\data\pg_tblspc
I have noticed one minor difference between the 2 versions of
CYGWIN as below, though it seems mine is more latest, but
don't know if it can cause such a behaviour difference.
CYGWIN_NT-6.1 Athenae
CYGWIN_NT-6.2 WIN-J9UR5I3UTSM
At this moment, it is quite hard to believe that I am doing any
basic mistake due to which I am not able to see behaviour as yours,
do you think installation or something else needs to be taken care
in this regard?
With Regards,
Amit Kapila.
Amit Kapila wrote:
On Sat, Jul 5, 2014 at 1:14 PM, Linda A. Walsh g...@tlinx.org mailto:
g...@tlinx.org wrote:
Amit Kapila wrote
Please extract in order as mentioned to ensure that symlink folder
gets extracted first.
Please check contents of C:\Data\pg_tblspc, ideally it should contain
symlink to C:\tbs.
---
It appears to work as you are wanting it to (console output below).
Thats great.
But I also note that if they are created in the wrong order (something
that would work on linux). -- then I get the behavior you mention:
instead of a symlink, I get a file containing text:
C:\data\pg_tblspchexdump -C 16386
21 3c 73 79 6d 6c 69 6e 6b 3e ff fe 63 00 3a 00
|!symlink..c.:.|
0010 5c 00 74 00 62 00 73 00 00 00
|\.t.b.s...|
001a
I am always getting file having contents as above.
This might have to do with the version of cygwin you have.
You might make sure your cygwin is up to date -- as I vaguely remember
somewhat older versions of 'cygwin' didn't create actual windows
symlinks and hardlinks but created dummy files with the information
in them (like you are seeing). But that changed... maybe 2-4 years
ago?
If you are running on a 64-bit windows, there is also a 64-bit
cygwin that runs in native mode that gives some advantages over
32-bit cygwin on a 64-bit machine (like 32-bit processes can't
see the real C:\windows\system32, but are redirected to
C:\windows\syswow64
syswow64 = windows[32] on windows64)
you can tell which version you are running by running uname -a:
CYGWIN_NT-6.1 Athenae 1.7.30(0.272/5/3) 2014-05-23 10:36 x86_64 Cygwin
32-bit cygwin would say:
CYGWIN_NT-6.1-WOW64 Athenae 1.7.28(0.271/5/3) 2014-02-09 21:06 i686
Cygwin
I am working Windows 64-bit and uname -a gives below info
on my m/c.
CYGWIN_NT-6.2 WIN-J9UR5I3UTSM 1.7.30(0.272/5/3) 2014-05-23 10:36 x86_64
Cygwin
I have downloaded cygwin for windows64-bit from below link:
https://www.cygwin.com/install.html
Do I need to update my cygwin installation, any recommendation
for same?
With Regards,
Amit Kapila.
Re: [Bug-tar] Regarding untaring tar files containing symlinks
Amit Kapila wrote: On Mon, Jun 30, 2014 at 8:10 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de mailto:joerg.schill...@fokus.fraunhofer.de wrote: Amit Kapila amit.kapil...@gmail.com mailto:amit.kapil...@gmail.com wrote: On Mon, Jun 30, 2014 at 6:42 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de mailto:joerg.schill...@fokus.fraunhofer.de wrote: It will create copies as Win-DOS does not support symlnks. Maybe not, but Windows has since Vista. Maybe you should give up Win-DOS? For my usecase, I need it to maintain symlinks even after it gets untarred. I have noticed that WinRar is able to maintain symlinks after extraction. I am not sure how this should work, given the fact that Win-DOS does not correctly support symlinks. Internally the software uses junction points as mentioned in below link to create symlinks. http://www.codeproject.com/KB/winsdk/junctionpoints.aspx Technically, it should use 'mklink' which are windows symlinks. Junctions are similar but are used to do mounts of file systems. For most practical purposes, they should work 'fine'... Cygwin walks all over Window's mountpoints and refuses to honor them as mountpoint, so untaring things under cygwin will overwrite mount points... the mountd/linkd (types of junctions) are honored in one case in cygwin, but not in the other. Cygwin will use the native windows symlinks when requested(an ENV var), to create symlinks. Given that Windows (What is win-dos? never heard of such -- I hope you don't believe windows = dos -- as that'd be really wrong) is windows, and it has symlinks, I don't see how you can claim windows symlinks don't work. If you want to claim they are not POSIX, that's a bit disingenuous as Windows isn't POSIX. Best bet might be to use cygwin to untar your files... it can create win-symlinks. As for making multiple copies of files when untarring them? That's not the same as creating links. My font dir is 13.5 Meg if the links are ignored while only ~4-5M if linked. (actually use hardlinks... which, BTW, windows also has).
Re: [Bug-tar] Regarding untaring tar files containing symlinks
Amit Kapila amit.kapil...@gmail.com wrote: I send already a hint on how to do this using star. Thanks for your suggestion. I have tried to search on net to download this utility and found below link: ftp://ftp.ffokus.gmd.de/pub/unix/cdrecord/alpha/win32/ I am not able to get from above link, it says location doesn't exist. Could you let me know from where can I get this utility. It is on Sourceforge now. One more question, after using copysymlinks, will it retain the symlinks in Extracted data(folder)? It will create copies as Win-DOS does not support symlnks. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [Bug-tar] Regarding untaring tar files containing symlinks
On Mon, Jun 30, 2014 at 6:42 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Amit Kapila amit.kapil...@gmail.com wrote: One more question, after using copysymlinks, will it retain the symlinks in Extracted data(folder)? It will create copies as Win-DOS does not support symlnks. For my usecase, I need it to maintain symlinks even after it gets untarred. I have noticed that WinRar is able to maintain symlinks after extraction. Thanks for all your help. With Regards, Amit Kapila.
Re: [Bug-tar] Regarding untaring tar files containing symlinks
Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Jun 30, 2014 at 6:42 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Amit Kapila amit.kapil...@gmail.com wrote: One more question, after using copysymlinks, will it retain the symlinks in Extracted data(folder)? It will create copies as Win-DOS does not support symlnks. For my usecase, I need it to maintain symlinks even after it gets untarred. I have noticed that WinRar is able to maintain symlinks after extraction. I am not sure how this should work, given the fact that Win-DOS does not correctly support symlinks. See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa363866%28v=vs.85%29.aspx On the other side, are you sure that winrar supports tar archives? You may like to check to list this archive: http://sourceforge.net/projects/s-tar/files/testscripts/pax-big-10g.tar.bz2/download and test whether the listing looks correctly, similar to this: 10737418240 -rw--- jes/glone Jun 15 23:18 2002 10g 0 -rw-r--r-- jes/glone Jun 15 16:53 2002 file star: 1048576 blocks + 3072 bytes (total of 10737421312 bytes = 10485763.00k). I would guess that winrar is unable to list both filenames. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [Bug-tar] Regarding untaring tar files containing symlinks
On Mon, Jun 30, 2014 at 8:10 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Jun 30, 2014 at 6:42 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: It will create copies as Win-DOS does not support symlnks. For my usecase, I need it to maintain symlinks even after it gets untarred. I have noticed that WinRar is able to maintain symlinks after extraction. I am not sure how this should work, given the fact that Win-DOS does not correctly support symlinks. Internally the software uses junction points as mentioned in below link to create symlinks. http://www.codeproject.com/KB/winsdk/junctionpoints.aspx See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa363866%28v=vs.85%29.aspx On the other side, are you sure that winrar supports tar archives? No, it won't support tar format. I want to mention about the case when archive is created using WinRAR. Even though WinRAR can extract symlinks if archive is created originally with WinRAR, it doesn't serve my usecase because it can't do so if I try with archive created with tar which seems to be obvious. I have mentioned about WinRAR, just to highlight the case that on Windows symlinks can be extracted, so I think ideally it should be possible for tar to be extracted even if it contains symlinks by some other s/w on Windows. You may like to check to list this archive: http://sourceforge.net/projects/s-tar/files/testscripts/pax-big-10g.tar.bz2/download and test whether the listing looks correctly, similar to this: 10737418240 -rw--- jes/glone Jun 15 23:18 2002 10g 0 -rw-r--r-- jes/glone Jun 15 16:53 2002 file star: 1048576 blocks + 3072 bytes (total of 10737421312 bytes = 10485763.00k). I would guess that winrar is unable to list both filenames. Output on extracting with WinRAR E:\basebackupls -l total 5 -rw-r--r--1 edb Administ0 Jun 16 2002 10g -rw-r--r--1 edb Administ 110 Jun 30 20:20 @PaxHeader -rw-r--r--1 edb Administ 458 Jun 26 19:02 RM31185.txt -rw-r--r--1 edb Administ 7808 Jun 17 2002 pax-big-10g.tar.bz2 Output on extracting with tar.exe E:\basebackuptar.exe -xvf pax-big-10g.tar.bz2 tar.exe: Record size = 2 blocks tar.exe: Extended header size=10737418240 is out of range 0..2147483647 10g tar.exe: Exiting with failure status due to previous errors With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Re: [Bug-tar] Regarding untaring tar files containing symlinks
Amit Kapila amit.kapil...@gmail.com wrote: I wanted to know that is there any tool on Windows which can extract from tar when it contains symlinks. I send already a hint on how to do this using star. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [Bug-tar] Regarding untaring tar files containing symlinks
Amit Kapila amit.kapil...@gmail.com wrote: While working on one software, I face the requirement to untar a tar file on windows and it contains symlinks as well. While untaring it using tar -xvf abc.tar, it gives me below error for symlinks: tar.exe: xyz/1234: Cannot create symlink to `E:\\sym_loc': Not a directory The directory sym_loc already exists, is this a known limitation of tar on Windows? Win-DOS does not support symlinks. Even newer versions that claim to support symlinks make a difference between links to files and links to directories. If you are not able to live with the Cygwin symlink emulation, I recommend to use star and to tell it to make copies by using the option -copysymlinks Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [Bug-tar] Regarding untaring tar files containing symlinks
On Fri, Jun 27, 2014 at 3:51 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Win-DOS does not support symlinks. Even newer versions that claim to support symlinks make a difference between links to files and links to directories. If you are not able to live with the Cygwin symlink emulation, I recommend to use star and to tell it to make copies by using the option -copysymlinks Is it similar to what tar.exe -h option which can be used when we form the tar on folders/files? The important point here is that the tar is created by using standard tar format described in link below: http://en.wikipedia.org/wiki/Tar_(computing) While forming tar, it uses link indicator as '2' as described in above link and my requirement is to untar that on windows; the tar is created using above format on Windows. With Regards, Amit Kapila.
