Re: SVN: PCSWMM Use
On 9/14/2021 9:26 AM, Jason Kimmet wrote: (snipped to avoid top-posting) *From:* David Chapman *Sent:* Thursday, September 9, 2021 4:29 PM *To:* Jason Kimmet ; users@subversion.apache.org *Subject:* Re: SVN: PCSWMM Use On 9/9/2021 1:53 PM, Jason Kimmet wrote: Subversion Users, We are working on a large stormwater modeling program where we will be keeping track of a lot of PCSWMM file type models. We are exploring version management options and have come across the SVN community. I understand this is great for text-style files, however, I would like to know if you have seen users utilize this for engineering plans such as PCSWMM? Many version control systems, Subversion included, work best when the new version of a file looks much the same as the previous version. Otherwise you could be storing multiple full versions of the files. Some features of Subversion (like "show lines that changed") aren't available for binary files, but if the changes in a binary file are limited to specific sections, you still get space-saving benefits in the repository; see http://help.collab.net/index.jsp?topic=/faq/svnbinary.html <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhelp.collab.net%2Findex.jsp%3Ftopic%3D%2Ffaq%2Fsvnbinary.html=04%7C01%7Cjkimmet%40allworldmail.com%7C0ff1c55f5f0a4b082c3508d973d8e298%7C59b582c195624de2a434b64a585e99be%7C1%7C1%7C637668197637323704%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000=s%2FpDi4MrtvGm8sAsF0ZyPJTQ0QJNdbMw%2FjFjVAoFm%2BM%3D=0> and https://svnbook.red-bean.com/en/1.7/svn.forcvs.binary-and-trans.html <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvnbook.red-bean.com%2Fen%2F1.7%2Fsvn.forcvs.binary-and-trans.html=04%7C01%7Cjkimmet%40allworldmail.com%7C0ff1c55f5f0a4b082c3508d973d8e298%7C59b582c195624de2a434b64a585e99be%7C1%7C1%7C637668197637333663%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000=C1%2B%2BHu5%2FQzQ4cJW6Fn9ArCS4uBeU4mWUfZF%2BG0%2BYphw%3D=0> (both links are old, but binary file handling shouldn't be worse now than then). A quick search didn't tell me anything about the PCSWMM format; how much of a file changes between model runs? Is it text or binary? Are the files very large (gigabytes or more)? Are they inputs to software, or outputs? Generally it is assumed that output files can be recreated given the full input configuration, so program outputs are often left out of the repository. -- David chapmandcchap...@acm.org <mailto:dcchap...@acm.org> Chapman Consulting -- San Jose, CA EDA Software Developer, Expert Witness www.chapman-consulting-sj.com <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.chapman-consulting-sj.com%2F=04%7C01%7Cjkimmet%40allworldmail.com%7C0ff1c55f5f0a4b082c3508d973d8e298%7C59b582c195624de2a434b64a585e99be%7C1%7C1%7C637668197637333663%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000=JV2Vs3t2okpH3UziGFo7iUQzoRdBxwheToMdIAsTzzk%3D=0> 2018-2019 Chair, IEEE Consultants' Network of Silicon Valley David, PCSWMM files normally won't exceed 1 gigabyte. Rest of answers: * how much of a file changes between model runs? Really depends … ranging from if perhaps a global model parameter edit is done during an iteration, or just changes to a study area like a detention pond. * Are the files text or binary? Text * Are they inputs to software, or outputs? Input file gets modified during design/analysis by the modeler. Hopefully, this helps. thanks, ** *Jason Kimmet* www.allworldpm.com <https://www.allworldpm.com> You should be able to use Subversion. My normal use case is for much smaller files (software source code) but all you need is disk space, and any version control software will have to deal with the differences between file versions. I presume that users are not typing a gigabyte each time, and that the PCSWMM files are assembled by a tool from other data. In theory you could store the inputs to whatever tool is generating a particular PCSWMM file version, but that may be too much of a change from your present workflow. The fact that they are text files is good too. I work in the semiconductor industry, and the binary files our tools generate (GDS and OASIS format, in case anyone cares) tend to have major differences between versions. We'd end up storing the entire file each time, even if the files are functionally equivalent - in large part, the formats are order independent, and tool A's ordering convention tends to differ from Tool B's convention. That is a problem when a chip-level GDS file can be a terabyte in size! -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA EDA Software D
Re: SVN: PCSWMM Use
On 9/9/2021 1:53 PM, Jason Kimmet wrote: Subversion Users, We are working on a large stormwater modeling program where we will be keeping track of a lot of PCSWMM file type models. We are exploring version management options and have come across the SVN community. I understand this is great for text-style files, however, I would like to know if you have seen users utilize this for engineering plans such as PCSWMM? Many version control systems, Subversion included, work best when the new version of a file looks much the same as the previous version. Otherwise you could be storing multiple full versions of the files. Some features of Subversion (like "show lines that changed") aren't available for binary files, but if the changes in a binary file are limited to specific sections, you still get space-saving benefits in the repository; see http://help.collab.net/index.jsp?topic=/faq/svnbinary.html and https://svnbook.red-bean.com/en/1.7/svn.forcvs.binary-and-trans.html (both links are old, but binary file handling shouldn't be worse now than then). A quick search didn't tell me anything about the PCSWMM format; how much of a file changes between model runs? Is it text or binary? Are the files very large (gigabytes or more)? Are they inputs to software, or outputs? Generally it is assumed that output files can be recreated given the full input configuration, so program outputs are often left out of the repository. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA EDA Software Developer, Expert Witness www.chapman-consulting-sj.com 2018-2019 Chair, IEEE Consultants' Network of Silicon Valley
Re: svn.haxx.se is going away
On 11/4/2020 12:32 PM, Mark Phippard wrote:
Just a general fyi ... I went to https://svn.haxx.se/
<https://svn.haxx.se/> today to search the lists and noticed there is
a banner on the site saying it is going offline forever soon.
I am not sure what the ramifications will be as I know there are a lot
of historical links in the docs and site but I guess it is what it is.
--
Thanks
Mark Phippard
Daniel Stenberg is in the process of moving the Curl Web site from
https://curl.haxx.se/ to https://www.curl.se/. I'm not sure why
https://svn.haxx.se/ is not following along (there is no
https://svn.curl.se as of a few minutes ago), but then again I'm just a
Curl user, not a dev. Curl development is hosted on Github, so maybe he
lost interest in a Subversion archive?
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
EDA Software Developer, Expert Witness
www.chapman-consulting-sj.com
2018-2019 Chair, IEEE Consultants' Network of Silicon Valley
Re: issue with relative externals after a rename
On 3/28/2018 9:37 AM, Bo Berglund wrote: Just curious since I am working on setting up our svn repositories migrated from CVS. How does one "lock down" tags to disallow further commits? In CVS that was built-in but svn works differently... Bo B. Put a check for the commit path(s) in the pre-commit hook. If a tag directory is in the commit list, return an error. http://svnbook.red-bean.com/nightly/en/svn.ref.reposhooks.pre-commit.html -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA EDA Software Developer, Expert Witness www.chapman-consulting-sj.com 2018 Chair, IEEE Consultants' Network of Silicon Valley
Re: subversion not detecting apr and apr-utils libraries
On 3/6/2018 6:54 PM, Guido granda muñoz wrote: Hello, I'm trying to install subversion 1.9.7 in Ubuntu 16.04 using its source code. When I do ./configure ./configure --with-apr=/usr/lib/apache2 --with-apr-util=/usr/lib/apache2 , I get the following error: configure: Configuring Subversion 1.9.7 configure: creating config.nice checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking if gcc accepts -std=c90... yes checking if gcc accepts -w... yes checking if gcc accepts -Werror=unknown-warning-option... no checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking if g++ accepts -std=c++98... yes checking if g++ accepts -w... yes checking if g++ accepts -Werror=unknown-warning-option... no checking how to run the C preprocessor... gcc -E checking for a sed that does not truncate output... /bin/sed checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking target system type... x86_64-unknown-linux-gnu checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking whether ln -s works... yes checking for a BSD-compatible install... /usr/bin/install -c configure: Apache Portable Runtime (APR) library configuration checking for APR... configure: error: the --with-apr parameter is incorrect. It must specify an install prefix, a build directory, or an apr-config file. I'am pretty sure those provided locations are right because I used the find / -name "apache2" command to obtain them. Please tell me what is wrong. Kind Regards, Did you look for "apr-config" or just "apache2"? See https://unix.stackexchange.com/questions/41910/how-to-make-apr-available-for-subversion-install and of course the reference https://svn.apache.org/repos/asf/subversion/trunk/INSTALL You can specify the location of the "apr-config" file directly and it should proceed from there. Disclaimer: I haven't tried to build Subversion in a long time, and I use CentOS, not Ubuntu... -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA EDA Software Developer, Expert Witness www.chapman-consulting-sj.com 2018 Chair, IEEE Consultants' Network of Silicon Valley
Re: svnignore?
On 12/29/2017 8:56 PM, Branko Čibej wrote: On 30.12.2017 00:27, Daniel Shahaf wrote: Ryan Schmidt wrote on Fri, 29 Dec 2017 14:05 -0600: On Dec 29, 2017, at 12:55, Bo Berglund wrote: I guess it is not valid to split the list on several lines in they config file? I've never heard of that being possible. It doesn't seem to be (to my surprise), but the following does work: [miscellany] gi1 = foo1 foo2 gi2 = bar1 bar2 global-ignores = %(gi1)s %(gi2)s That's equivalent to [miscellany] global-ignores = foo1 foo2 bar1 bar2 . Yes and this also works: [miscellany] global-ignores = foo1 foo2 bar1 bar2 Note the space at the beginning of the continuation lines. The syntax of Subversion config files is compatible with https://docs.python.org/2/library/configparser.html Is this documented in The Book? I was looking at http://svnbook.red-bean.com/nightly/en/svn.advanced.confarea.html today and it made no mention of this. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA EDA Software Developer, Expert Witness www.chapman-consulting-sj.com
Re: svnignore?
On 12/29/2017 10:55 AM, Bo Berglund wrote: On Fri, 29 Dec 2017 19:05:06 +0100, Bo Berglund <bo.bergl...@gmail.com> wrote: On Fri, 29 Dec 2017 09:24:10 -0800, David Chapman <dcchap...@acm.org> wrote: But if I add the content of my cvsignore file to the config file as a whitespace separated list on one line that line becomes VERY log (over 600 chars). Can I put the info in separate lines? It turns out that enabling this option by removing the # character makes SmartSvn barf when I want to check out a project! So it is necessary also to remove all whitespace *before* the global-ignores text. After this whitespace is permissible. This is how my file looks now for ignores (the ignore globs are all on one line, the newsreader has added the line breaks): [miscellany] ### Set global-ignores to a set of whitespace-delimited globs ### which Subversion will ignore in its 'status' output, and ### while importing or adding files and directories. ### '*' matches leading dots, e.g. '*.rej' matches '.foo.rej'. global-ignores = *.o *.lo *.la *.al .libs *.so *.so.[0-9]* *.a *.pyc *.pyo *.a *.aps *.bak *.BAK *.bpt *.bro *.bsc *.cdb *.cfg *.cgl *.class *.compiled *.clw *.db *.dbg *.dcu *.DCU *.ddp *.dsk *.dti *.err *.exp *.gid *.gz*.identcache *.ilk *.jar *.lnk *.local *.log *.lps *.lrt *.mod *.MOD *.ncb *.NCB *.ocx *.OPT *.opt *.pch *.pjt *.plg *.PLG *.pdb *.PDB *.ppu *.o *.obj *.OBJ *.or *.rsj *.rst *.sbr *.scc *.SID *.sln *.tar *.tds *.tlh*.tli *.trg *.vbw *.zip __history bak *.~* .#* I guess it is not valid to split the list on several lines in they config file? I tried to use a backslash character as a line continuation for "global-ignores" and it did not work. Apparently you are stuck with a very long line here. I don't know if anyone else has complained about it or if an enhancement request would be seen favorably by the Subversion developers. You could shorten the line a bit by reducing the number of spaces between words, and on Windows the names should be case-insensitive so that you can remove entries that differ only in case. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA EDA Software Developer, Expert Witness www.chapman-consulting-sj.com
Re: svnignore?
On 12/29/2017 7:57 AM, Bo Berglund wrote: What is the equivalent of the CVS .cvsignore file? I have a lot of files that I don't want to get into the repository because they are only temporary files created by the compiler every time I do almost anything in the IDE. These are ignored when we use CVS by the use of a global cvsignore setting plus .cvsignore files in projects that need some more files ignored. My main cvsignore file has something like 40-50 entries.. I tried to study the svnbook on this matter but the closest I get is to use: svn propedit svn:ignore But this seems to be an exercise to be done on directory levels and my IDE creates a lot of directories that are not versioned to begin with... Is there no way to make a global setting on the client side such that the known files types will be ignored from svn? If it matters I use the SmartSvn program as a user interface on my development PC. It was recommended for users wanting a GUI interface like we had for CVS. Tell Subversion to ignore temporary files in a working copy by editing the "global-ignores" line in the file "config". On Windows systems this is in "AppData\Roaming\Subversion" within the user's home directory; on Linux systems this is in ".subversion" within the user's home directory. These are global per user, not per working copy. For example, my "global-ignores" line for Windows is: global-ignores = *.obj *.lib *.ciz *.map *.exe *.bak *.pdb *.ilk *.idb Note that these directories are not present for a given user until that person has run some Subversion command on the machine. "svn --version" should be enough. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA EDA Software Developer, Expert Witness www.chapman-consulting-sj.com
Re: Clarification on hot-backup.py usage
(please don't top-post; it makes the thread hard to follow)
On 6 October 2017 at 16:08, William Muriithi <william.murii...@gmail.com> wrote:
Hello,
I am looking at using the script below for backup purpose and a bit at
a loss at how the script verify the backup.
For context, I am referring to the script below and the flag below.
http://svn.apache.org/repos/asf/subversion/trunk/tools/backup/hot-backup.py.in
--verify Verify the backup
My question is, what should be the verify directory? Is it the source
directory on which the SVN server use for writing or is it the backup
directory? I am suspecting its the source as the backup directory is
already compressed. Would this be correct?
Regards,
William
On 10/6/2017 1:16 PM, William Muriithi wrote:
Hello,
Just realized my problem. One don't need to specify a directory for
verification. Would still be happy if someone can confirm if
verification is done on the source repo or on the resulting hot
backup
Thanks in advance.
Regards,
William
It verifies the hot backup directory.
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Software Development Done Right.
www.chapman-consulting-sj.com
Re: upgrading server
On 7/25/2017 11:00 AM, Andy So wrote: We have an old subversion /version 1.4.3 (r23084) /running on Solaris. We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), possibly version 1.8.x or 1.9.x Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go through dump and load of the repository as described in various online post and documentations. The repository is quite large…guessing the size to be in the order of 20-40GB Before we start undertaking such tasks 1.Does anyone know if there are there any problem/gotcha in migrating the repository? 2.Does anyone know how long it would take to export the repository of this size? This will give us an estimate how long to schedule down time and cut off time. Thanks for any insight. I recommend CentOS 7.x; CentOS 6.x is nearing the end of its support lifetime. I also installed an SSD on my new machine - much faster than rotating media, and Subversion's "write once" philosophy (old revisions are essentially immutable) works well on an SSD. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Facing issues in Enable editing log messages.
On 7/6/2017 10:22 PM, Ramamurthy, Manochitra wrote: *From:*Branko Čibej [mailto:br...@apache.org] *Sent:* 06 July 2017 20:13 *To:* Ramamurthy, Manochitra *Cc:* users@subversion.apache.org *Subject:* Re: Facing issues in Enable editing log messages. EXTERNAL EMAIL On 06.07.2017 14:18, Ramamurthy, Manochitra wrote: Hi Brane, Yes. I can use csvn password and it is working. I checked to login with csvn account. You showed us that you logged in to Windows with an account called 'csvn'. That's not the same as authenticating to the Apache server that's the front end for Subversion. The passwords could easily be different, even if the account name is the same. -- Brane Hi Brane, I’m not sure how to process this issue. I mean I’m unable to guess anything. Thanks, *Mano* Applications Administrator Office : (+91) 8046552670 Mobile : (+91) 9786931882 What he means is that the Subversion account is unrelated to your Windows account. In effect it is on another machine. In fact, nothing requires the Subversion account have the same username or password as your Windows account. What you need to do is obtain the Subversion account password from your repository administrator. Then use that password whenever you need to perform a Subversion action. By the way, please don't top-post - it makes the conversation hard to follow. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Commit fails - "Can't set position pointer in file, access denied"
On 12/8/2016 2:22 PM, Ryu, Ryan wrote: *From:*David Chapman [mailto:dcchap...@acm.org] *Sent:* Thursday, December 08, 2016 2:10 PM *To:* Ryu, Ryan <r...@moog.com>; users@subversion.apache.org *Subject:* Re: Commit fails - "Can't set position pointer in file, access denied" On 12/8/2016 1:49 PM, Ryu, Ryan wrote: Hi, I’m getting the same error as the linked issue below using a very similar setup, but the discussion has no replies. https://subversion.open.collab.net/ds/viewMessage.do?dsForumId=3=browseAll=575976 <https://urldefense.proofpoint.com/v2/url?u=https-3A__subversion.open.collab.net_ds_viewMessage.do-3FdsForumId-3D3-26viewType-3DbrowseAll-26dsMessageId-3D575976=DgMDaQ=ByDrzdYw8tO08sJlHDO_Vg=Z9Jz6AFP7WIqS66fTa2NVw=snq8MwvvURTCGdHC43plueocubhvjS6cEFiw1kZYsqk=MnGlewr7BaoQyDKbQnVPoISEJw2JEAy70aTgj09kKzI=> Markphip was able to point me in a promising direction, but he recommended that I also post here. The output from TortoiseSVN looks like the following (with sensitive information removed): Modified 'C:\qqq...qqq\qqq.docx' Sending content 'C:\qqq...qqq\q.xlsx' Sending content 'C:\qqq...qqq\qqq.docx' ErrorCommit failed (details follow): ErrorCan't set posiiton pointer in file Error '\\qqqvnxnas01.qqq.com\qqq\Projects\qSVSUBVML1\CSVN\Repositories\qqq\db\txn-protorevs500-en.rev': ErrorAccess is denied. Completed! If anyone has any ideas, it’d be greatly appreciated. Thank you! Do you have antivirus software running? If so, can you try an experiment with the antivirus software disabled? -- David chapmandcchap...@acm.org <mailto:dcchap...@acm.org> Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.chapman-2Dconsulting-2Dsj.com=DgMDaQ=ByDrzdYw8tO08sJlHDO_Vg=Z9Jz6AFP7WIqS66fTa2NVw=snq8MwvvURTCGdHC43plueocubhvjS6cEFiw1kZYsqk=k6OFSgUZdxL5ATqH98O5V1OI1pamywoeS3Vl2blcfc0=> Yes. What do you have in mind? Thanks. Turn off the antivirus software and rerun the commit. If your problem is intermittent, you may have to leave the antivirus software off for a little while. When a file is created, many antivirus programs will queue that file for immediate analysis. That analysis may interfere with use of the file. For example, in a non-SVN context, I had a test program that would run multiple tests. Each would create a small file, write to it, check it, then delete it. If I didn't change the file name each time, tests would sometimes fail, because the deleted file was being held back for analysis by the antivirus software. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Commit fails - "Can't set position pointer in file, access denied"
On 12/8/2016 1:49 PM, Ryu, Ryan wrote: Hi, I’m getting the same error as the linked issue below using a very similar setup, but the discussion has no replies. _https://subversion.open.collab.net/ds/viewMessage.do?dsForumId=3=browseAll=575976_ Markphip was able to point me in a promising direction, but he recommended that I also post here. The output from TortoiseSVN looks like the following (with sensitive information removed): Modified 'C:\qqq...qqq\qqq.docx' Sending content 'C:\qqq...qqq\q.xlsx' Sending content 'C:\qqq...qqq\qqq.docx' ErrorCommit failed (details follow): ErrorCan't set posiiton pointer in file Error '\\qqqvnxnas01.qqq.com\qqq\Projects\qSVSUBVML1\CSVN\Repositories\qqq\db\txn-protorevs500-en.rev': ErrorAccess is denied. Completed! If anyone has any ideas, it’d be greatly appreciated. Thank you! Do you have antivirus software running? If so, can you try an experiment with the antivirus software disabled? -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Subversion dump/load - author
The convention on this E-mail list is not to top-post, because it is more difficult to follow the thread. Thanks. On 9/13/2016 7:45 AM, Tom Sorensen wrote: Hi – Thank you for returning my email. To answer your question in the first sentence, yes, I am talking about the files within the repository tree. Let me first explain that the current repository is running on Linux. I access the repository through TortoiseSVN on my workstation using the link that connects me to the repository on the Linux machine such as, https://pathtorepository/ Below is an example of the TortoiseSVN repository display header. FileExtension Revision Author Size Date Accounting App 70 jjones 01/01/2012 Budget App 135 bsmith 04/16/1996 InventoryApp 16 tfoxworth 03/22/2001 When I create the dump file, I do so by directly logging into the Linux machine. From the command line, I do the following; $svnadmin dump /app/svn/csvn/data/repositories/nameofrepository –r 7500 > prod1.dump The prod1.dump file is create and I have a sys admin copy it to the Windows machine. I create an empty repository on the Windows machine using subversion edge(collabnet) administration console. I then perform the svnadmin load command. (I don’t remember the exact syntax but I use force uuid). The load is successful and when I open the repository using TortoiseSVN on the Windows machine, all of the Author’s names, in all the folders and sub folders, in all revisions are the same, such as bsmith. Each time I do this, the Author’s name will change, but the effect is the same. All of the Author’s names in all of the directories and sub directories contain the same name. I’d like to keep the Author’s names as they are displayed in the repository before the move. Note: At one time I had used a repository that was on a dev box, one that I used for experimentation. I was able to load the entire repository without having to do the latest revision. Everything displayed as expected. I tried to do a full dump of the production instance but the resulting dump file was so large that I ran out of room. I also tried to dump it to a zip file, but didn’t have much luck. When I talked about "repository files", I was referring to the back end files managed by the server (which are read by "svnadmin dump" and written by "svnadmin load"). You are talking about the front end display in TortoiseSVN, which is a separate product on top of Subversion. Someone (not me) on the Subversion E-mail list might know the answer to your question, but you should probably ask on the TortoiseSVN E-mail list. Good luck. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Subversion dump/load - author
On 9/12/2016 7:12 AM, Tom Sorensen wrote: Hello – I am a self taught Subversion administrator. I hope I am posting to the correct place. I am moving a subversion repository from Linux to Windows. I performed the dump command on the Linux machine and copied the resulting repository. dump file to the windows machine. On the windows machine I created an empty repository via a subversion Edge console. I then performed the load command from a command line prompt. The repository loaded but the ‘author’ on all folders have the same person’s name assigned. For example jbrown is the author throughout the entire repository. Some time ago I had a test repository and performed the same as above and all of the resulting ‘authors’ were correctly loaded. I have reviewed the subversion website, but haven’t seen anything on this. Can you help? Are you asking about files within the repository directory tree on the server, e.g. revision files? It is typical for them to be created by a server process and thus be owned by the user ID for that process. I host my repositories on Linux using the "http://; access method, and all files within the repository have the user ID "apache". The user IDs of the committing developers are stored within the repository data structures for each revision, so I can still determine who committed each revision. If I loaded a dump file under Windows, I'd expect all revisions to be created with my user ID, not "apache". (I don't have multiple accounts on my Windows machines, so I can't test this.) If you were using the "file://" method for repository access then I can see how individual revisions would be owned by the committing developers, but this is not the recommended method of hosting a multi-user Subversion repository. If you were not using the "file://" method, I'd tend to look at ownership of repository files by different user IDs as a bug, or at best a quirk - not the expected or "only correct" result. If I misunderstood your question please list the files which have surprising ownership. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: A couple thousand mp3 files (this is not spam I swear )
On 8/14/2016 1:22 PM, Adam Jensen wrote: On 08/13/2016 09:33 PM, Branko Čibej wrote: But note that a rename is represented as an add+delete, so the hook would have to be rather smarter than it would seem at first glance to detect and allow renames without content modification. The literal file names are composed of a date and a sequence number, and like the contents of the files, the names should never change. The core data-set directory structure (trunk, maybe) will most likely be calendar-like (years->months->days->sequence->file). The analysis tools and meta-data will probably be kept in a separate fossil[1] repository. [1]: http://www.fossil-scm.org/ The near-term goal is to maintain an indelible record of the physical measurements of reality. Any analysis [of which, there will be plenty], annotations, and other meta-data generation must not alter the fundamental instrumentation data. Given that, by "rename" do you mean a change of the literal file name like what I tried to describe above, or are you referring to something more like the file references, links, or pointers within the repository [internal implementation], similar to David's use of the term "rename" (included below)? Yes, he and I are referring to the same thing. The file contents are not copied, which is your primary goal. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: A couple thousand mp3 files (this is not spam I swear )
On 8/13/2016 12:29 PM, Adam Jensen wrote:
On 08/13/2016 02:21 PM, David Chapman wrote:
On 8/13/2016 11:07 AM, Adam Jensen wrote:
When a branch is created, are the files under revision control in the
trunk copied to the branch (is there any duplication of files in the
repository)?
No, the files are not copied; a rename is stored. These are "cheap
copies", and this is an advantage over simple backups - if you want to
save history using backups (per another suggestion), you need to retain
one backup per significant event. That can add up.
Thanks! That's a critical issue for my case where there is a large &
growing core data-set and where it might be useful to have hundreds of
branches, each representing a particular configuration of a subset,
slice, or view of the core data-set.
Subversion is most often used to store text files because it stores
intra-file deltas when content is modified. Your use case is unusual,
but as long as you don't make a lot of changes to the binary files, it
will be efficient.
Thanks [again] for the [vindicating] confirmation. I am inspired to set
up a test case and explore this approach further :)
Since, in my case, the binary files should/must never change, is there a
way to configure a read-only attribute on specific files in the
repository such that any subsequent attempt to check-in a change to any
of those files will be rejected and an alert raised? The directory
structures should remain changeable.
I don't know about an attribute, but you could define a hook script that
would check the files being committed to ensure that no existing large
binary files are being modified. I haven't done any work with hook
scripts for several years, so I'll have to let someone else assist if
you have more questions.
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Software Development Done Right.
www.chapman-consulting-sj.com
Re: A couple thousand mp3 files (this is not spam I swear )
On 8/13/2016 11:07 AM, Adam Jensen wrote: On 08/12/2016 08:56 PM, Adam Jensen wrote: Here's the situation: I have ~1500 mp3 files (not pirated music), and the collection is growing. The sizes range from ~100kB to ~300MB. The content of these files will never change. The directory structure will change, files will be moved, and new (additional) [mp3] files will be added. When a branch is created, are the files under revision control in the trunk copied to the branch (is there any duplication of files in the repository)? No, the files are not copied; a rename is stored. These are "cheap copies", and this is an advantage over simple backups - if you want to save history using backups (per another suggestion), you need to retain one backup per significant event. That can add up. Subversion is most often used to store text files because it stores intra-file deltas when content is modified. Your use case is unusual, but as long as you don't make a lot of changes to the binary files, it will be efficient. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: E165001 pre-commit hook failed
On 7/31/2016 2:30 PM, Patrik Jonsson wrote: Hi all, I've been banging my head against this problem for a day and I need some help. We recently updated the machine hosting our svn repo, and this broke commits using svn+ssh. Here's the setup: * Machine runs debian 3.16.7, svn 1.8.10. * svn runs as user "www-data" (the apache user). * svn+ssh access uses a forced ssh command which sudos to the www-data user and executes an svnserve wrapper. * svnserve wrapper sets umask to 022 and then executes "svnserve -t" with a specific tunnel user. This setup is identical to what it was on the old machine. However, there must be something different about it, because now https commits work fine, but the svn+ssh commits give the error: Transmitting file data .svn: E165001: Commit failed (details follow): svn: E165001: Commit blocked by pre-commit hook (exit code 255) with no output. If I access the repository directly using file:// and sudo to the www-data user when executing svn, commits work fine. This, in combination with the fact that https access works, makes me conclude it is not a permissions or hook problem on any of the files since all these access methods run as the www-data user. Nevertheless, the error comes from the hook, because if I remove the hook file completely, the failure moves to the post-commit hook. It's not a problem finding !#/bin/sh either, because I tried replacing the hook with a compiled C program that just returns 1, and I still got the 255 return code. When I attempt to commit, I can see successful authentication in the syslog, like: sudo: : TTY=unknown ; PWD=/home/ ; USER=www-data ; GROUP=www-data ; COMMAND=/usr/local/bin/svnserve-wrapper sudo: pam_unix(sudo:session): session opened for user www-data by (uid=0) sshd[26903]: Received disconnect from : 11: disconnected by user sshd[26897]: pam_unix(sshd:session): session closed for user The svnserve log file gets this (the name of the repo here is "test") test open 2 cap=(edit-pipeline svndiff1 absent-entries depth mergeinfo log-revprops) / SVN/1.8.10%20(x86_64-pc-linux-gnu) - and then nothing. (I don't know what the "open" command does, it's not included in the list of commands on e.g. http://svnbook.red-bean.com/en/1.8/svn.serverconfig.operational-logging.html) I've seen some similar reports of this, but no suggestions apart from permissions or corrupted hook files, which this can't be. I don't even know how to proceed with debugging this. Is it possible to see what svnserve attempts to do with the hook file? Since it's spawned on demand, I don't know how to attach to it with a debugger, or where in the source code this error originates. Any ideas would be much appreciated, Regards, /Patrik J. Is SELinux enabled on the new server? I've seen some oddball permission problems result when upgrading Linux systems if SELinux is enabled on the new server but not the old. I don't use svnserve, so I can't offer specific advice other than the security context in which the new user runs may be different than that of svnserve, and SELinux may be blocking it. On Red Hat/CentOS, you would look in "/var/log/audit/audit.log" for signs of trouble. I don't know if that is the location of the SELinux log files under Debian. In particular, watch out for files (scripts, configuration files) copied directly from an older server without SELinux into a new server with SELinux. They don't get a context appropriate to the directory in which you put them. I use Apache, and I had to track down these files afterward and fix them one by one - very painful. This isn't just a Subversion problem but is a general Apache problem. If you do have SELinux running, a quick way to determine whether you have a security context problem is to turn SELinux off briefly. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Debian Linux 32 vs. 64 bit
On 1/25/2016 10:45 AM, Philip Martin wrote:
David Chapman <dcchap...@acm.org> writes:
On 1/25/2016 3:59 AM, Niemann, Hartmut wrote:
I want to upgrade my Linux box from Debian Jessie (32bit) to Debian
Jessie (64bit).
For the transition time, the machine will boot alternating the 32bit
and the 64bit OS.
I have several SVN repositories and working copies on it.
Is it safe to share SVN repositories and working copies between
32bit and 64 bit?
(Jessie ships with 1.8.10, as far as I know.)
It should just work for both repositories and working copies.
Is that documented somewhere, such that system administrators can rely
on it? Or could a Subversion developer decide to put endian- or
size-dependent binary data into a repository or working copy file somewhere?
I wouldn't try this approach for a machine with repositories. I had a
repository on a 64-bit Linux machine, then bought a 32-bit micro
server that I could keep running all the time. I was unable to copy
the repository directory tree directly, even though both were Intel
architecture machines. I had to dump and load (with svnadmin). I'd
be surprised if this has changed in the last three years.
It's hard to work out what went wrong from that vague description. In
the unlikely event that you were using a BDB repository then there are
BDB library compatibility issues: a recover/upgrade may be needed and
downgrading to older BDB is not always possible.
My notes are vague, unfortunately. This was back in 2007, when I was
compiling from tarballs:
1) configure/compile with BDB support on both platforms
2) copy repository directory to 32-bit machine
3) problem found -> dump/load to avoid system dependencies
4) problem solved
5) hey, my repositories are actually FSFS!
I was trying to take good notes, as is my practice for system
administration tasks that I do once every three years, but somewhere in
the migration process I failed to write something down. Item 5) was
recorded some weeks after the migration, so I don't know when the build
switched to FSFS support only, relative to the repository directory
move. Nor do my notes record what actually got me to a
production-worthy repository. So yes, kind of vague. But enough to get
me spooked.
I also know from experience that it is very easy to let platform
dependencies leak into outside data files. Avoidance takes a concerted
effort and detection is extremely difficult when testing a build on a
single machine (here, by definition we're talking about running a single
repository on two separate machines). Thus my question about a
documented promise from Subversion developers.
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Software Development Done Right.
www.chapman-consulting-sj.com
Re: Debian Linux 32 vs. 64 bit
On 1/25/2016 3:59 AM, Niemann, Hartmut wrote: Hello! I want to upgrade my Linux box from Debian Jessie (32bit) to Debian Jessie (64bit). For the transition time, the machine will boot alternating the 32bit and the 64bit OS. I have several SVN repositories and working copies on it. Is it safe to share SVN repositories and working copies between 32bit and 64 bit? (Jessie ships with 1.8.10, as far as I know.) I wouldn't try this approach for a machine with repositories. I had a repository on a 64-bit Linux machine, then bought a 32-bit micro server that I could keep running all the time. I was unable to copy the repository directory tree directly, even though both were Intel architecture machines. I had to dump and load (with svnadmin). I'd be surprised if this has changed in the last three years. It is possible that the working copies would be OK, but I haven't tried that migration path. I'd worry that there are binary files in the working copy format as well (e.g. SQLite), and these might have native long integers which will switch from 32 to 64 bits on Linux machines. I just did a 32-bit to 32-bit migration, and I found that the safest approach was to have a separate machine (or at least a separate hard drive) so that I always had a stable system running. I took the original offline only after the new one was running and stable. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Unexpected HTTP status 400 'Bad request'.
On 12/7/2015 2:56 PM, Chris Capon wrote:
Hi.
We are running a Subversion server using Apache2 2.4.17-3, modDAV, and
Subversion 1.9.2-3+b1 (the latest Testing release) under Debian
GNU/Linux. We use HTTPS for security along with client certificates.
This server has been running for many years with the same configuration.
A week or so ago, when trying to commit to ONE of the repositories on
this server, from a long-time client Windows machine running
TortoiseSVN 1.9.2, Build 26806, the commit failed with the error:
Unexpected HTTP status 400 'Bad request' on {one of the files}
Since then, we have not been able to commit anything to that
repository. On a commit with about 6 files, it seems to fail on
different files periodically. It isn't always the same file name in
the error message.
The thing is, we can still do commits to other repositories on the
same server and folder tree without this error happening and even from
the same Windows machine. So I don't think the communications
themselves are the problem. There is no hardware firewall between the
client and the server.
To diagnose the problem, I tried to check out the repository on the
subversion server itself to a local folder (hoping to eliminate the
network as the problem). When I execute:
svn checkout https://server/svn/repository/dev/trunk --username
myself dev
the checkout begins to download files then will randomly stop after
about 10 files with this error:
svn: E175013: Access to 'filename' forbidden.
Repeating the experiment will cause it to fail at different files
seemingly randomly. Trying to 'svn cleanup' and 'svn update' the
partially checked out folder will give the same error after bringing
down a few more files.
I have made sure permissions are set correctly for the Apache user in
the folder with the subversion repository.
None of the log files under /var/log/apache2 seem to catch or record
anything about the errors, nor is there anything in the subversion.log
file in the same folder. I am not sure how to capture the cause of
the error on the server side.
Can anyone help me diagnose this problem?
Thanks.
Have you verified that the repository on the server is not corrupt?
Perhaps the disk has a bad sector on the drive, and only that repository
is affected. Or maybe the hard drive itself is failing, and the other
repositories have simply been "lucky" so far.
# svnadmin verify /path/to/repository/root
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Software Development Done Right.
www.chapman-consulting-sj.com
Re: SVN doesn't like .so files?
On 10/10/2014 11:19 AM, James wrote: I am trying to add an existing project into the SVN repository. It seems work but when I check them out in a new location, I found all .so files are not present. Then I look at the repository, these .so files from JDK are not there. Any workaround? I am using 1.8.10(r1615264) svn. The command I used to add project is svn import -m my message . svn://homeNetworkIP/repositoryName. Please help, Thanks, James Look in ~/.subversion/config, section [miscellaneous] for global-ignores. This is a listing of files that are ignored by default when you import a directory. You can add any file manually, but derived files like *.o, *.so, and *.exe (Windows) are often not added to the repository and so they tend to be omitted from automated processes. You can of course change the configuration file so that *.so files are not automatically excluded. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Missing update-report close tag on CheckOut
On 5/4/2014 10:57 PM, Lloyd wrote: Hi We are using VisualSVN server version 2.1.4 (svn server version 1.6.13) When I try to checkout our project it throws an error message svn: E175009: Missing update-report close tag. What could be the reason for this error message? Can something could be corrupt in our server? If so is there any tool to verify and correct our repository? Try svnadmin verify /repository_dir/ on the server machine. If that reports a problem, describe it here and you might get information on how to repair it. For example, I routinely run svnadmin verify /var/www/html/svn/repo on my server. This requires read privilege on all directories in the repository, of course, so I run it as root. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: encountered error with subversion version 1.8.0
On 1/29/2014 8:21 PM, Austin Mico wrote: Hi, I have encountered error with subversion. The conf for apache is LocationMatch ^/repos/endeca/pipeline DAV svn SVNParentPath /var/repos SVNPathAuthz off AuthBasicProvider file AuthType Basic AuthName SVN Login AuthLDAPURL AuthUserFile /etc/httpd/conf/svnuser_local Limit GET PROPFIND OPTIONS REPORT PROPPATCH POST MKCOL MERGE PUT COPY DELETE MKACTIVITY CHECKOUT require user svnuser /Limit /LocationMatch I encountered error is httpd: subversion/libsvn_subr/dirent_uri.c:2543: svn_fspath__join: Assertion `svn_fspath__is_canonical(fspath)' failed. [Wed Jan 29 22:43:20 2014] [notice] child pid 2740 exit signal Aborted (6) httpd: subversion/libsvn_subr/dirent_uri.c:2543: svn_fspath__join: Assertion `svn_fspath__is_canonical(fspath)' failed. [Wed Jan 29 22:43:21 2014] [notice] child pid 2742 exit signal Aborted (6) My svn client is version 1.8.5. These are Apache errors, so the first question is what versions of HTTPD and Subversion are present on the server. The client version probably doesn't matter. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Unable to open repository following reboot
On 12/4/2013 5:20 PM, Pat Haley wrote:
You can cause strace to write to a file with -o myoutputfile
So if you want to post I'd do:
strace -o myoutputfile -s 0 svnadmin verify
/home/phaley/Papers/2011/ArpitVel/SvnPaper
That line is complaining about a stale file handle:
fcntl(3, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824,
len=1}) = -1 ESTALE (Stale NFS file handle)
write(2, svnadmin: disk I/O error\n, 25) = 25
write(2, svnadmin: disk I/O error\n, 25) = 25
exit_group(1) = ?
From a couple of lines above it looks like the file it's
complaining about is
/home/phaley/Papers/2011/ArpitVel/SvnPaper/db/rep-cache.db
however, I can use the file command successfully on that file
You say that the NAS server was rebooted. Were any of the machines
accessing the NAS rebooted? Stale file handle looks like a problem
within the server accessing the data, not the NAS. If you can't reboot,
can you umount/mount the NAS from one of the affected machines and then
try again on that machine?
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Software Development Done Right.
www.chapman-consulting-sj.com
Re: svn delete removes read-only files
On 11/19/2013 6:28 PM, Ryan Schmidt wrote: On Nov 18, 2013, at 10:37, Rick Varney wrote: So for Redhat 5, rm does seem to give read-only files some special treatment. The fact that rm on your OS does not makes me wonder if I am wrong about how typical this behavior is in other Linux/Unix flavors. Both GNU rm (used on Linux) and BSD rm (used on OS X and *BSD) do this. From the BSD rm manpage on OS X 10.9: It's quite possible that I haven't read the rm man page since the 1980s. :-) Also, the only time I tend to have read-only files is when I am running as superuser, in which case I run rm -f to get rid of stuff without the prompt from the rm -i alias that is the default within the superuser account. So I have never seen the prompt. For the record, my machines mostly run CentOS, so they have the behavior described. I just confirmed this by creating a read-only file in a non-superuser account. Guess I learned something today. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: svn delete removes read-only files
On 11/15/2013 3:57 PM, Rick Varney wrote: Branko Čibej brane at wandisco.com writes: It really depends on what you think Unix rm actually is. The unlink() or remove() functions don't care about permissions on the target, only that you have write permission on the parent directory. It does not surprise me to hear that system functions don't care about file permissions. The OS flavor I'm most familiar with is rhel5. Here is how the interactive rm command behaves in this case: $ ll foo.txt -r--r- 1 usb00219 nxp 4 Nov 15 15:42 foo.txt $ rm foo.txt rm: remove write-protected regular file `foo.txt'? y So while rm is perfectly capable of removing the read-only file in a directory where I have write permissions, it politely asks first. I had assumed the rm command behaves similarly on many flavors of Unix (as well as Linux). This is probably due to an alias set up in your login script, e.g. .bashrc. Type alias at the command prompt; you will probably find that rm is aliased to rm -i, which prompts before removing files. rm does not prompt like this by default. ll is not a Unix/Linux command either, so it's a pretty safe bet that you have aliases set up, either in the distro you have installed, or configured by your sysadmin (on my CentOS machines, this alias is present only for the root account by default). Try \rm foo.txt (backslash before executable name); I'll bet that it does not ask you for permission to remove a write-protected file. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: server config
On 8/19/2013 9:07 AM, Scott Frankel wrote: Hi all, I'm new to SVN server configuration and find myself setting up a CentOS 6.4 server with svn version 1.6.1, following the red-bean book. I'm having difficulty with authorization /or authentication: my repo appears to be accessible by anyone in spite of requiring valid-user and specifying digest authentication. I believe this because 1) I can download a full working copy of the repo to a 3rd-party logged into a foreign computer, and 2) I have dozens of entries in apache's logfiles, like these from this morning, *prior* to any known/legitimate access to my repos today: Step 1: Take the real repo offline until you get the security problems resolved. Every IP address on the Internet is scanned every single day, and your repo is vulnerable right now. Step 2: Run experiments with a dummy repo to resolve the security problems. Don't ever run security experiments with production data. Put production data online only after you know that all security problems have been resolved. svn_logfile: [19/Aug/2013:00:46:32 +] - checkout-or-export / r1 depth=infinity This does not look good at all. It's quite possible that someone else has downloaded your repository. The access_log file may tell you the IP address from which the request was made; with luck it's the one you were experimenting with and the time stamp matches your access. access_log 93.174.93.213 - - [19/Aug/2013:07:23:50 +] GET /w00tw00t.at.blackhats.romanian.anti-sec:) HTTP/1.1 404 319 - ZmEu If you watch access_log over time, as I do, you will see these sorts of probes constantly. Too many to list, they are meant to check for known vulnerabilities in older versions of Apache or tools installed (SQL administration, PHP administration, etc.). These will never go away. Get used to them. They should all fail; if not then you have other security problems to fix. error_log [Mon Aug 19 07:23:51 2013] [error] [client 93.174.93.213] File does not exist: /var/www/html/MyAdmin This doesn't look good at all. My Location directive follows below. The /etc/svn-auth.htdigest exists and appears to be valid. My goal is to setup the repo, serve it via Apache, provide access to only a small number of people that I approve, use cmd-line svn and do so securely. These are standard goals. This is my first brush with Apache, OpenSSL, and general server config. Thanks in advance for your suggestions! BTW, I'm not subscribed and would appreciate being cc'd on any replies. Scott Location /svn DAV svn SVNParentPath /var/svn # Authentication: Digest AuthName Subversion repository AuthType Digest AuthUserFile /etc/svn-auth.htdigest # Authorization: Authenticated users only Require valid-user /Location How many repositories do you have? You shouldn't use SVNParentPath if you have only one repository; use SVNPath. I don't know if that is the direct cause of your problem, but you should fix it. You can also try working with AuthType Basic, creating passwords with /usr/sbin/htpasswd, until you figure things out. I have more experience with that than AuthType Digest, so I can't help you there. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: server config
On 8/19/2013 12:13 PM, Scott Frankel wrote: This is my first brush with Apache, OpenSSL, and general server config. Thanks in advance for your suggestions! BTW, I'm not subscribed and would appreciate being cc'd on any replies. Scott Location /svn DAV svn SVNParentPath /var/svn # Authentication: Digest AuthName Subversion repository AuthType Digest AuthUserFile /etc/svn-auth.htdigest # Authorization: Authenticated users only Require valid-user /Location How many repositories do you have? You shouldn't use SVNParentPath if you have only one repository; use SVNPath. I don't know if that is the direct cause of your problem, but you should fix it. I chose to use SVNParentPath on purpose. As soon as I have a secure, working server config, I'll be hosting multiple repos. Daniel and Thomas, thanks for your suggestions. My standard operating procedure is to get things working properly with a simple configuration, then add features one at a time. This helps isolate problems, especially when you are a beginner. I have VirtualHost containers in my httpd.conf files for various Web sites, domains, and SVN repositories, and you better believe I didn't try to configure everything at once! Try using SVNPath the first time and get it working securely before adding new capabilities. It's trivial to move a repository directory under Unix-like systems (and not very hard under Windows) to set up SVNParentPath later, so there is no reason to use SVNParentPath for a single repository. One last question, just in case: did you run svnadmin create /var/svn or svnadmin create /var/svn/repo1? In other words, is /var/svn a repository or a directory in which a repository was created? The directory in SVNParentPath shouldn't point to a repository, but to the parent directory of the repository (and its friends). I don't know if this would cause the security issues you are seeing, but it is a common enough mistake that I thought I would ask. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Strange behavior
On 8/12/2013 12:27 PM, John Maher wrote: Thanks Bob, that may be exactly what I am looking for. Something that would affect all the files without having to issue over 200 commands or build a dummy directory just for importing. Although that second suggestion provided by Andrew is definitely better than the first. I couldn't find where it discusses the global config in the book, if it does at all. And even if it does I doubt it would help because it won't tell me where to find the file. Unless there is a command to edit it. I tried a search and someone says there is a site-wide config (what I need) and a user config but not where they are. I am using Windows XP and an having a difficult time finding this file. I can't even find the name of it. If someone can provide that I could at least search for it and hope it has some clue inside as how to alter it. First link from Google (search was windows xp subversion configuration file location, http://stackoverflow.com/questions/6310539/where-is-the-subversion-global-config-file-for-the-slik-svn-client-for-windows) sez: C:\Documents and Settings\%USERNAME%\Application Data\Subversion\config I no longer run on Windows XP, so I don't remember if this is the proper place for the file, but I have no reason to doubt it. For Windows 7 it's in: C:\Users\%USERNAME%\AppData\Roaming\Subversion\config Which I can confirm. In the config file, I have my global-ignores for Windows set to: global-ignores = *.obj *.lib *.map *.exe *.bak *.pdb *.ilk *.idb There might need to be a few more; it's been several years since I have imported existing code into my Subversion repositories. But you get the idea. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Tags - Symbolic names instead of Directory copy?
On 5/22/2013 4:57 PM, Varnau, Steve (Seaquest RD) wrote: Hi all, I'm hoping to have slightly less controversial discussion than the recent branches-as-first-class-objects thread. That topic did, however, touch on tags. As discussed previously, tags as a convention use the same mechanism as the branches convention. The mechanism of svn cp works well for branches. The semantics work as expected (for the most part). That is, the results of the basic operations on the branch work as expected. (checkout, update, commit) In my opinion, the same semantics work less well for tags. My biased mind-set is that a tag is a name identifying a specific version of code (a cross product of branch and revision). In subversion, a directory-path@revision, (e.g., ^/trunk@123) give the correct semantics of a tag. I can checkout the exact version of code. When I am ready to update to a later (or latest) version of the branch, update does the right thing. When I commit, my changes go to the right branch, based on my checkout. Tags are intended to be immutable. By your convention (not a bad one at all), a tag would be a symbolic name for some part of the repository tree at a specific revision. You could name all your tags this way, e.g. ^/tags/trunk_at_r123 for the above-named peg revision. This would be simpler than asking developers to remember the revision from which they are working, then type in the appropriate syntax. If a tag becomes obsolete (i.e. a fatal error is found in it), just delete it, then create a new tag from the appropriate trunk revision. But a tag that is the result of an svn cp (e.g., ^/tags/TRUNK-STABLE) does not give the same semantics. * Checkout is fine, I get the right version of the code. * Update gives me the message that my workspace is up to date. So I silently miss the fact that the latest code changes I wanted to pull in are over on trunk, not on this tag I checked out from. I think I'm working on ^trunk@HEAD, but I'm not. * Commit does not send changes the tagged branch -- oh, I thought I had an version of trunk, but my commit does not go to trunk. If I (or my repo admin) properly protects the tags, I get an error and realize I forgot a switch command. If my hook script isn't set up right, it's even worse. I have a change to roll back, when I eventually notice the mistake. Usually only the build system (or developers trying to fix a specific bug) will check out a tag. Developers modifying code would not check out tags. Updating a tag, if it is immutable, is expected to be a no-op. A pre-commit hook would prevent commits to the tag. Due to those unfortunate semantics, we've do not use tags at all. Whenever we want to specify a version of code, we use branch@revision. We don't have symbolic names, but we do get the right semantics. We have a couple hundred developers and hundreds of branches, but we do without symbolic tags. Yes, I know I'm stupid and that all our developers should be able to understand how to checkout from tags and then switch instead of update, but I think we have saved a lot of grief. (Aside from the fact that I do not have server-side access and can't implement proper hook scripts on our replicated repos.) You could have the same problem by checking out a branch and then switching to trunk. Which set of updates are you going to get? Which set of updates are going to be committed, and to where? I avoid that kind of confusion by not using svn switch. But I work in a simpler repository structure (a small number of developers with most development on trunk). It sounds like the root of your problem is not being able to implement a proper pre-commit hook to make the tags directory immutable. Tools are supposed to help perform your tasks more easily and reliably, and help you catch the mistakes you will inevitably make. If you aren't allowed to configure the tool to do this, it's not the tool's fault (or the concept). A new programmer recently asked me when she would stop making stupid mistakes. I replied never - I spend half of my time making mistakes and half of my time fixing them. After 1,000,000+ lines of code, I still make stupid mistakes. That's what testing is all about. So, am not saying there is anything fundamentally wrong with how tags work now. They just don't fit our desired semantics, so we don't use them. I am also not saying how a better tag or label feature should be implemented, but for our usage, a symbolic name or symbolic link for a path@revision would be a very useful thing. I'd say tags don't fit your repository configuration. See if you can get a pre-commit hook that blocks modifications to the tags directory tree within the repository. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Tags - Symbolic names instead of Directory copy?
On 5/22/2013 7:56 PM, Branko Čibej wrote: On 23.05.2013 04:33, David Chapman wrote: On 5/22/2013 4:57 PM, Varnau, Steve (Seaquest RD) wrote: So, am not saying there is anything fundamentally wrong with how “tags” work now. They just don’t fit our desired semantics, so we don’t use them. I am also not saying how a better tag or label feature should be implemented, but for our usage, a symbolic name or symbolic link for a path@revision would be a very useful thing. I'd say tags don't fit your repository configuration. See if you can get a pre-commit hook that blocks modifications to the tags directory tree within the repository. I'm confused ... why would you need a special pre-commit hook for something like that? I would expect access control to be good enough. Just make the tags tree in the repository read-only for most users. (N.B., having actually immutable tags would be a nice feature, but they're not required to solve your problem.) Access controls implies more than what the original poster was asking. Right now anyone can make a tag in his development environment simply by recalling the revision number. Access controls could work and are a reasonable suggestion in many cases. I tend not to think of them because I usually work in more-open environments where anyone can commit. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Subversion Doesn't Have Branches aka Crossing the Streams aka Branches as First Class Objects?
On 5/18/2013 9:37 AM, Zé wrote: On 05/15/2013 06:59 PM, Les Mikesell wrote: On Wed, May 15, 2013 at 12:06 PM, Andrew Reedick andrew.reed...@cbeyond.net wrote: Plus, telling people not use to svn's touted directory manipulation features because of side-effects is a bit self-defeating. Not if you want it to act like SCM's that have branches that don't allow such things. Don't you understand that *that's precisely the problem*? Currently, subversion does not support branching, and it's only possible to manipulate subversion to *act* like it does by copying around subdirectories in the repository, and in the process screw up with the repo's revision history. You are pretty insistent that there is One True Way to use branches in development. A lot of people do not agree with you. I've seen branches used as long-term development vehicles (think years), with only cherry-picked merges coming in from or going back to trunk. This does not match the definition of use once and discard that you are promulgating. Subversion was designed as a versioned file system, in response to the shortcomings of CVS; concepts like branching and tagging have always been naming conventions built on top of that (later, merge tracking was added to assist branching). If you go back and look at the archives of this list, you will see this quite clearly. trunk, branches, and tags are simply naming conventions. People don't even need to follow those, as has been noted time and again. This gives people a lot of flexibility, which they quite naturally use. And yes, ordinary file systems do support branching and tagging. I've seen it done. It's expensive, but it works. If you want Subversion to be extended in a particular way, learn its internals and write a spec which comprehends the internals and current usage. Maybe then someone will be inclined to work on it. Better yet, offer help. This is a community project, after all, and what better way to be a member of the community than to help? Right now you are not. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Subversion Doesn't Have Branches aka Crossing the Streams aka Branches as First Class Objects?
On 5/18/2013 12:01 PM, Zé wrote: On 05/18/2013 07:16 PM, David Chapman wrote: You are pretty insistent that there is One True Way to use branches in development. No, I'm stating that if all a SCM does is track changes made to the contents of a directory and you rely on changes made to that directory to emulate branches, then there are some significant downsides to this approach when compared with SCM systems which do offer support for branching. You've missed the point. You have a specific definition of branching and do not believe that anything else can be called branching. In your message to Thorsten Schöning, you said that branch history should be deleted if the branch is deleted. That is fundamentally in opposition to the definition of Subversion, which is meant to retain all of a project's history. People ask for an obliterate feature all the time after committing a file with private information, e.g. passwords. That is hard by design within Subversion, and most of its users are glad. I, for one, would object strenuously if the history of a branch were deleted just because it was never merged into trunk. In my business (Electronic Design Automation), knowing what doesn't work (and why) is often just as important as knowing what does work. As you may understand, not everyone has a lot of time to spend on side projects, and when they do then there's the problem of attaining the insight and technical know-how to do so. In spite of that, I don't believe that not being able to spend time contributing to a project justifies declaring a specific suggestion to be tabu. Forbidding anyone from, or attacking them for mentioning a downside or a shortcoming doesn't make it go away, and doesn't make the project any better than what it already is. What does contribute to its improvement is providing suggestions on ways to improve it, such as suggesting that implementing a sorely missed feature would be a significant improvement. Do you agree? As I said, you have a specific definition of branching and are insisting that it is the only valid definition even though it would require a fundamental revision of Subversion's data model. You are insisting that branching be implemented your way even though others disagree. This is not helpful. Saying +1 for branches as a first class object is helpful because it allows developers to prioritize their donated time, and to choose a definition and implementation that balance features vs. implementation complexity. But unless you are willing to dig into the internals of Subversion and understand how it is used, insisting on a particular definition and implementation is not helpful. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Error while updating
On 4/23/2013 11:43 PM, Ajay Pawar wrote: Hi Thorsten, Thanks for your reply. SVN Admin verify is showing an error called Can't read end of file. This means your repository was corrupted somehow. svnadmin verify checks the integrity of the repository where all of your data is stored. Thus it is a first step in diagnosing unusual error messages. Perhaps there was a hardware failure on the machine or the disk drive. It is also possible that the Visual SVN installation had a problem, because error reports began soon afterward. If only a small number of revisions within the repository were damaged, then it is possible that only a subset of the Subversion file system would fail to work. Thus not everyone would get error messages. You should paste the entire svnadmin verify error message (after removing anything site-specific, like host names, from the URLs) so people can help you identify the location of the error and suggest fixes. You are providing only partial information. In particular, summarizing the error message is not as useful as pasting all of it. A fix may be as simple as restoring individual revision files from a backup (you do have backups of the repository, right?), but the Subversion users list needs to know more about the problems you are having. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Subversion Rev number - quick question
On 12/31/2012 11:48 AM, Z W wrote: Hi All We are new to SVN. We are unsure where to configure in SVN, if needed, while using it with Jenkins/Hudson CI tool. We have a big chained job that runs for about an hour. Chained job meaning several jobs combined using downstream connections of jobs. We have the following problem, which is better explained in use case scenario: snip Issue: For those files that User C checks in, we like to pick up the previous versions of those files, prior to User C's checkouts after the job B begins running and not pick up the files versions that User B checks in during the job B run. We like to know if there is anything needed to be done in SVN to make it work ? Tags are cheap copies; make a tag at the start of the build process and have the build system use only the tag. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Commit - 500 Internal Server Error in response to POST request
On 9/20/2012 2:19 PM, Mitchell Killian wrote: Guys, I've been beating my head against a wall for the last several days trying to figure this out. I'm moving my SVN repo to a new server. Shouldn't be too big a deal. I made the data dump, transferred it to my new server, installed svn v1.7.6, created the repo and imported the data dump into the new repo. No problem at all. In fact, I can browse to the repo and view all of the files. However, when I try to create a directory or commit changes to the new repo, I get this error message: Commit failed: Server sent unexpected return value (500 internal server error) in response to POST request for '/repos/!svn/me' I've thought it was a permissions issue, so I've tried every kind of user:group combo that I can and even given 777 permissions to the SVN repo folder and contents but to no avail. I've assigned nobody:nobody, username:nobody, and username:username but no user:group combo seems to work. For the life of me, I can't figure it out. Let me give some info about my system. Centos 5.8 WHM/cPanel 11.32.4 Apache 2.2.22 Subversion v1.7.6 Following up to the list: I suggested he look for a version mismatch between the installed DAV .so files and the new Subversion executables (i.e. that he may need to install new DAV .so files as well), and that turned out to be the case. He has reverted to Subversion 1.6.18 for the time being. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: general questions
On 9/10/2012 10:43 AM, John Maher wrote: Thanks again, I'm learning. I appreciate the time put in to help me and I really don't want to cost you more time, so I have a couple of yes/no questions. So the only time to use svnadmin create without having a dedicated server would be a single user (like me at home)? At some level, svnadmin create will be called once per repository. Whether that is done through a GUI-based interface or from the command line is immaterial. This is the first step in setting up a repository, and it has to be on the machine that will serve the repository. It may be helpful to think of Subversion as a program package that runs on a server. If you are a single user working on a non-networked machine, then your local machine can be a Subversion server by reading the repository directly, using the file:// protocol. This protocol has major problems with multiple (and sometimes even remote) access, so it is safely run only on the machine where the files reside and only by one user at a time. If you need to access a repository on another machine, particularly if multiple users will be accessing the repository, you need some kind of server process running on that machine to manage internal operations safely and arbitrate between simultaneous requests. Subversion includes the svnserve program to serve files using the svn:// protocol and has code that allows Apache HTTPD to serve files using the http://; or https://; protocol. Personally, my repositories are all served using Apache HTTPD. I have multiple machines, and although it is unlikely that I would ever commit code from two different machines at the same time, the ease of use for the file:// protocol just wasn't worth the risk. I host some Web sites too, and it was easier for me to adapt my HTTPD setup knowledge than to learn how to configure svnserve. Your mileage may vary. As for as the dll extensions, those are not a concern. I am talking about ide setting files. And if we have a project made up of 44 repositories I need to enter the command 44 times, no eaiser way, right? Subversion does not provide repository administration or sandbox configuration tools; it provides a repository hosting mechanism. What you are asking for is not part of Subversion, so yes you need to enter the command 44 times. Scripting languages are your friends here. Write one script to invoke the configuration commands for a single repository, then another to invoke the first script for every repository in a list. This has multiple benefits: 1) You can call the first script each time you add a new repository, rather than type in the commands all over again. 2) Automation of this kind allows you to configure all of your repositories identically. 3) The scripts document the configuration you used (rather than scraps of paper somewhere, or the memory of an employee who may leave). -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: general questions
On 9/10/2012 12:31 PM, John Maher wrote: Thanks Dave, that was helpful. I saw the svn prefix in the book but didn't know what it meant. Your explanation was good. The scripts are a good idea, but I was thinking about a gui for the client side, kinda like Subversion Edge; basically a wrapper for the command line. Even though my first computer didn't have a mouse (or hard drive) the gui is the way to go, typing commands is just not the future. I may start something to make my job easier. I think HTML would benefit the most people. But I need to learn a lot more first. Hmm, my first personal computer had a hexadecimal keypad and 256 bytes (not even kilobytes!) of memory. :-) Scripts (aka typing) allow repeatability. A GUI that allows you to specify a set of options for every repository can be helpful, but down inside it will be doing the same thing as a script - and a script is easier to customize or debug when the existing tools don't do what you need. Also, scripts don't disappear if the GUI goes down. For this reason many sysadmins prefer scripts over GUI-based tools, and I don't see this ever changing. As a result, I can't help you find a GUI that will help you administer your repositories. TortoiseSVN is a client-side GUI for Windows-based machines but I haven't used it. I don't know how close it comes to meeting your needs. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Question about Basic Authentication
On 9/6/2012 11:08 AM, Anastasio, David M CTR USAF AFMC AFLCMC/HNID wrote: Yes, I think that is exactly the problem here. I will try to create the password file with htpasswd. Does Apache suggest where the password file should reside? Is it restricted to a certain location? I couldn't find this in the documentation. Thank you. Dave There is no standard password file location, as AuthUserFile is specified directly in the Location block in your httpd.conf. Under Linux I put the password file in /etc with the rest of the system password files. The essential requirement is that the file *not* be visible from outside the server, i.e. don't put it into your repository directory or another directory under your DocumentRoot. Unfortunately, I have seen this happen - hey, what's in http://server.name/passwd.txt;? It wasn't a Subversion repository that time (and worse yet, the passwords were plaintext), but password file location is a trap for the unwary. I've never set up Apache under Windows, so I can't suggest a good location. Maybe the directory in which httpd.conf is stored? -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: SVNSYNC Error
On 7/31/2012 11:16 PM, Honeylyn O. Fukuoka wrote: Hi, The repositories and the destination repositories are all running on windows. Im not really an expert on batch files/shell script development...That's why Im asking for your help :( I just followed what I was told to do here but I don't want to mess things up. Here are the commands I ran on a Windows machine to synchronize from a remote repository: svnadmin create c:\user\work\repos cd c:\user\work\repos\hooks echo exit 0 pre-revprop-change.bat svnsync initialize file:///user/work/repos http://repos.sourcedomain.com/repos svnsync sync file:///user/work/repos This ran just fine with SVN 1.6. Can you run something similar on your Windows machine and let us know what results you get? Run the steps in a new directory, not in the repository you just created. Note that the source repository in my example is served by Apache, so it uses a http: prefix. You would want to use a svn: prefix if you are using svnserve access, as was implied by your original message. Note that the destination repository is being accessed using the file: protocol. The repository I created was not meant to be accessed by other users; it is intended only to serve as a backup. If you are trying to set up a write-through proxy then you will need something like svnserve or Apache HTTP on the destination repository. Also, depending on your repository directory names, you may need to use quote marks around arguments. I recommend avoiding space characters in your directory names for this test. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: SVNSYNC Error
On 7/31/2012 9:13 PM, Honeylyn O. Fukuoka wrote: Hi, Thanks for the past response. Can you tell me what changes do I have to make in the prerevprop-change hook? I tried some but nothing happened. I created a pre-revprop-change script with the single line exit 0 in it and put it into the hooks directory of the destination repository (not the original repository). Worked fine on Windows. Most of the issues that arise from allowing revision property changes are security-oriented; if you want to prevent some (or most) users from changing revision properties then your script must guard against it. But if you are just creating a repository backup that no one else will use unless the original crashes irretrievably, you don't need anything more. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: svnversion feature
On 6/7/2012 9:27 AM, Elmar Grom wrote: Hi, I am in the process of putting together make files for firmware builds and run into an issue that I didn't see addressed in any of the issues lists. One of the things I want to do is to tag the SVN revision that a binary was built from onto the file name (this was actually requested by manufacturing and I agree with their view point). The revision is easy enough to get from svnversion. The problem is, that when I run it in my working directory, inevitably I get a mixed revision result, even if I have just committed (something like 3:26). While this may be informative to me as a developer, this is not something I can embed in a file name. At the same time Make has no ability to extract just the part after the colon. I see three solutions to this problem: 1. create another directory and do a clean checkout of the version I just committed. I tried this and it works well. Cumbersome though. 2. implement an external tool that runs svnversion and then disassembles the result to get the correct number for use in a file name. Ugly, too many tools needed already to get every day work done. 3. implement an option in svnversion that makes it to only spit out the highest revision number (no colon). Alternatively, the colon could be replaced by any character the user likes. That way something compatible with the file system can be chosen. I guess what I meant to say is, it would be nice to have control over the /format/ in which svnversion reports the version. Thanks for taking the time to read and form an opinion about this. Elmar Rather than a clean checkout; you can run svn update in the root directory of your working copy. The colon says you have a mixed revision working copy, meaning that parts of it represent an earlier state of the repository. This is not what you want. An update assures all commits by you or other developers are incorporated in your build. You should not trim off the lower revision number, because it is telling you something: the binary you just built cannot be reproduced directly later. You would somehow have to rebuild the on-disk state your working copy. That is not the point of a revision control system! A freshly checked out working directory has the advantage of ensuring that there are no local changes to files that would also prevent exact reproduction of the build. What you choose is up to you, but exact reproduction is the goal. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: New to subversion, need to find documentation on writing hooks
On 5/20/2012 2:16 PM, Ted Byers wrote: I have just installed subversion on a Ubuntu VM, and trying to learn how to administer both. I have successfully created my first VN repository. :-) Now, I have read through the documentation a couple times, and know what hooks are and why they're used, but I am not clear on how to write one as the examples I have found so far are written in programming languages I don't know (such as Python). In the first project I am going to put into this repository, I am writing a test suite based on Boost's Test library (written in C++). What I want to do is write a program that assesses code coverage, so that all new code has at least one unit test, and then require that the codebase, including all existing and new tests, compile and execute successfully; and have a precommit hook that prevents a commit unless this test suite program indicates that the tests all passed. I have read arguments that recommend against this, claiming it can slow down commits, but then I am more concerned about code quality, and always having a codebase, which to me includes all tests, that compiles and runs properly than I am in the convenience of any programmers working on the project. I also write in Perl and JavaScript (and not so much anymore in Java or C#), so I will want the same requirement on all code in all the languages I routinely use. The usual convention is to put the checks in the post-commit hook, then back out offending changes if any. Otherwise commits can grind to a halt. I once worked at a company where the pre-commit test suite (not part of a hook) ran for an hour. We were supposed to ensure that no other changes had been made to the affected modules before the commit. However, this would sometimes require several iterations (a dozen developers and a couple dozen modules under active, though unrelated, enhancement efforts meant lots of file has been changed conflicts at commit time). Occasionally we simply had to merge, compile, commit, and then rerun the test suite. (Yes, I would have liked to reduce that suite to 30 seconds or less, but that was a much larger problem than what I could change at that company.) -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Subversion restore
On 2/22/2012 9:52 AM, svenk1 wrote: Les - It is an older Centos 4.8 install. I ran yum install subversion and copied the repository directory to the new server. I tried the import because I didnt have dumps. I don't know what I am doing. I tried the dump but got the same message svnadmin dump /svn svn: Expected version '3' of repository; found version '5' Dave - I just installed subversion by running yum install subversion on this older centos 4.8 install. Is there a way I can tell what version the repository is? I would prefer to install the same version to avoid conflicts with the older OS. http://svn.apache.org/repos/asf/subversion/trunk/notes/repos_upgrade_HOWTO Repository version 5 corresponds to Subversion 1.4 and later. Your CentOS 4.8 installation has Subversion 1.3.x in all likelihood if you're getting this message. Try svn --version; it will tell you what's been installed. Don't continue to use an old version of Subversion; upgrade as soon as possible. You won't get any support for older versions. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Compatible with Microsoft Office and Internet Explorer
On 1/25/2012 12:51 AM, Thorsten Schöning wrote: Guten Tag Les Mikesell, am Mittwoch, 25. Januar 2012 um 07:11 schrieben Sie: The fact that text is different on different systems isn't really something you can disagree with. I don't like it any more than you do, but that's just the way it is - something decided back in a different century. That is really no argument as almost any developer tool is able to handle both line endings today. You just have to configure it in your IDE, TextEditor and whatever you use once and for all the team, just like you define any other coding rules and there will be exactly no problem. A lot of things were decided centuries ago and we don't do them anymore. Are you going to pay me (and all my clients) to buy those tools for every machine in the known universe? I didn't think so. And what happens when files get mixed line endings? Are you sure that all tools handle them reliably? I use svn:eol-style=native because it works reliably across platforms. Without it I can't always edit text files on client Windows machines - Notepad does not handle bare newlines *at all*. (That alone is a good reason not to copy files from Linux to Windows directly, whether directly or by viewing from a network file system.) -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: 'svnadmin verify' failed after hotcopy with 'svnadmin: E160004: Revision file (r2255) lacks trailing newline'
On 1/12/2012 4:51 AM, D D wrote: On Thu, Jan 12, 2012 at 3:41 PM, Andy Levy andy.l...@gmail.com mailto:andy.l...@gmail.com wrote: Have you run svnadmin verify against the *actual* repository? Yes. No problem detected. If you make a new hotcopy of the repository, is that corrupted as well? No. I've only got a corrupted hot copy once in the last two weeks (backups are daily). Before that the repository was hosted on another WinXP system with an earlier version of the Subversion server. I only added the call to verify the hot copy to the backup script recently so I do not know if the problem ever occurred with the repository on the other system. I'd be interested to know if svnadmin employs some memory buffer to hot-copy. The offsets where data corruption starts look like multiples of 0x1000 which is 4K. The NTFS cluster size on the disk is exactly 4K. If svnadmin just calls the OS to copy each file the problem should either be in the OS or the disk. This really, really, really looks like a hardware problem or an OS-related corruption. I agree with Andy Levy - Windows XP is not a good OS to use as a server; it's 10 years old and they still haven't bothered to fix bugs that I can invoke on a daily basis (most commercial backup programs will cause a Windows service to go into an infinite loop). Upgrade if you possibly can. Linux is free and runs on cheap hardware, so I recommend it rather than try to find a Windows version that is inexpensive but can still act as a server. The fact that the problem is intermittent also points to something outside of Subversion. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Space Constrain
On 1/11/2012 10:57 PM, sureshkumar nandakumar wrote: Dear Expert Our Subversion server is RedHat Linux. We have lot of repositories which is maintaining in Linux server. Each repositories taking huge size in our server. Our Maximum size limit is 100GB, but the size almost reached 98%. We are in trouble when we are using repository in Tortoise SVN. We are getting space constrain issues. For temporary purpose we deleting unused repositories in Server. Even though the size in increasing daily basis. Can anyone suggest me, how to save space. Is that any good way to keep it SVN server without space constrain? Is that any way to compress and reduce the repositories size without any impact? Please advise me with good practice. Your suggestion is more use to me. What are you storing that is so big? Can you store only the inputs and methods used to generate each version of these large file, rather than the large files themselves? -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: control-M characters that are NOT end of line characters
On 1/4/2012 12:56 PM, James Hartleroad wrote: For autopopulate I’ve setup for shell scripts to be text/plain, native eol and executable *.ksh= svn:mime-type=text/plain;svn:eol-style=native;svn:executable But I have a shell script removeCTLM.ksh that has an embedded cntl-m as part of a regular expression, for example sed 's/^M//g' $file tmpfile I’ve ended up renaming it to removeCTLM.ksk so I could not include the native eol style *.ksk= svn:mime-type=text/x-ksh;svn:executable But I’m wondering if anyone has a better solution to this so I don’t have to play games with the filetypes to avoid Subversion from complaining about the ^M in the file? It's been awhile since I've played with the Korn shell, but this works in bash: #!/bin/bash -f sed 's/\r//g' $1 tmpfile Rather than use a special character, I used the shell's escape sequence. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: Request for thoughts on working copy enhancement request
On 12/21/2011 4:31 PM, Randon Spackman wrote: Hmm... It's close, but the limitations on only assigning one per file are a bit onerous; it's very often in the same file that I'm working on that I find bugs, etc., that I want to commit separately. Randon Normally I have one main sandbox and one quick updates sandbox. The latter doesn't get used very often - only when I need to fix something quickly while the code in the main sandbox is in an unstable state. Updating this backup sandbox takes much less time than a full update, and my machine is not so small that a few gigabytes of duplicated storage would fill it. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: New to SVN
(The convention on this list is not to top-post. Also, please reply all so that the entire conversation can be searched in the Subversion archives.) On 12/7/2011 8:56 AM, James Lopes wrote: *From:*David Chapman [mailto:dcchap...@acm.org] *Sent:* Wednesday, December 07, 2011 11:48 AM *To:* James Lopes *Cc:* users@subversion.apache.org *Subject:* Re: New to SVN On 12/7/2011 8:16 AM, James Lopes wrote: I created a project in our repository and used import to load the source code. Now my working folder has every icon that svn has any ideas? You need to provide more details. What were the commands you used to create the repository (if it did not already exist) and what were the commands you used to create the project and import the initial source code? Finally, what do you mean by every icon? Can you give us an example of an unexpected file name? The repository had already been created I used Tortoise to create the project folder, then used it again to import the files to the project. Now some file have the red !, green check, no icon at all, a few have the blue ?. I also have folders that don't have an .svn subfolder. I can't add, update my changes. For the most part, this is a TortoiseSVN question and is probably best directed to us...@tortoisesvn.tigris.org. I don't use it, so I can't help with TortoiseSVN-specific questions. If you are using Subversion 1.7.x under TortoiseSVN, the working copy format has changed and there will be only one .svn directory at the root of your working copy. Furthermore, if your working copy already existed when the Subversion 1.7.x client was installed, it must be upgraded. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: New to SVN
On 12/7/2011 9:51 AM, James Lopes wrote: *From:*David Chapman [mailto:dcchap...@acm.org] *Sent:* Wednesday, December 07, 2011 11:48 AM *To:* James Lopes *Cc:* users@subversion.apache.org *Subject:* Re: New to SVN On 12/7/2011 8:16 AM, James Lopes wrote: I created a project in our repository and used import to load the source code. Now my working folder has every icon that svn has any ideas? You need to provide more details. What were the commands you used to create the repository (if it did not already exist) and what were the commands you used to create the project and import the initial source code? Finally, what do you mean by every icon? Can you give us an example of an unexpected file name? -- David chapmandcchap...@acm.org mailto:dcchap...@acm.org Chapman Consulting -- San Jose, CA Do I have to have my working folder and svn folder have the same name? The .svn folder (for 1.6.x and earlier, folders) is created automatically when the working copy is created. It is managed by Subversion and you should not modify anything in it. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: workqueue.c line 672: assertion failed (checksum !=NULL)
On 11/16/2011 4:20 PM, AlfredD wrote: Stefan Sperling-7 wrote: Get a new checkout. This problem will be fixed in 1.7.1 which will be released in 4 days or so. I downloaded version 1.7.1 (upgrading from 1.6.something) and the bug's still there... No, he meant your working copy is corrupted and you need to check out another. The working copy upgrade process cannot handle certain types of working copy corruption. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: svn commit does not work with SVNParentPath in location and 1.7-client (HTTPv2)
On 11/16/2011 11:08 PM, Sergey Skvortsov wrote: On 16.11.2011 19:56, Stefan Sperling wrote: On Wed, Nov 16, 2011 at 06:37:39PM +0400, Sergey Skvortsov wrote: s/svn update/svn commit/ On 16.11.2011 18:31, Stefan Sperling wrote: On Wed, Nov 16, 2011 at 02:43:17PM +0400, Sergey Skvortsov wrote: Configuration for Apache: Location /svn DAV svn SVNParentPath /usr/home/svn SVNListParentPath on /Location Location /svn/foo DAV svn SVNParentPath /usr/home/svn/foo SVNListParentPath on /Location [Wed Nov 16 10:32:58 2011] [error] [client 10.10.10.10] (20014)Internal error: Can't open file '/usr/home/svn/foo/foo/format': No such file or directory You are nesting locations. This isn't a valid configuration because it leads to ambiguity when splitting a URL into the part which is outside of the repository and the part which is inside the repository. Why this configuration is not valid? Before Subversion gets called, Apache HTTPD decides which Location block gets to handle a request. Subversion has no control over this decision. Note that Apache HTTPD looks only at URLs. It has no idea and does not care whether or not there is a Subversion repository anywhere within the URL http://svn.example.com/svn/foo/bar/baz With your configuration the URL has two possible repositories: http://svn.example.com/svn/foo http://svn.example.com/svn/foo/bar So how is Apache HTTPD supposed to tell which of your Location blocks should handle requests for the URL http://svn.example/com/svn/foo/bar/baz? ShouldLocation /svn handle it or shouldLocation /svn/foo handle it? Of course /svn/foo. Really? And what if you check out a copy of http://svn.example.com/svn/foo and create a directory named bar in it? Are you expecting Apache or Subversion to notice that this is the path to another repository and thus ban creation of the new directory? What if the repository http://svn.example.com/svn/foo already has a directory named bar in it when you first move the child repository into /svn/foo/bar? I could go on, but the possibilities for ambiguity are endless. Don't nest repositories. Locations are worked down from shortest URI. So correct SVNParentPath must be passed to mod_dav_svn. The documentation at http://svnbook.red-bean.com/en/1.7/svn.serverconfig.httpd.html#svn.serverconfig.httpd.basic says: == Be sure that when you define your new Location, it doesn't overlap with other exported locations. For example, if your main DocumentRoot is exported to /www, do not export a Subversion repository inLocation /www/repos. If a request comes in for the URI /www/repos/foo.c, Apache won't know whether to look for a file repos/foo.c in the DocumentRoot, or whether to delegate mod_dav_svn to return foo.c from the Subversion repository. The result is often an error from the server of the form 301 Moved Permanently. == The error you see is not 301 but the root cause of the problem is the same. I disagree. This cite is irrelevant to my issue. It worked in 1.6 servers. It seems like regression. I am surprised that it worked. I am not, because it's correct way. It is not the correct way. Don't nest repositories. Also, in 1.7 reading repos is working fine: OPTIONS /svn/foo/bar HTTP/1.1 200 REPORT /svn/foo/bar/!svn/me HTTP/1.1 200 but only POST fails: POST /svn/foo/bar/!svn/me HTTP/1.1 500 So the problem is not in Apache or configuration but in mod_dav_svn itself. It is obviously a regression. It was never intended to work, so it cannot be a regression. Don't nest repositories. I want to avoid use SVNPath for every leaf repo in the root. SVNParentPath is intended for such purpose. Also I want to use SVNListParentPath to autogenerate normal listing. It's impossible with multiple SVNPath. You have repositories in /usr/home/svn and also /usr/home/svn/foo. This is a bad way of organising your repositories because there is no unique way to map URLs to repository roots. I disagree, see above. There are robust way to handle Locations in Apache. I prefer to organize my repos in this way because is natural. Similar to filesystems: I like to place files and organize them to folder/directories as I want to. You are mixing file system references and Subversion references. This leads to ambiguity. Don't do it, even if it feels natural to you. Put your repositories into distinct, and flat, directories. Don't nest them in a tree. Move some of your repositories into /usr/home/svn/foo and some into /usr/home/svn/bar, and change your configuration file to look like this: [skip] This will give you a setup that works reliably. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Upgrading repo without using svnserve
On 11/10/2011 12:05 AM, Richard Boyce wrote: This may be a stupid question, but I'm using svn (via TortoiseSVN and client 1.16.16) to a file-based repo (i.e. no subversion server). When I try to re-integrate a branch, I get Retrieval of merginfo unsupported. Searches for this say I need to upgrade the repo with svnadmin --upgrade - but svnadmin is part of svnserver, which I don't use. Do I have to install svnserve (or just svnadmin) to do this (then uninstall svnserve if I don't want to continue with it?) svnadmin is a command-line executable. I don't know how TortoiseSVN is packaged, but you do not have to go through svnserve to use svnadmin. You can either install svnadmin.exe by itself (if possible) or install the svnserve package, then use only svnadmin.exe. Not knowing which DLLs are needed to run svnadmin.exe, I can't tell you how much to delete afterwards, but I would expect that you can leave everything in place. If TortoiseSVN insists on going through svnserve.exe, you should be able to delete just that. It may be better to ask this question on the TortoiseSVN list. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Request for Subversion Server 1.5.3 binaries for AIX OS link
On 11/1/2011 11:51 PM, sagade.sub...@vodafone.com wrote: Hi Can you please help. Need to download Subversion Server 1.5.3 binaries for AIX OS. Can you please point me to the link. Regards Subodh If you really need 1.5.x (the current version is 1.7.1), you should probably get the newest one possible, not 1.5.3. You can find 1.5.9 here: http://www.perzl.org/aix/index.php?n=Main.Subversion This was from a link on the main Subversion Web site, subversion.apache.org. If you absolutely must have a version of 1.5.x with extra bugs, I can't help you; I use Linux and Windows myself. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Request for Subversion Server 1.5.3 binaries for AIX OS link
On 11/2/2011 12:58 AM, sagade.sub...@vodafone.com wrote: Thanks David. Can you please help me in locating link from where I can download binaries of SVN server 1.7 for AIX OS. I wish to download pure binaries (not the rpms nor any source code/packages to be compiled) Regards Subodh (Please don't top-post; the convention for this list is bottom-posting. Thanks.) The RPMs are compiled and maintained by a volunteer who has not yet compiled 1.7.1. I don't know when that will occur; perhaps someone else on this list will know. I do not know of anyone else who has compiled Subversion for AIX, and I don't know if you will find anyone who supplies raw binaries only. Compiling the source code is not terribly difficult; is there a reason you don't want to try to compile it yourself? -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: svn 1.7 assertion failed
On 10/17/2011 8:24 AM, Tony Sweeney wrote: -Original Message- From: Ulrich Eckhardt [mailto:ulrich.eckha...@dominolaser.com] Sent: 17 October 2011 16:02 To: users@subversion.apache.org Cc: ponomarenko yaroslav Subject: Re: svn 1.7 assertion failed Am 17.10.2011 16:42, schrieb ponomarenko yaroslav: Got this error while trying svn cleanup svn: E235000: In file 'D:\Development\SVN\Releases\TortoiseSVN-1.7.0\ext\subversion\subversi on\libsvn_wc\workqueue.c' line 672: assertion failed (checksum != NULL) win7 64 Is there a way to recover? This has been reported ad nauseam. Please do as the message box tells you to, i.e. tell as much as possible about what you did and please first search the web before being the onehundredandtwentieth to report the same issue. - 1). There is no search facility in the Apache mailing list archive 2). Searching for his exact error on svn.haxx.se returns zero results 3). Searching for his exact error on gmane.org returns zero results The error message clearly states: Please take the time to report this on the Subversion mailing list. The caveat to search the mailing list is clearly worthless if there is no obvious way to do what it suggests. Tony. Major search engines crawl the archives. This seems to be an obvious way. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: svn merge operation extremely slow
On 10/2/2011 2:08 PM, Kyle Leber wrote: I was able to capture a profile from svn (after remembering I have to link statically). I compiled with -pg -O0 Here is the top of the file: Each sample counts as 0.01 seconds. % cumulative self self total time seconds secondscalls s/call s/call name 88.88133.49 133.49 2002891836 0.00 0.00 svn_diff__snake 5.97142.45 8.96 56 0.16 2.54 svn_diff(long, char, short) 1.98145.42 2.97 4163001 0.00 0.00 MD5Transform 0.41146.04 0.62 4163001 0.00 0.00 Decode Is it OK to attach the full report to this user list? The resulting text file is 1.3MB and I wasn't sure if the list would tolerate an attachment of that size. It's a weekend, so you might not get a lot of replies from people who know SVN source code, but it's likely that the full report won't be needed. There are two billion (!) calls to svn_diff__snake(), and the question is why there are so many. It might help the devs if you pasted in the entries for functions which directly called svn_diff__snake() (quite possibly svn_diff() only) and perhaps the functions which svn_diff__snake() called directly (none of any significance, if I read the above report correctly). This should be only a few dozen lines of the report. Note that you'll have to trace through the report (the top-level function is listed first, followed by its children, grandchildren, etc.) to find the entries for these functions. I have a suspicion that one of the devs will be able to identify the issue from just the above report, but a little more information might turn out to be helpful. They certainly won't need to see information for all of the zillion functions in SVN. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: File access control
On 10/1/2011 6:07 PM, Grant wrote: Ouch. If testing is done on the developer's local machine, he will need a full working copy and it will be impossible to restrict read access to one or a few files as I had planned to do with path-based authz. Instead, could I check out a working copy of the repository to a staging machine of mine and allow the developer to check out only one or a few files as defined via path-based authz, thereby requiring him to commit his changes and update the staging machine's working copy in order to test his changes and debug? I suppose each developer would require a separate staging machine to work this way. If a developer has access to only part of the Web site, he/she will be able to test only part of the Web site. That is not scalable in the long run. As soon as one part of the Web site relies on another for functioning, your development methodology will break. I'm really getting the square peg round hole feeling. The workflow you describe is elegant and very enticing but it gives all of my code away. The prevailing attitude seems to be that developers should be trusted, but this goes against everything I've learned about system administration where security is taken seriously and trust doesn't cut it. I could go the NDA/NCC route but that seems only slightly more secure than trust to me. Gives all my code away is really an exaggeration; as soon as you have employees (contract or otherwise), there will be more than one person who knows the code. It is said that the most valuable assets of Silicon Valley corporations go home at night - i.e. it is the employees and their knowledge who are valuable, and not the source code. I personally know of one major corporation with a 15 million line code base and no one person understands all of it (quoted from an employee). The developers in that group can see all of the source code if they need to, but it is their collective knowledge that keeps the program valuable and not the source code itself. The company has NDAs and other legal agreements with its employees and contractors, and all of them know they will be sued if they walk away with that source code. And that is all the company can do. The walled-off approach can only work if the pieces are truly independent, and that requires significant design effort up front. If your files are all in one directory, you're just asking for a security leak (by your definition) - one mistaken permission setting, and that secret is out. Directory-level permissions are much easier to manage because there is only one permission setting to track. Security is simpler when there are fewer points of attack; a file-level system is going to be wide open. All of the big enterprise websites allow each of their developers to check out a full working copy of the company code with only an NDA/NCC to protect them? It would be so easy for any of them to use, sell, or give the code away, or even to accidentally allow an unauthorized person access to it. I can't speak for Web sites, but in the rest of the software world, developers can see all of the source code for the product they are working on. Otherwise they can't debug it; the first time a crash occurs in some other code, the developer is dead in the water until the error can be handed over to a responsible party. Basically, if it is in the same executable, the developer needs to see the source code. It's tempting to protect your interests by blocking all access except for a very small window, but the productivity cost is generally judged to be too high. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: File access control
On 10/1/2011 8:26 PM, Grant wrote: Ouch. If testing is done on the developer's local machine, he will need a full working copy and it will be impossible to restrict read access to one or a few files as I had planned to do with path-based authz. Instead, could I check out a working copy of the repository to a staging machine of mine and allow the developer to check out only one or a few files as defined via path-based authz, thereby requiring him to commit his changes and update the staging machine's working copy in order to test his changes and debug? I suppose each developer would require a separate staging machine to work this way. If a developer has access to only part of the Web site, he/she will be able to test only part of the Web site. That is not scalable in the long run. As soon as one part of the Web site relies on another for functioning, your development methodology will break. Your post has given me a lot to think about, but I don't agree that one part of the website relying on another part means the development methodology will break. Whenever something needs to be worked on, I would give the developer access to all necessary files. If it turns out I didn't grant access to enough files, I can always add one or more files to the accessible list. Can you give me an example of a situation that would break my proposed development methodology? Easy: you go out to lunch right before your developer finds he/she needs a file. Now you're paying for someone to sit and wait for you to come back from lunch. Any time you are not immediately available to change the permissions, you are losing money. Not scalable means that the management burden will grow tremendously as the amount of code in your Web site grows. You may be willing to adjust the permissions list regularly if there are only 100 files, but what if there are 1,000? Or 10,000? What if there are 20 developers? Or 50? What if some of them are in Europe, or India? For every project, for every developer, you are going to have to build a list of files to access, and that list is likely to need tuning as the project proceeds. Limits on source code access are very common; for example, a developer working on analysis software doesn't need to see the internals of the GUI (and vice versa). Usually the source code for different products is in different directories, making access control manageable. It requires some work to partition code, but the way of thinking it induces is good for Software Engineering (as opposed to hacking something together). I'm not a commercial Web site developer, but I would expect that any feature which involves a fair amount of computation is going to require quite a bit of code. You're going to want to isolate that code (with APIs etc.) for testability anyway; why not move it into a separate directory? Look back at the statement you made above: you are willing to make testing more difficult in order to protect your source code. If testing is harder, less will be done - it's human nature. That could easily degrade the quality of the product to be delivered. Most companies accept the theft risk (using the legal system as a hammer) in order to get a higher quality product. That's why you're finding so little support for your methodology in the tools. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: svnserve update + commit hangs system
On 8/10/2011 8:12 AM, Markus Rännare wrote: Hi, The last weeks we have been seeing troubles with our svn server (svn:// server). The repository is quite large (158gb of data), and many files are quite big. The problem if a person commits a large file (I've maily tested files in the size 200-500mb) to the server as someone else makes a update, then the whole server computer grinds to a halt and becomes unresponsive. So unresponsive that pressing the numlock button on the server keyboard doesn't light the keyboard light, and if the screen is running, the same screen is just shown with no signs of showing any responsiveness. This has resulted in me running to the server-room rebooting the server all day long today. Does anyone have any idea how to debug this issue? I tried getting a log, (giving log-file to svnserve), that didn't give me much. Any help would be GREATLY appriciated. Kind regards/ Markus Rännare How much memory does the server have, and is it being entirely consumed when the server misbehaves in this way? Is the disk drive screaming while this occurs? These are signs that you don't have enough memory and the operating system is page faulting. Can you run the top command on a terminal window connected to the server, or logged into the console screen? It will tell you how much memory is free and how much swap space is being used. There will probably be a swap process at the top of the process list if you are page faulting to any degree. The svn:// server process may also appear here; you can look at its process time and memory consumption. Even if you're not running out of memory, you might be able to get more information by watching processes with the top command. I am of course assuming a Linux/Unix machine as the server; under Windows you can get the same thing by running the Task Manager. In both cases you want the system monitor running before a problem occurs. If you can reproduce it reliably, so much the better - you don't have to worry about waking up the screen from screensaver state. Just launch the monitor process, trigger the problem, and then watch on your console screen. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Problems compiling 1.7.0 on redhat el4 64bit
On 8/10/2011 4:12 PM, michael_rytt...@agilent.com wrote: It is set to 1 On Aug 10, 2011, at 3:51 PM, Philip Martinphilip.mar...@wandisco.com wrote: michael_rytt...@agilent.com writes: If I disable optimizations by doing make CFLAGS=-O0 the program no longer crashes. That suggests it could be a compiler bug. Sorry to jump in so late, but I had significant problems with optimization for 64-bit executables under CentOS 4.x. If you can't upgrade, I suggest dialing down the optimization flags. I had to drop from -O3 to -O2 so that Crypto++ 5.5.2 would run, for example. At some point I could drop support for RHEL/CentOS 4.x from my client's products and try going back to -O3, but the required testing hasn't been real high on the priority list. Assuming the SVN configuration scripts can distinguish between the releases, I suspect that changing the default optimization flags for RHEL 4.x will be simpler than trying to fix the new UTF-8 code. Have you tried intermediate levels of optimization? We went to production with -O2, so it's not as if all 64-bit optimizations were broken. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: My Backup Script
On 7/25/2011 11:33 PM, Andy Canfield wrote:
For your information, this is my backup script. It produces a zip file
that can be tranported to another computer. The zip file unpacks into
a repository collection, giving, for each repository, a hotcopy of the
repository and a dump of the repository. The hotcopy can be reloaded
on a computer with the same characteristics as the original server;
the dumps can be loaded onto a different computer. Comments are welcome.
The dump should use the hot copy as its source. Otherwise it may differ
from the hot copy. See my note inline.
#! /bin/bash
# requires root access
if [ ! `whoami` == root ]
then
sudo $0
exit
fi
# controlling parameters
SRCE=/data/svn
ls -ld $SRCE
DEST=/data/svnbackup
APACHE_USER=www-data
APACHE_GROUP=www-data
# Construct a new empty SVNParent repository collection
rm -rf $DEST
mkdir $DEST
chown $APACHE_USER $DEST
chgrp $APACHE_GROUP $DEST
chmod 0700 $DEST
ls -ld $DEST
# Get all the names of all the repositories
# (Also gets names of any other entry in the SVNParent directory)
cd $SRCE
ls -d1 * /tmp/SVNBackup.tmp
# Process each repository
for REPO in `cat /tmp/SVNBackup.tmp`
do
# some things are not repositories; ignore them
if [ -d $SRCE/$REPO ]
then
# back up this repository
echo Backing up $REPO
# use hotcopy to get an exact copy
# that can be reloaded onto the same system
svnadmin hotcopy $SRCE/$REPO $DEST/$REPO
# use dump to get an inexact copy
# that can be reloaded anywhere
svnadmin dump $SRCE/$REPO $DEST/$REPO.dump
svnadmin dump $DEST/$REPO $DEST/${REPO}.dump
I generally use curly braces when punctuation is present to make sure
variable substitution occurs the way I want it.
fi
done
# Show the contents
echo Contents of the backup:
ls -ld $DEST/*
# zip up the result
cd $DEST
zip -r -q -y $DEST.zip .
# Talk to the user
echo Backup is in file $DEST.zip:
ls -ld $DEST.zip
# The file $DEST.zip can now be transported to another computer.
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Re: Moving Repositories to New server
On 7/26/2011 8:08 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 10:35:02AM -0400, Phil Pinkerton wrote: Are there any know issues with regards to moving Repositories from one platform to another ? Will the old Repositories maintain their current SVN revision ? Current platform Sun Solaris 10: SVN 1.6.5 Target platform Red Hat Enterprise 5 SVN 1.6.17 ( Subversion Edge ) snip If your repositories are in FSFS format (check the file db/fs-type) you don't need to perform a dump/load cycle. It doesn't hurt, but it can take longer than copying the repositories directly. If the processor architectures differ, copying the repositories directly won't work unless changes to the repository format have been made recently. I had a problem when copying a repository from a 64-bit x86 machine to a 32-bit x86 machine, for example. (I think this was in 1.4.x, but it was years ago and I don't remember the details.) Solaris 10 suggests a SPARC machine as the source. Because of the byte ordering difference, I'd expect major trouble when copying a repository directly from a SPARC machine to an x86 machine. I've used (and recommend) the svnsync approach recommended by another poster. Downtime can be even less than when copying repository directories directly. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Moving Repositories to New server
On 7/26/2011 9:48 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 09:22:04AM -0700, David Chapman wrote: On 7/26/2011 8:44 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 08:35:31AM -0700, David Chapman wrote: If the processor architectures differ, copying the repositories directly won't work unless changes to the repository format have been made recently. I had a problem when copying a repository from a 64-bit x86 machine to a 32-bit x86 machine, for example. (I think this was in 1.4.x, but it was years ago and I don't remember the details.) Solaris 10 suggests a SPARC machine as the source. Because of the byte ordering difference, I'd expect major trouble when copying a repository directly from a SPARC machine to an x86 machine. This only applies to repositories using the BDB backend. There is no such problem with the FSFS backend because it uses flat files. The bad copy was a FSFS repository. IIRC, the problem was writing binary data (e.g. integers) into files. The 64-bit machine wrote 64-bit integers into some of the files, and the 32-bit machine got confused. I don't think there are any platform dependent bits in an FSFS revision file itself. It's much like you can copy, say, a PDF document or a jpg image via the network from one platform to another and have it work. Mounting a big-endian filesystem that was written on a sparc on an x86 box is of course a different story. It might work, or it might not. There is nothing an application can do to account for that though. It would be good to know what really went wrong in your case. Maybe something went wrong during the copy process? How was the repository transferred? Over the network (good idea), or via a disk that used a big-endian filesystem (possibly a bad idea if the x86 box has no support for reading it properly despite the byte ordering difference)? The original copy was over a local area network, using tar cf - | ssh on the 64-bit machine and pushing the files to the 32-bit machine. My notes then say something to the effect of trouble - redone with dump and load cycle. This was in January 2007 with Subversion 1.3.0 on both ends. I never tried the directory copy again. I don't have better notes about what problems I found. The FSFS spec links are interesting and on first reading suggest portability, but I don't have enough time to study them in detail. I'd have to look at all of the specs for repository files to be sure. For now I back up my repositories nightly using hot copies and full dumps (about 2 GB total dump size, so still feasible). One of the repositories came from a remote hosting service via svnsync; we decided that local hosting was better. I don't use svnsync locally now because all but one of my machines are powered off at night, but it worked without any problems then. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Setuid
On 7/22/2011 8:38 AM, Andy Canfield wrote: Had what seems to be a bright idea. It is a bright idea for a Subversion server on Linux or OS X; AFAIK this idea has no relevance to Windows: Take note of the user and group that Apache runs as. Call this combination APACHE, meaning APACHE_USER and APACHE_GROUP. Whatever mod_dav_svn does to any repository will be done by APACHE. Whatever WebSVN does to any repository will be done by APACHE. If I set the svnserve program to be owned by APACHE, and setuid and setgid, then whatever svnserve does to any repository will also be done by APACHE. Only root, or the APACHE user, can make this change to the svnserve program binary. If I set the svnadmin program to be owned by APACHE, and setuid and setgid, then whatever svnadmin does to any repository will also be done by APACHE. If I set the svnlook program to be owned by APACHE, and setuid and setgid, then whatever svnlook does to any repository will be done by APACHE. So the three access paths - http:, svn:, and direct - will all operate using the same user and group. You don't want to do this to any program, such as 'svn', which relies on user authentication, since it needs to know the actual user that is running the program. But any program which operates directly on the repository can be set this way and that ensures that the repository is always manipulated, at a low level, by the same user all the time. So a post-installation setup would include: *sudo bash cd /usr/bin chown APACHE_USER svnadmin svnlook svnserve chgrp APACHE_GROUP svnadmin svnlook svnserve* This idea also allows me to make the the respository itself accessable only by APACHE. This ensures that the repository can only be manipulated by Subversion code itself (or by root): *sudo bash mkdir /var/svn chown APACHE_USER**/var/svn* *chgrp APACHE_GROUP**/var/svn* *chmod 0700 /var/svn* If I also do this: *sudo bash rm /usr/bin/svnadmin* then all repository creation would have to be done via something like WebSVN, which I assume requires authentication. How does that sound to you guys? Where are the flaws in this idea? Thank you very much. It prevents privately owned and managed repositories. Normally, individual users can set up their own repositories for their own (personal) projects or files. If you are planning to do this on a server that you wish to lock down, then I see no reason why it would cause problems, but if users can login and do other work on this machine, you are constraining them. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: empty pre-commit hook fails with svn+ssh with some accounts
On 7/21/2011 4:00 PM, Daniel Neuberger wrote: On Thu, Jul 21, 2011 at 2:13 PM, Nico Kadel-Garcianka...@gmail.com wrote: Don't give the shared svn user a valid shell If an administrator needs to run operations as that user, to manipulate config files or create new repositories, they can do sudo -s -H -u svn to get a valid shell as the administrative user. Sudo can even be configured to allow designated users such administrative access without requing local root privileges at all. Hmm, why didn't I think of that? It doesn't seem to work though. Setting the shell to /bin/nologin or even just fakeshell breaks everything. Is there another way to give an invalid shell? How about /bin/false? This is the shell defined for all of the non-login (e.g. daemon) accounts on my machines. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: how to maintain two trunks?
On 5/7/2011 12:06 PM, Totte Karlsson wrote: Hi, I'm working on a codebase that is to be used by two different compilers. I have a trunk-branch layout but realize that does not work that well for this. What seems to be needed are two trunks somehow. Is that possible? I would like to be able to avoid to clutter the code with #ifdefs in regards to the compilers, if possible. However, it goes further. Both compilers will for example read certain project files. These files can't be shared. I would like to keep the same naming on files for both compilers too. The problem now is, as being told in this forum, feature branches are to be abandoned after merged back into the tree. There are some ways to avoid that, but it seem to esoteric for me. Anyway, anyone having some suggestions on how to deal with this scenario? Two trunks is no different than having a trunk and a branch; Subversion just tracks the state of a directory structure over time. The directory names you create and the way in which you manage those directories are your decisions. So the troubles you will have trying to keep a branch in sync with a trunk will be the same as those you will have trying to sync two trunks. Don't do it. The usual way of dealing with compiler differences (I use Watcom C++, Visual C++, and g++) is to define your own types etc. so that the conditional compilation is restricted to the places where those types are defined. Macros, used carefully, can also help. If all else fails, define an abstraction layer of your own routines, create copies of this abstraction layer for each compiler, and then call only the abstracted code. The abstraction layer should have as little as possible in it; don't copy any code. The goal in both cases is to minimize the number of places where you have compiler-dependent code. If compiler-dependent code is sprinkled everywhere (either with #ifdef or as differences in trunk 1 vs. trunk 2 copies of the files) then you are most likely not organizing your code in the most effective way. As for project files, I create copies, then the post-checkout script (run manually in my case) renames the appropriate copy. So I would have a makefile.wat, makefile.msvs, and makefile.unx, then rename one of them to Makefile after checking out. Sometimes I set up build scripts, then run the appropriate script and have it use make -f makefile.wat etc. To minimize edits to the makefiles, I often define defs.mk and deps.mk files that do some of the abstraction. So makefile.wat would have something like: !include ..\watcom.mk !include defs.mk !include deps.mk target... Master Watcom configuration parameters are in the parent directory watcom.mk file, and it defines compiler flags, file extensions, and generic build commands (e.g. .cpp.obj). Then the defs.mk file defines the include and object file lists in machine-independent form, while the deps.mk file defines the dependencies in machine-independent form. Anything that cannot be machine-dependent goes into the compiler-specific makefile, e.g. makefile.wat. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: How to create a link that works between OS X and Ubuntu
(moving top posting to bottom) - Original Message - From: richard Cavell Sent: 04/26/11 10:36 AM To: users@subversion.apache.org Subject: How to create a link that works between OS X and Ubuntu Hi everyone. I'm developing a program on Ubuntu 10.10. The directory in which my project lives is part of my PATH. The executable that is built is called 'autobot'. I type: ln autobot a a (And my program runs correctly). svn add a svn propset svn:executable ON a svn ci -m Create shortcut Now on my OS X box, with the current directory set to the project directory, and with that directory also being a part of PATH (although it is not named identically to the Ubuntu one), I type: svn up a And I get: -bash: /source/Autobot/autobotwiki/a: cannot execute binary file ls -l a gives me: -rwxrwxrwx 1 richard admin 55295 26 Apr 10:33 a The same thing happens if I create the link on OS X and try to run it under Ubuntu. So how do I do this? Richard On 4/25/2011 7:38 PM, richard Cavell wrote: Further experimentation shows that symbolic links work (ln -s autobot a for the first command). Are hard links supposed to work? Richard The hard link simply creates a new name for the file, which is probably operating system dependent (you didn't describe the build process completely). Subversion won't know the difference between the name autobot and the name a; each will look like an ordinary file. A symbolic link, however, is a different object type and Subversion can store it as such. Try this: ln autobot a1 ln -s autobot a2 ls -l The link count for autobot and a1 will be 2; each name references the same file on disk. The symbolic link, however, is a pointer to a name. You can replace the file autobot without affecting a2, but if you replace autobot (rm autobot; make autobot) you will find that the connection between autobot and a is broken. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Hosting multiple subdomains with a common SVNParentPath under apache
On 3/10/2011 12:59 PM, Daniel F. Garcia wrote: I have an apache server that has an alias of *.mydomain.com. What I want is to setup svn paths that switch to different repositories according to the subdomain. E.g. http://subdomain1.mydomain.com/svn - /var/svn/subdomain1 http://subdomain2.mydomain.com/svn - /var/svn/subdomain2 http://subdomain2.mydomain.com/svn - /var/svn/subdomain3 I setup a repository that used the SVNParentPath directive, then had a bunch of mod_rewrite rules to do all the mapping. I almost got it working. It worked in a web browser and repo browser but I could not do a checkout. Any ideas for a different approach? Has anyone done this successfully? Daniel F Garcia I don't use SVNParentPath because I have a limited number of repositories and don't add them on a regular basis. Instead I use VirtualHost directives: VirtualHost 1.2.3.4:80 ServerName subdomain1.mydomain.com Location /svn DAV svn SVNPath /var/svn/subdomain1 ... /Location ... /VirtualHost Note that VirtualHost directives are incompatible with https: access because the destination host name is encrypted with the rest of the packet data and Apache can't decrypt the packets before directing them. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: pre-lock.bat Failed in Repo browser
On 3/8/2011 6:24 AM, Waseem Bokhari wrote: I don't think %VISUALSVN_SERVER% is set in your lock script. What is meant by this? How can I found svnlook.exe?? Can you please re-edit this script for me. Thanks in advance What he means is that %VISUALSVN_SERVER% will not have a value when running from a hook script. From a Command Prompt window, type: echo %VISUALSVN_SERVER% Then substitute the returned value into your script. If you run this echo command in the hook script during a commit operation, it will not print anything. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Help with files changed outside of svn
On 2/28/2011 8:13 AM, Christopher D Haakinson wrote: I mean that I have a working copy of my project under: /svn/hooks This folder gets automatically updated with svn update on every commit with a post-commit hook This folder also is live, meaning that the files here are in use at all times, so nobody will be using this working copy as their development copy, it's just there to receive all the newest changes. The scenario I want to test and know how to handle is when a file in /svn/hooks gets changed outside of svn and these changes never get commited into svn. I understand what svn is and is not, but the files I will have inside svn will also be available and writable on a network, so I need to know how to handle files that get changed inside my live working folder outside of svn. This is asking for trouble. You want your Subversion repository to be up and running at all times. Version control systems cannot prevent all merge conflicts, so your repository could very well go down whenever a conflict occurs. The repository control directories (such as the hooks directory) should be tightly controlled, not open to any and all modifications, and certainly not changing on the fly. As Andy Levy said, Subversion does not care how the files are modified. It is a lightweight system, meaning that it doesn't run unless you tell it to. And so it doesn't know that files in a working copy are modified until you tell it. Also is there a way to change what gets written to a file when this occurs? All my scripts get corrupted because of the and I'd like to at least put a # in front so that it's a commented line and doesn't affect the execution... This particular conflict might not have affected execution, but you can't guarantee this will always occur. What if you have a conflict of the form: exit 0 vs. exit 1? How is Subversion supposed to know which is correct? This is why conflict resolution is manual. Trust me - you really, really, want to have conflicts be obvious and cause problems; you don't want them hidden. If you really want to have your repository's hooks directory updated through Subversion (which I do not recommend), make the directory writable only by the process that is running Subversion so that only the post-commit hook script can change it. Don't *ever* allow users to modify files in that directory. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: can svnadmin dump tolerate read-only repository access?
On 2/28/2011 10:37 AM, James Ralston wrote: snip Question: if I svnadmin dump a repository, is it safe to allow read-only access to the repository while the dump is in progress? Or is it the case that permitting even read access could interfere with the svnadmin dump operation? I ask because it will take a non-trivial amount of time to perform svnadmin dump/load operations on all of Subversion repositories. If we can at least permit read-only access while this upgrade is in progress, that will have much less of an impact on our users then it we have to block all access during the upgrade... A hot copy by definition is one that will safely allow read-only access while the backup is in progress. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: can svnadmin dump tolerate read-only repository access?
On 2/28/2011 11:00 AM, James Ralston wrote: On 2011-02-28 at 10:49-08 David Chapmandcchap...@acm.org wrote: A hot copy by definition is one that will safely allow read-only access while the backup is in progress. But I asked about svnadmin dump, not svnadmin hotcopy. Are you asserting that read-only repository access (in our case, via WebDAV via mod_dav_svn) will *not* perturb a running svnadmin dump operation? Sorry, missed that detail. Per the book: http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.tk.svnadmin Note that *svnadmin dump* is reading revision trees from the repository just like any other reader process would (e.g., *svn checkout*), so it's safe to run this command at any time. You may have to scroll down a bit to find this text. I always do a hot copy first; since no one else can access the hot copy I don't have to worry about svnadmin dump conflicts. Of course, my repositories are still writable while I'm running my backups. The main issue as I understand it is determining exactly which revision is the last and not copying any data beyond that one. For you, hot copy plus dump would take longer, which would be an issue for your developers. But if you prevent commits (including revprop changes) during the transition, you should be fine. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: ^M Appends to every line?
On 2/24/2011 8:02 AM, Christopher D Haakinson wrote: OK, so I've been testing out the svn:eol-style prop and it appears to work, but it seems like an awful lot of work for such a simple issue. Is there something server-side I can setup to ensure that all files contain the correct eol style? Also I've noticed that my shell scripts are now failing with an EOF error? Does this mean that there's a style setting for the end of file too?? You can define a pre-commit hook script that looks at the file name and then verifies the property is present. These are described in the Subversion book. Multi-platform work is an awful lot of work; it is not as simple as it seems. Heuristics to determine whether a file is plain text can fail, with catastrophic results. File transfers done carelessly will corrupt binary files; in integrated circuit design the OASIS geometry file format has an almost text string defined solely to catch this error. If you try to use the same text files across platforms, things will fail unless *every* tool you use - all editors, all file analysis software, all file transfer programs - deals with mixed or wrong platform line ending styles properly. This is a high standard that has never been met in my experience. I haven't seen script errors related to end of file; Windows no longer puts a ^Z at the end of files, so you shouldn't need to strip that out. Have you done an octal dump of the scripts to see what is at the end of the files? On which platform are they failing - Windows or Linux? -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: ^M Appends to every line?
On 2/24/2011 9:55 AM, Christopher D Haakinson wrote: I'd like someone to explain how this small shell script, which works fine, gets corrupted simply by creating a new file and copy/pasting the text in it. Here's what I'm doing: 1) I have a test shell script that runs fine. Here's the content: - start lock=/tmp/deleteme if [ -f $lock ] then echo Lock file exists. Wait until it's gone to proceed. . while [ -f $lock ] do echo Waiting for lockfile to be removed. . sleep 1s done echo Now creating lock file echo locked $lock else echo No lock file found. Creating it. . echo locked $lock fi echo Lets try this again, hopefully for the last time exit 0 -- end 2) I copy this entire script from notepad in windows into a new file named test2.sh, also in notepad. 3) Using TortoiseSVN, I add test2.sh into my project 4) I commit the changes to the server 5) On my linux server, I run: svn update ... to get the test2.sh file 6) Now when I try to run it I get this: syntax error: unexpected end of file Now, this is the most simple task I could think of doing, and this doesn't work. I've also tried creating the new shell script with Komodo EDIT with the same results. This occurs even though I've tried adding svn:eol-style from both my linux server and my windows wrx... I'm lost!! In Notepad, can you move your cursor below the last line of text in the file? If not, there won't be a newline after the final line. Under Linux, try od -c test2.sh | more and verify that there is a \n at the very end of the listing. If not, you'll need to add one in Notepad, Komodo, or your favorite Linux editor. I have no idea why you would see this problem only with svn:eol-style defined (if in fact this is the problem). -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: ^M Appends to every line?
On 2/23/2011 8:19 AM, Christopher D Haakinson wrote: I have SVN up and running and things are working well, except when editing shell scripts from windows. My server is RHEL5-based and I will have clients connecting from linux and windows. Linux commits work great, and even some times from windows. However some times I commit a file from Windows(running tortoiseSVN and Komodo EDIT) it corrupts the script and adds ^M to the end of every line, often times ruining the structure of the file too. Is there a way to disable this or do I have to run a dos2unix script after every commit to remove them? Look up propset and eol-style. In particular, in the [auto-props] section of my configuration file I have a number of lines of the form *.c = svn:eol-style=native *.txt = svn:eol-style=native *.pl = svn:eol-style=native;svn:executable I do this both in Windows and in Linux so that I don't have files with only ^J when I check them out under Windows. You can also add this property manually after the files are committed: svn propset svn:eol-style=native file1 file2 ... Since you are seeing ^M now, you will need to do this manually for the affected files. Other useful commands: svn proplist file svn propget svn:eol-style file where file is the name of the file you are looking at. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: ^M Appends to every line?
On 2/23/2011 4:44 PM, Nico Kadel-Garcia wrote: On Wed, Feb 23, 2011 at 11:39 AM, Les Mikeselllesmikes...@gmail.com wrote: On 2/23/2011 10:19 AM, Christopher D Haakinson wrote: I have SVN up and running and things are working well, except when editing shell scripts from windows. My server is RHEL5-based and I will have clients connecting from linux and windows. Linux commits work great, and even some times from windows. However some times I commit a file from Windows(running tortoiseSVN and Komodo EDIT) it corrupts the script and adds ^M to the end of every line, often times ruining the structure of the file too. Is there a way to disable this or do I have to run a dos2unix script after every commit to remove them? Long version: http://svnbook.red-bean.com/nightly/en/svn-book.html#svn.advanced.props.file-portability Short version: set the svn:eol-style property to native on the files where you want subversion to manage line endings. Your client may have a list of file suffixes where this would be set automatically. But in general, avoid it. If you're in a mixed platform environment, and you are tweaking files back and forth in end-of-line settings when you check them out in UNIX versis checking them out in Windows, you are in for a *world* of hurt. This is a source of enormous confusion for programmers when it works right, on one system, but not on the other due to local re-writing. If you're on the UNIX or Linux sides, the dos2unix and unix2dos utilities are available with almost every distribution. For Windows, there are other tools, including the same tools under CygWin. Uh, no. Use of svn:eol-style avoids a world of hurt - programmers do not have to run a script *every* time they check out a file. Requiring users to run a script to fix line endings in every sandbox is a recipe for disaster. dos2unix and unix2dos are precisely the kind of local rewriting you want to avoid. My two cents (and one million lines of code) worth... -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: svn status comes out empty
On 2/14/2011 7:23 PM, Ed wrote:
Hi,
Can someone check if this is a bug?
Script to reproduce:
~/test$ svnadmin create bar
~/test$ mkdir foo
~/test$ cd foo
~/test/foo$ svn co file:path/to/bar
~/test/foo/bar$ echo testing this file A.txt
~/test/foo/bar$ svn add A.txt
~/test/foo/bar$ svn ci A.txt -m test
~/test/foo/bar$ find -type f -exec sed -i 's/testing/running/g' {} \;
~/test/foo/bar$ svn status
~/test/foo/bar$
But A.txt is changed.
What should happen is
M A.txt
What happens:
Nothing.
Is this what is supposed to happen?
Your script is not supposed to go into the .svn directories within the
sandbox. You have changed the reference copy of A.txt in the .svn
directory and it is likely that svn status isn't noticing that it no
longer matches the repository. See:
http://myridia.com/dev_posts/view/687
So you would use:
find// -name .svn -prune -o -type f -exec sed -i 's/testing/running/g' {} \;
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Re: mod_dav_svn + automatic per-directory user authorization
On 2/12/2011 8:34 AM, m irya wrote:
The problem:
1) there's a directory on the server containing multiple svn
repositories roots, say /var/svn, with /var/svn/a, /var/svn/b, etc.
being repository roots
2) these repositories are accessed via HTTP, handled by Apache2 + mod_dav_svn
3) there's a MySQL user-password database we need to use to
authenticate access to those repositories
4) at last the problematic place itself: the repository
/var/svn/$username must be accessible (both read and write) only to
those who authenticated themselves as $username with a valid
password from the MySQL database
Currently i've stuck with two solutions (both incomplete) from tons of
manuals and how-tos:
a) AuthzSVNAccessFile could be a key to such an automatic repository
- user binding, but updating the access rights file on changes to
MySQL database looks no way elegant, and will probably become a
bottleneck with growing users database
b) mod_auth_mysql (whatever one) makes it possible to transparently
use the users database in Apache config, but still I'm unable to find
a way to automatically Require user $username for a given repository
/var/svn/$username.
Please reply, if someone has an experience with such a configuration,
any ideas/thoughts are welcome.
I'm not sure I understand the problem. Except for not using mysql to
access passwords, this is what I have now, and it works fine (with one
very important caveat: see below). Here is part of my httpd.conf file:
VirtualHost 1.2.3.4:80
ServerName repos1.mydomain.com
ServerAlias mydomain.com
ServerAdmin ad...@mydomain.com
# just a placeholder:
DocumentRoot /home/user1
Location /user1
DAV svn
SVNPath /home/user1
AuthType Basic
AuthName Subversion repository
AuthUserFile /etc/passwd.user1
Require valid-user
AuthzSVNAccessFile /home/user1/conf/access.conf
/Location
Directory /home/user1
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
/Directory
# log user1 operations separately from other users' operations
CustomLog /var/log/httpd/svn_logfile %t %u user1 %{SVN-ACTION}e
env=SVN-ACTION
/VirtualHost
and the file /home/user1/conf/access.conf has:
[user1:/]
user1 = rw
There's been some name changing here, so this configuration has not been
tested in this precise form :-) but you get the idea. Every repository
has its own VirtualHost and its own access file; unless you have the
password for the one user in the repository, you can't read any of it.
You don't need to have VirtualHost entries for each repository; I host
multiple domains on a single machine and so I chose to have a different
host name (e.g. user1.mydomain.com) for every repository. This also
allows me to have user-specific access logging with CustomLog. You
could simply have a set of Location entries at the top level of the
httpd.conf file.
Note: VirtualHost with name-based hosts is incompatible with SSL
access (https://). I have a small number of repositories (and some
spare IP addresses), so in theory I could use network adapter aliasing
with ifconfig, but I haven't got that far yet. Last time I tried
certificates I made a mess, and I spent too much time last month working
on computers anyway.
Here's the big caveat: whenever a repository is added or removed (in
your case, every time a user is added or removed), httpd must be
restarted because a Location entry must be added or removed. This
could be the big bottleneck, and if you use httpd to serve your
repositories, it's unavoidable. I know that some publicly accessible
Subversion hosting services (e.g. unfuddle.com) use http:// access, but
I don't know how they manage adding or removing users (unfuddle.com
advertises up and running in as little as one minute).
So you need some method of getting the username and password from mysql
to httpd (replacing AuthType and AuthUserFile), but otherwise it's
pretty straightforward.
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Re: mod_dav_svn + automatic per-directory user authorization
On 2/12/2011 1:11 PM, m irya wrote: Yep, i could manage to get to such configuration, but here's when it comes to the real difference with your setup: we need to manage a huge and frequently changing user database, with each user having a small repository. Here's the big caveat: whenever a repository is added or removed (in your case, every time a user is added or removed), httpd must be restarted because aLocation entry must be added or removed. And that's what i'm trying to solve: the users database is not so tiny, so 1) we can't afford restarting httpd on every user added or removed 2) we can't afford duplicating the rules in authz svn access file, i guess it may be handled inefficiently when it contains too much records (at least much less efficiently than just requesting the authorization from MySQL). SVNParentPath will allow you to store multiple repositories under a single master directory, and the Subversion book says this is a good way to avoid restarting httpd when adding new repositories. I'd be very leery of _removing_ repositories without restarting httpd, however - what would happen if a transaction were being processed at the moment you deleted the repository directory? You're expecting mod_dav_svn not to crash in a way that brings down the server. I'll try to describe it the other way, one abstraction level upper: 1) the user goes to http://somehost/repo/$something 2) authenticates him/herself as $username:$password (suppose Basic auth, just will change it to https later) 3) Apache checks the credentials against MySQL database (mod_auth_mysql?) 4) If everything is ok, the user gets a WebDAV access to /var/repo/$username/$something (regardless of the method, read or write) The database may change frequently, adding the user will mean `svnadmin create /var/repo/$username`, removing will mean `rm -rf /var/repo/$username`, but i prefer that nothing is done beyond that, especially no httpd configuration changes and restart, all information is stored in DB and nothing else should be changed. Repositories are meant to preserve data, implying they are relatively long-lived. Are you suggesting that repositories will be deleted all the time? If so, a master server-based configuration like httpd may not be appropriate for you at all. You may need something more dynamic like svnserve via inetd (look for that subject heading in the Subversion book). I don't have any experience with that, however. Your access requirements (many small repositories, many users, fine-grained path-based authorization) don't sound like what Subversion is designed for. It may not be an appropriate tool for you. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: svnadmin create complains about subrepositories
On 2/10/2011 10:19 AM, Fabian Richter wrote: Am Thu, 10 Feb 2011 13:07:27 -0500 schrieb Bob Archerbob.arc...@amsi.com: I expect that this all happens inside the repository. A repository's contents can have whatever folder layout you want. However, the repositories themselves on the server each need to be peers. You can't nest repos nor should you need to. To be frank: I dont see why I am not able to create nested repos or what you mean that they need to be peers... You say a repo folder can have the content i need it to have. I need them to have other repos. To be sure we are talking about the same thing: /repos1 /repos1/repos2 /repos1/repos3 something like that. And the comparison to mysql DB doesnt make sense, because in a dbms I am able to deklare multiple databases for the same server or root dir, thats what is apparently not possible for svn... In Subversion, the subdirectories are stored within a single repository. You are not creating a repository that holds a single directory of files; a repository is a versioned file system that can have subdirectories within it. One repository directory = multiple controlled file directories in a virtual, versioned file system. Rather than nested repositories as you suggest, the usual way of doing things is to have subprojects within a single repository: /repos /repos/subproject1 /repos/subproject2 /repos/subproject3 These are all virtual directories, and you can check out or traverse them separately even if they are managed by a single repository and its server. Otherwise you would have multiple repositories stored in directories which are peers of each other (cf. SVNParentPath in httpd.conf), and your workspace setup script would check out the necessary directories from the individual repositories. Storing repository files (which are a kind of database, BTW) inside a directory that hold files for another repository is very likely to cause problems someday. The repository files are managed by the Subversion software, and it has certain implicit assumptions about what will be there. It's not expecting arbitrary data in arbitrary places. Support of this feature would greatly constrain the developers and I wouldn't expect them to like it (I know I wouldn't, if I were a Subversion developer). -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Need help in restoring the svn repository (server side)
(reordering to remove top-posting) *From:* ankush chadha ankushchadha2...@yahoo.com *To:* users@subversion.apache.org *Sent:* Fri, February 4, 2011 1:47:34 PM *Subject:* Need help in restoring the svn repository (server side) Hi All I am trying to recover the repository from a corrupted hard drive. I have very huge source repository, about 136000 revisions. Luckily I have a 4 month old backup. I think I was able to recover the contents of db/revs and db/revprops folder as it contains 136000 + 1 files each. ( 1 file per revision and a 0 revision file) There is a node-origins folder that has a bunch of files. I am not sure if I was able to fully recover files under this folder. After restoring the files, when I booted up the svn server, its reading till 133000 revisions. Its not recogizing the revisions after 133000. Does anyone know how to sync that up? Thanks AC On 2/4/2011 11:56 AM, ankush chadha wrote: Found that there is a file named 'current' that stores the HEAD revision. When I kicked off svn verify on 133001 it complained that revisions must not be greater than the youngest revision so I knew they stored the HEAD revision somewhere. Once I updated the value of HEAD, I can see all the revisions :) Running a svn verifier on the entire repository to make sure that nothing else is corrupted. Ankush From this I hope you (and everyone else reading this list) learned that you need to be backing up your repository much more often. Success should not be based on luck. You need to think about how much work you're willing to lose, worst-case, should your server hardware crash or otherwise go offline. Then back up more often than that. If you're not willing to lose as much as a day's worth of work then you need to backup multiple times per day, and those backups need to be copied off the server. Every night I copy my repositories using hot-backup.py, then dump the repositories and copy the dump files off the server. Once a week copies of the dump files go off-site (along with full backups of all my other data). I never have all copies of my data in the same place at the same time. The worst-case scenario is having the building catch fire as I'm doing weekly backups, taking all of the computers with it and forcing me to redo a week's work. But I can live with that; I work solo and do a couple dozen commits a week. With 136,000 revisions in your repository, you should backup your repository to a second machine multiple times per day (or use svnsync) and store at least an incremental backup of the repository off-site once per day. Think about it - you very nearly lost four months of history. Maybe your team could have reconstructed much of the changed data using giant commits from their surviving sandboxes, but that takes a lot of time and effort, is risky, and wouldn't allow you to see why the changes were made or distinguish between changes (e.g. was this line of code implementing a feature or fixing a bug?). -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
missing carriage returns from output of svn diff in Windows
Running Subversion 1.6.15 under Windows 7, if I redirect the output of svn diff to a file and then try to edit that file using Notepad, the line breaks are garbled. In particular they have only the '\n' character, not the '\r' character. For example, if svn_usage_notes.txt is in my repository, and I do the following in a sandbox: echo foo svn_usage_notes.txt echo bar svn_usage_notes.txt echo zot svn_usage_notes.txt svn diff svn_usage_notes.txt wowow.txt od -c wowow.txt I get: 000 I n d e x : s v n _ u s a g e 020 _ n o t e s . t x t \r \n = = = = 040 = = = = = = = = = = = = = = = = * 120 = = = = = = = = = = = = = = = \r 140 \n - - - s v n _ u s a g e _ n 160 o t e s . t x t \t ( r e v i s i 200 o n 1 1 1 6 ) \r \n + + + s v 220 n _ u s a g e _ n o t e s . t x 240 t \t ( w o r k i n g c o p y ) 260 \r \n @ @ - 6 1 2 2 , 3 + 6 1 300 2 2 , 6 @ @ \r \n \n M a r k 320 P h i p p a r d \n h t t p : 340 / / m a r k p h i p . b l o g s 360 p o t . c o m / \n + f o o \n + 400 b a r \n + z o t \n 413 Note how the first part of the file has \r\n while the last part (showing the extra lines) has only '\n'. The file itself has \r\n for every line, and I have the following line in C:\Users\David\AppData\Roaming\Subversion\config: *.txt = svn:eol-style=native The absence of newlines makes it impossible to view the differences file in Notepad. Years ago before I knew about unzip -a, I wrote a program to add carriage returns to text files being copied from Linux to Windows and so I can use that, but it would be nice to have system-appropriate line endings in the output of svn diff without this extra step. Is anyone else seeing this? Is there a configuration parameter I can set to fix it, or is it a bug that should be bumped to the dev list? (For the record, I have UnxUtils.zip loaded, so I have a number of Unix/Linux utilities such as od available under Windows, and only the Windows Subversion executable ever writes into a Windows sandbox.) Thanks. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: missing carriage returns from output of svn diff in Windows
and is running Subversion 1.4.5 from svn-win32-1.4.5.zip (don't know where I got it, but the download file is dated December 2007). Even with the UnxUtils executable directory removed from the laptop shell's search path, I get the same result. It too is a bare Subversion installation operating from the Command Prompt without Cygwin or TortoiseSVN. So this has been going on for a while. The server, BTW, is running Subversion 1.6.11 under Slackware 13.1. This is the build that came with the distro; I didn't see a need to upgrade (though I could if there is a compelling reason). Not that I think server-side is the cause... Thanks for your comments. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Error on Post Commit Hook.BAT
On 2/1/2011 10:54 PM, Waseem Bokhari wrote:
http://www.netsoltech.com///
*From:*David Chapman [mailto:dcchap...@acm.org]
*Sent:* Wednesday, February 02, 2011 11:48 AM
*To:* Waseem Bokhari
*Cc:* users@subversion.apache.org
*Subject:* Re: Error on Post Commit Hook.BAT
On 2/1/2011 10:16 PM, Waseem Bokhari wrote:
_Here is a Post-Commit.bat for generating email on every-commit._
*@echo off*
**
*:*
*::: ARGUMENTS :::*
**
*SET REPOS=%1*
*SET REV=%2*
**
*:*
*::: GENERAL INFO *
*SET DIR=E:\Repositories\WSBokhari\hooks*
*SET MESSAGE_FILE=%DIR%/message.txt*
**
*:*
*::: SVN INFO *
*SET DIR=E:\Repositories\WSBokhari\hooks*
*SET REPO_PATH=file:///E:/Repositories/WSBokhari
file:///E:%5CRepositories%5CWSBokhari*
**
*::: Get the author ::*
*For /F Tokens=* %%I in ('svnlook author %REPOS% -r %REV%') Do Set
author=%%I*
**
*::: Get the log messsage ::*
*For /F Tokens=* %%I in ('svnlook log %REPOS% -r %REV%') Do Set log=%%I*
**
*::: Set the message body ::*
*ECHO Commit - rev %REV% (#%author%): '%log%' - %REPOS% %MESSAGE_FILE%*
**
*:*
*::: EMAIL INFO ::*
**
*set to=-to ntpk@netsolpk.com mailto:ntpk@netsolpk.com*
**
*set subj=-s SVN Commit (Revision %REV%)*
**
*set server=-server mail.netsolpk.com*
**
*set auth=-u waseem.bokh...@netsolpk.com
mailto:waseem.bokh...@netsolpk.com --pw 123 *
**
*set from=-f waseem.bokh...@netsolpk.com
mailto:waseem.bokh...@netsolpk.com*
**
*:*
*::: SEND THE EMAIL ::*
**
*E:/Repositories/WSBokhari/%MESSAGE_FILE% %server% %to% %from% %subj%
%auth%*
**
**
*_Error Message:_*
**
*post-commit hook failed (exit code 1) with output:*
*The filename, directory name, or volume label syntax is incorrect.*
**
**
*_PS:_*
Even file *message.txt***is being creating in my Hooks Folder
according to this::
**
*SET MESSAGE_FILE=%DIR%/message.txt*
//
This line looks wrong. Try using a backslash ('\') instead of a
forward slash ('/'). Although many Windows programs tolerate forward
slashes in file names, the Windows batch processor is not one of
them. It uses forward slashes to mark parameters (as is shown in the
For /F lines) so it is trying to interpret /message.txt as a
parameter, not part of your message file name.
If you're going to be creating or editing Windows batch files, I
strongly encourage you to buy a book on how to create them. It's been
so long that I can't recommend one to you, unfortunately.
--
David Chapman dcchap...@acm.org mailto:dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Even I have changed the slashed but the message was same.
Please advice.
Did you change both lines? I also see forward slashes in the line which
is described as SEND THE EMAIL at the very end.
For that matter, this line looks wrong too. Earlier, you set the
variable DIR to E:\Repositories\WSBokhari\hooks. The value of this
variable will be substituted whenever %DIR% appears. Thus if you have
SET MESSAGE_FILE=%DIR%\message.txt this is the same as SET
MESSAGE_FILE=E:\Repositories\WSBokhari\hooks\message.txt and the value
of this variable will be substituted whenever %MESSAGE_FILE% appears,
as it does on the last line.
If you replace the forward slashes in the last line with backslashes and
perform the substitution of %MESSAGE_FILE%, the first word of the line
will become
E:\Repositories\WSBokhari\E:\Repositories\WSBokhari\hooks\message.txt,
which is obviously an invalid Windows path name.
The message file is clearly not intended to include the name of the mail
program, so you need to supply the name of the E-mail program. I don't
know what that would be. Nor do I know the command-line parameters for
the E-mail program, so I cannot tell you whether the syntax of this line
is correct. But if your E-mail send program is
E:\Repositories\WSBokhari\hooks\send_message.exe and it accepted the
parameters you have listed, the last line of the batch program would be:
E:\Repositories\WSBokhari\hooks\send_message.exe %MESSAGE_FILE% %server%
%to% %from% %subj% %auth%
You really need a book on Windows batch file processing.
//
--
David Chapman dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Re: Error on Post Commit Hook.BAT
On 2/1/2011 11:36 PM, Waseem Bokhari wrote: I am not using any email program But Only using my SMTP SERVER E:\Repositories\WSBokhari\hooks\send_message.exe%MESSAGE_FILE% %server% %to% %from% %subj% %auth% Then how would I mention this? You need to find a program that contacts your SMTP server from the Windows command line prompt and sends a message to the server. You may have one already installed on your machine; if you are defining a repository for a group of developers you will also need to ensure that it is installed (or at least accessible on the search path) on every developer's machine. My Subversion installation does not send E-mail to developers and so I do not know of any programs that send mail to an SMTP server from a Windows command line prompt. Perhaps someone else on the list will know. I can't help you further. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
