Re: [CFT] packaging the base system with pkg(8)
On (19/04/2016 13:36), Adrian Chadd wrote: > It's cool. I have positive and negative reactions, and I'm totally > happy to let people try it out at a larger scale and learn from > mistakes. > > Because, honestly - fuck it, we've been behind for too long. We need > more mature tools and knowledge with this. > > The irony of course is the people rolling out docker are doing it > after finding that OS packages aren't the best granularity, but > "whatever the package systems in the software stack we're using, and > then tar the whole mess up and distribute it" method. Kinda like > FreeBSD It's close to the way I see the packaging issue.. Our use case may not be the most common one -- we sell software in 2U form factor (hardware included) :) Once in a while one has to roll updates for the base system as well as our firmware. Packages has never been much of a help but rather another obstacle to deal with for me personally and I bet for our support as well. The thing I care about most is to have the same base system for all customers who have installed updates. It's really hard to troubleshoot assuming that some package updates might have failed for a customer. Distributing images comes very handy in this situation. Although packages are useful in development: - Being able to install complete development environment into the base system (debug symbols, compilers). - Installing updates on *dev* machines. Which I'd rather avoid for release builds. By having packages for both release and dev one would need to have release and dev package repositories. Much like stable/unstable repositories in many linux distros. Revision control systems tend to be much better at this at much lower cost. Being able to do 'make happyworld whatever' out of the box on dev machine as well as installing packages built by CI is huge improvement comparing to packaging as it's often done in linux. - Keeping any package metadata up to date in dev is painful (version bump in most cases). How do we intend to do base system package versioning? Is it going to be the same version for all packages or do we intend to bump versions in individual Makefiles? Sorry if it has been discussed already, I must have missed it. IMHO >700 hardly makes any sense. I see no point in having individual package for each binary under /bin and .so. I'd rather tweak a couple Makefiles to remove stuff I don't need. WITH_* options in src.conf could be a great list of packages to start with. Having -doc and -include in separate packages would be appreciated as well. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: libc.so dependency on libssp_nonshared.a
On (02/02/2015 17:06), Jeremie Le Hen wrote: Hi Gleb, On Sun, Feb 1, 2015 at 9:24 PM, Gleb Kurtsou g...@freebsd.org wrote: I came across some build issues in libc.so and SSP. libc.ldscript (aka libc.so) unconditionally includes @@LIBDIR@@/libssp_nonshared.a libssp* are not built if WITHOUT_SSP defined. ObsoleteFiles.inc doesn't mention libssp*. Consider WITHOUT_SSP=yes case. As soon as one does clean installworld and/or removes stale libssp_nonshared.a ld fails to link anything because of missing libssp_nonshared.a I think nowadays it would make sense to get it of WITHOUT_SSP altogether. This will turn this into a non-issue. Do you mean building libssp_nonshared unconditionally? It makes perfect sense. The library is a single stub function. By building libssp* conditionally we may expect that -fstack-protector complier option won't work. I wasn't able to reproduce this potential problem. libc provides __stack_chk_fail and __stack_chk_guard. And I failed to create a test case that would generate code using anything like __memcpy_chk. Perhaps we should change MK_SSP to operate as documented (add -fstack-protector to CFLAGS) and consider adding MK_SSP_SUPPORT option for libraries. Although there is no reason to have MK_SSP_SUPPORT if that would imply failure to run binaries or compile with -fstack-protector. libc.so during buildworld (as found under /usr/obj) is symlink to the actual shared library, but not ldscript. Is it intentional? I wouldn't expect make -C /usr/src/bin/cat to match buildworld result closely assuming src and world are in sync, but they seem to have different idea of what libc is.. I don't remember the details except this was pretty twisted :). I don't work enough on this subject to keep it warm in my head and each time I need to go back to it, it takes me hours. The code is here if you want to give a try: http://svnweb.freebsd.org/base/head/share/mk/bsd.lib.mk?view=annotate#l335 indeed. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
libc.so dependency on libssp_nonshared.a
I came across some build issues in libc.so and SSP. libc.ldscript (aka libc.so) unconditionally includes @@LIBDIR@@/libssp_nonshared.a libssp* are not built if WITHOUT_SSP defined. ObsoleteFiles.inc doesn't mention libssp*. Consider WITHOUT_SSP=yes case. As soon as one does clean installworld and/or removes stale libssp_nonshared.a ld fails to link anything because of missing libssp_nonshared.a libc.so during buildworld (as found under /usr/obj) is symlink to the actual shared library, but not ldscript. Is it intentional? I wouldn't expect make -C /usr/src/bin/cat to match buildworld result closely assuming src and world are in sync, but they seem to have different idea of what libc is.. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Committing PEFS to CURRENT
Hello, I would like to ask everybody's opinion regarding committing PEFS to CURRENT. PEFS is a stacked cryptographic file system for FreeBSD. Development started as Google Summer of Code project in 2009. It has been in ports since Sept 2011. I maintain the project. Conceptually PEFS is similar to nullfs adding encryption layer on top of it. But it differs technically by not using vop_bypass. Another popular stacked cryptographic file systems include eCryptfs (linux) and encfs (fuse). There is also pam_pefs pam module to allow user authentication with their PEFS-encrypted home directory password. For those interested in high level introduction I would highly recommend article by Kris Moore in the BSD Magazine Issue 09/2013(50) - http://bsdmag.org/magazine/1848-day-to-day-bsd-administration We are very close to branching 10-STABLE now, but patch is non-intrusive, it only adds new functionality, enabling PEFS for i386 and amd64 (platforms it's known to work on). Patch passes make universe. Patch is available here: https://github.com/glk/freebsd-head/commit/b4d2c4a5f42f88fdd07cb75feba3467e4d4c043c.patch Pros/cons: - Having PEFS in base would be a huge maintenance help for PCBSD/TrueOS who are already committed to use PEFS in next product releases, e.g. PCBSD provides encrypted home directories. - There is steady interest in the project from users (emails, etc). Many of them note that file system is not well known yet. Moving PEFS to base would greatly increase its exposure. - Committing PEFS to base would also simplify maintenance by keeping it in sync with other subsystems, e.g. it will be updated on large scale changes like VM locking. - There are no bugs known at the moment. I've been using it to encrypt home directory since day one. pho@ ran stress test suite on it a while back, number of bugs was fixed. - PEFS is known to work on amd64 and i386 only. Big endian system and systems with page size larger than 4k are not tested. - NOTE! There has been no cryptography review. I'd like to suggest to add warning about file system and crypto used is experimental and hasn't undergone professional review. Similar to one we had in tmpfs. BSD Magazine article: http://bsdmag.org/magazine/1848-day-to-day-bsd-administration Port: http://www.freshports.org/sysutils/pefs-kmod/ Source code repository: https://github.com/glk/pefs FreeBSD DevSummit'2011 - pefs presentation slides: https://pefs.googlecode.com/files/pefs-devsummit.pdf FreeBSD wiki page: https://wiki.freebsd.org/PEFS I would really appreciate any comments or suggestions. Thank you, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Committing PEFS to CURRENT
On Mon, Oct 7, 2013 at 9:53 AM, Nikolai Lifanov lifa...@mail.lifanov.com wrote: On 10/07/13 12:31, Gleb Kurtsou wrote: Hello, I would like to ask everybody's opinion regarding committing PEFS to CURRENT. PEFS is a stacked cryptographic file system for FreeBSD. Development started as Google Summer of Code project in 2009. It has been in ports since Sept 2011. I maintain the project. Conceptually PEFS is similar to nullfs adding encryption layer on top of it. But it differs technically by not using vop_bypass. Another popular stacked cryptographic file systems include eCryptfs (linux) and encfs (fuse). There is also pam_pefs pam module to allow user authentication with their PEFS-encrypted home directory password. For those interested in high level introduction I would highly recommend article by Kris Moore in the BSD Magazine Issue 09/2013(50) - http://bsdmag.org/magazine/1848-day-to-day-bsd-administration We are very close to branching 10-STABLE now, but patch is non-intrusive, it only adds new functionality, enabling PEFS for i386 and amd64 (platforms it's known to work on). Patch passes make universe. Patch is available here: https://github.com/glk/freebsd-head/commit/b4d2c4a5f42f88fdd07cb75feba3467e4d4c043c.patch Pros/cons: - Having PEFS in base would be a huge maintenance help for PCBSD/TrueOS who are already committed to use PEFS in next product releases, e.g. PCBSD provides encrypted home directories. - There is steady interest in the project from users (emails, etc). Many of them note that file system is not well known yet. Moving PEFS to base would greatly increase its exposure. - Committing PEFS to base would also simplify maintenance by keeping it in sync with other subsystems, e.g. it will be updated on large scale changes like VM locking. - There are no bugs known at the moment. I've been using it to encrypt home directory since day one. pho@ ran stress test suite on it a while back, number of bugs was fixed. - PEFS is known to work on amd64 and i386 only. Big endian system and systems with page size larger than 4k are not tested. - NOTE! There has been no cryptography review. I'd like to suggest to add warning about file system and crypto used is experimental and hasn't undergone professional review. Similar to one we had in tmpfs. BSD Magazine article: http://bsdmag.org/magazine/1848-day-to-day-bsd-administration Port: http://www.freshports.org/sysutils/pefs-kmod/ Source code repository: https://github.com/glk/pefs FreeBSD DevSummit'2011 - pefs presentation slides: https://pefs.googlecode.com/files/pefs-devsummit.pdf FreeBSD wiki page: https://wiki.freebsd.org/PEFS I would really appreciate any comments or suggestions. Thank you, Gleb. Just a personal note: I hoped that you would commit pefs to base someday. It works well, and is the type of a core functionality that would be nice to have as early as the install ISO, before skel is copied over for the first user. I would be happy if this happened. Agree. It would also be nice to have standard way to mount pefs file systems because they need to be mounted at later point during boot after other file systems mounted. Small rc.d script should do the trick. I think those issues should be addressed after PEFS is committed - Nikolai Lifanov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Committing PEFS to CURRENT
On Mon, Oct 7, 2013 at 12:58 PM, Julian H. Stacey j...@berklix.com wrote: Hi Gleb All Gleb Kurtsou wrote: Hello, I would like to ask everybody's opinion regarding committing PEFS to CURRENT. PEFS is a stacked cryptographic file system for FreeBSD. Development started as Google Summer of Code project in 2009. It has been in ports since Sept 2011. I maintain the project. Conceptually PEFS is similar to nullfs adding encryption layer on top of it. But it differs technically by not using vop_bypass. Another popular stacked cryptographic file systems include eCryptfs (linux) and encfs (fuse). There is also pam_pefs pam module to allow user authentication with their PEFS-encrypted home directory password. 2 others are also already in FreeBSD src/ (not just ports) gbde geli. geli and gbde are different concept, they provide encrypted block level devices. PEFS transparently encrypts data on existing file system. Here is what you can do with PEFS: % mkdir ~/Private % pefs mount ~/Private ~/Private % pefs addkey ~/Private % echo Hello WORLD ~/Private/test % ls -Al ~/Private total 1 -rw-r--r-- 1 gleb gleb 12 Oct 1 12:55 test % cat ~/Private/test Hello WORLD % pefs unmount ~/Private % ls -Al ~/Private total 1 -rw-r--r-- 1 gleb gleb 12 Oct 1 12:55 .DU6eudxZGtO8Ry_2Z3Sl+tq2hV3O75jq % hd ~/Private/.DU6eudxZGtO8Ry_2Z3Sl+tq2hV3O75jq 7f 1e 1b 05 fc 8a 5c 38 fc d8 2d 5f |..\8..-_| 000c Take a look a great article in the BSD Magazine or http://glebkurtsou.blogspot.com/2009/10/encrypting-private-directory-with-pefs.html Whether moved from ports to src or not, either way, I sggest add to man section SEE ALSO gbde(8) geli(8) Good point, thanks. Also, SEE ALSO of gbde geli should probably ref ports/sysutils/pefs-kmod ft: Command not found. No pefs yet i SEE ALSO of http://www.freebsd.org/cgi/man.cgi?query=gbdeapropos=0sektion=8manpath=FreeBSD+9.2-RELEASEarch=defaultformat=html http://www.freebsd.org/cgi/man.cgi?query=geliapropos=0sektion=8manpath=FreeBSD+9.2-RELEASEarch=defaultformat=html I suggest add an href inside: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/disks-encrypting.html Even if just a 1 liner to start, to expand to a section later. (None there for 'pefs', I just searched) Personaly I've been using gbde based on top of a file inside a UFS for a long time, I can't remember why I chose gbde rather than geli, I guess because it was there first ? A dummy's guide short notes along the lines of Which of these 3 should I use? might also later be nice at the top of that web page :-) There is no answer for the question, each system does it's own thing and does it differently: * With PEFS backups are much easier: - Use regular backup software for backing up encrypted data (lower level file system), that would allow delta backup only. - Create file system snapshots, e.g. zfs, then zfs send/receive, regardless whether file system is encrypted or not. * Setting up multiple encrypted file system is much easier -- no need to preallocate storage and create file system. * With PEFS it's possible to add key to encrypted home directory during login (pam_pefs). * PEFS let's you use multiple key in same file system. Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Committing PEFS to CURRENT
On Mon, Oct 7, 2013 at 4:17 PM, John-Mark Gurney j...@funkthat.com wrote: Gleb Kurtsou wrote this message on Mon, Oct 07, 2013 at 09:31 -0700: Patch is available here: https://github.com/glk/freebsd-head/commit/b4d2c4a5f42f88fdd07cb75feba3467e4d4c043c.patch Is there a reason you are writing your own AES-NI implementation instead of using the OpenCrypto framework? It reuses the same AES-NI implementation used by opencrypto, but code doesn't use opencrypto directly. Main limitation in opencrypto is that is incomplete implementation AES-XTS -- it doesn't implement ciphertext stealing. opencrypto contexts seemed to be too much overhead list time I looked at them especially in the case of multiple keys per file system in PEFS. AES-NI interface is not designed to be used outside of opencrypto thus some minor copy-past. I updated the kernel's AES-NI implementation to have a very fast AES-XTS... Upon looking at your implementation, you have a very slow implementation as you do not pipeline AES-XTS at all... Please switch to using the opencrypto version.. You'll then be able to make use of any accelerators that other platforms may have... I was looking into incorporating AES_XTS pipeline changes in PEFS. I'd like to avoid switching to opencrypto at this point. But pipelining is doable without opencrypto and copying code around. Are there plans to add authentication to this scheme? See that as a todo, but w/o authentication, you can't store anything reliably on it.. And w/ XTS, the attacker can take pot shots at your file in 16 byte chuncks... I have data authentication prototype. It's too far from being complete, and I keep working on it as time permits. There are a few more ideas I'd like to work on while redesigning the file system. Patch also includes hmac and pkcs5v2 implementations which in fact were generic versions to go under sys/crypto until yesterday. Considering we are late in code freeze already I've merged them into PEFS not to touch geli code with the patch. The only reason I'm running zfs on geli w/o authentication is that I'm using a 256bit checksum, so the chances of someone modifing two blocks to fool zfs into decrypting the correct new checksum value for their modified block is very small... In short, I'm trusting zfs to do the authentication for me... -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Committing PEFS to CURRENT
On Mon, Oct 7, 2013 at 5:23 PM, John-Mark Gurney j...@funkthat.com wrote: Gleb Kurtsou wrote this message on Mon, Oct 07, 2013 at 16:47 -0700: On Mon, Oct 7, 2013 at 4:17 PM, John-Mark Gurney j...@funkthat.com wrote: Gleb Kurtsou wrote this message on Mon, Oct 07, 2013 at 09:31 -0700: Patch is available here: https://github.com/glk/freebsd-head/commit/b4d2c4a5f42f88fdd07cb75feba3467e4d4c043c.patch Is there a reason you are writing your own AES-NI implementation instead of using the OpenCrypto framework? It reuses the same AES-NI implementation used by opencrypto, but code doesn't use opencrypto directly. Main limitation in opencrypto is that is incomplete implementation AES-XTS -- it doesn't implement ciphertext stealing. opencrypto contexts seemed to be too much overhead list time I I remember noticing that when I was working on it.. but as there are different modes of packing/padding, I decided not to do anything with that... I also don't like your lack of comments arround xts_lastblock and about why it is accessing dst - 1... To me, you should pass in the previous block as an arg to xts_lastblock instead of doing dst[-1]... You did comment what you're doing (m - 1), but not why it is safe to do that... There is no comment that you're implementing ciphertext stealing w/ the function which makes it even harder to understand that you'd going it properly... The code comes from University of Tsukuba. The function is internal to the module and is safe to use that way if you look at pefs_xts_block_encrypt and pefs_xts_block_decrypt. It wouldn't be hard to add ciphertext stealing to the opencrypto implementation if that is really all that is missing... but.. looked at them especially in the case of multiple keys per file system in PEFS. AES-NI interface is not designed to be used outside of opencrypto thus some minor copy-past. We have discovered that by the minor copy/paste we now have an inferior implementation of AES-XTS... If it performed similar to the one before it, it is over 10x slower than the one that I committed.. I updated the kernel's AES-NI implementation to have a very fast AES-XTS... Upon looking at your implementation, you have a very slow implementation as you do not pipeline AES-XTS at all... Please switch to using the opencrypto version.. You'll then be able to make use of any accelerators that other platforms may have... I was looking into incorporating AES_XTS pipeline changes in PEFS. I'd like to avoid switching to opencrypto at this point. But pipelining is doable without opencrypto and copying code around. I really don't like the idea of adding yet anothe AES-XTS implementation to our tree (especially considering how bad both the previous one and this one is)... Even if you do bring over the pipeline changes... It'll be yet another copy of code to maintain and port performance improvements too... We could always refactor the AES-NI code to make it more usable outside the opencrypto framework as a stepping stone to possibly using pjd's improved opencrypto framework... Refactoring AES-NI would be ideal, it would also be great to extract AES-XTS implementation and make it usable outside of opencrypto adding ciphertext stealing. But copy/pasting just because we don't want to do a bit more work isn't good justification... It's not that I don't want to do a bit more work I never said that, it's rather about avoiding changes after KBI freeze. Are there plans to add authentication to this scheme? See that as a todo, but w/o authentication, you can't store anything reliably on it.. And w/ XTS, the attacker can take pot shots at your file in 16 byte chuncks... I have data authentication prototype. It's too far from being complete, and I keep working on it as time permits. There are a few more ideas I'd like to work on while redesigning the file system. Patch also includes hmac and pkcs5v2 implementations which in fact were generic versions to go under sys/crypto until yesterday. Considering we are late in code freeze already I've merged them into PEFS not to touch geli code with the patch. Can't you keep them named the same under sys/crypto and just link w/ them as necessary to prevent repo churn when you finally integrate them into sys/crypto? That seems better than moving them arround, though I guess w/ svn, it isn't as big of a deal... Someone w/ a repo hat on should chime in here... It won't be possible at least because of pkcs5v2_genkey() (name collision) being defined internally in geli and pkcs5v2 using hmac from geli. Moving them to sys/crypto in 11-CURRENT is a minor issue IMHO. I was pushing those changes to HEAD years ago, but they got stuck somewhere in review process. The only reason I'm running zfs on geli w/o authentication is that I'm using a 256bit checksum, so the chances of someone modifing two blocks to fool zfs into decrypting the correct new checksum
Re: Committing PEFS to CURRENT
On Mon, Oct 7, 2013 at 6:24 PM, John-Mark Gurney j...@funkthat.com wrote: But will the work get done to clean it up after the freeze is over? What happens if it doesn't, will it get removed before 10.1 or will we have to live w/ the code? I still hope not to get hit by bus any time soon.. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Committing PEFS to CURRENT
On (07/10/2013 21:59), Outback Dingo wrote: On Mon, Oct 7, 2013 at 9:42 PM, Gleb Kurtsou gleb.kurt...@gmail.com wrote: On Mon, Oct 7, 2013 at 6:24 PM, John-Mark Gurney j...@funkthat.com wrote: But will the work get done to clean it up after the freeze is over? What happens if it doesn't, will it get removed before 10.1 or will we have to live w/ the code? I still hope not to get hit by bus any time soon.. on a side note, i applied the patch to stable/9 out of curiosity and the kernel failed to build the module however i could install fine from ports. Correct, there is no support for old kernels in the patch. Port will be maintained and will provide support for older versions in case PEFS finds its way to 10.0. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: r248583 Kernel panic: negative refcount 0xfffffe0031b59168
On (30/06/2013 13:18), Mateusz Guzik wrote:
On Sun, Jun 30, 2013 at 05:21:42PM +1000, Kubilay Kocak wrote:
I'm seeing what I believe is related panic, reliably being generated by
the Python regression test suite on a newly created FreeBSD 10-CURRENT
buildbot.
Symptoms first seen in an freebsd.org FTP snapshot dated Thu May 30
20:01:46 UTC 2013 and also reproducible on a freshly updated r252400
It is additionally reproducible after checking out pure upstream python
sources, using the following steps:
hg clone http://hg.python.org/cpython
cd cpython configure make buildbottest
An interesting possible correlation is that it seems to drop out
during/around test_socket
Turns out the bug is quite funny ;)
Patch fixes chrome for me.
Thanks!
Try this:
diff --git a/sys/kern/uipc_usrreq.c b/sys/kern/uipc_usrreq.c
index 5d8e814..7a4db04 100644
--- a/sys/kern/uipc_usrreq.c
+++ b/sys/kern/uipc_usrreq.c
@@ -1764,8 +1764,8 @@ unp_externalize(struct mbuf *control, struct mbuf
**controlp, int flags)
}
for (i = 0; i newfds; i++, fdp++) {
fde = fdesc-fd_ofiles[*fdp];
- fde-fde_file = fdep[0]-fde_file;
- filecaps_move(fdep[0]-fde_caps,
+ fde-fde_file = fdep[i]-fde_file;
+ filecaps_move(fdep[i]-fde_caps,
fde-fde_caps);
if ((flags MSG_CMSG_CLOEXEC) != 0)
fde-fde_flags |= UF_EXCLOSE;
--
Mateusz Guzik mjguzik gmail.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Audio Hints, T520?
On (05/05/2013 19:43), Sean Bruno wrote: On Mon, 2013-05-06 at 03:25 +0200, Michael Schmiedgen wrote: On 04.05.2013 06:14, Sean Bruno wrote: Speaker/headphones working great on Current. Was trying to get the microphone working, but it seems to not quite be working. I got the same problem with my X220. Before the last Lenovo HDA quirk commits everything worked fine, but I had to set the default sound unit to 1. Now I do not need to set the sound unit, audio output works out of the box but recording does not work anymore. I fiddled with nid config but got bored after the fifth reboot. The behaviour changed at my last world build a month ago. I can supply dmesg or sysctl output if you want. Thanks Michael Ok, this smells like a recent regression. Likely so. Works for me on 10.0-CURRENT r247601 with following in loader.conf: # Out : speaker + headphones hint.hdac.0.cad0.nid25.config=as=1 seq=15 # In : mic + external mic hint.hdac.0.cad0.nid35.config=as=2 hint.hdac.0.cad0.nid27.config=as=2 seq=15 Sean ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: r248583 Kernel panic: negative refcount 0xfffffe0031b59168
On (15/04/2013 10:35), Pawel Jakub Dawidek wrote: On Sat, Apr 13, 2013 at 09:43:14PM -0700, Gleb Kurtsou wrote: On (22/03/2013 11:51), Shawn Webb wrote: Hey All, I'm not sure if this is a result of r248583 or a different commit, but I hit a kernel panic when closing Chrome. I've linked to the info and core.txt files below. If you need me to ship you the vmcore file, let me know. It's 1.1GB in size. Other than the pasted files, I'm not too sure where to go from here. If there's any other info you need, please let me know. I'm a newb at submitting this kind of stuff. Paste of info file: http://ix.io/4Qo Paste of core.txt file: http://ix.io/4Qp Shawn, did you find workaround for the problem? I've just upgraded to recent HEAD and see the same panic on closing chrome. Switching back to r247601 just before Merge Capsicum overhaul commit makes panic disappear. I did receive Shawn's report some time ago, I even installed Chromium to try to reproduce it, but it didn't crash for me yet. I could send you chrome binary I'm using. It's a outdated chromium-22.0.1229.92. If there are some easy, but reliable steps to reproduce it, like open this webpage in tab 1, then this webpage in tab 2, then close tab 1 that would be great. This kernel coredump is not really useful, as we this is legitimate case of decrementing reference counter. The problem is that something decremented it earlier when it shouldn't or it wasn't incremented somewhere. DTrace might be useful tool here if we could instrument it to log backtrace of all increments and decrements done by the Chromium processes. I'll try to reproduce it in vm.. which is not that easy because virtualbox kmod needs patching to work on month old CURRENT and there is no binary packages to install vm in the first place. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: r248583 Kernel panic: negative refcount 0xfffffe0031b59168
On (22/03/2013 11:51), Shawn Webb wrote:
Hey All,
I'm not sure if this is a result of r248583 or a different commit, but I
hit a kernel panic when closing Chrome. I've linked to the info and
core.txt files below. If you need me to ship you the vmcore file, let me
know. It's 1.1GB in size.
Other than the pasted files, I'm not too sure where to go from here. If
there's any other info you need, please let me know. I'm a newb at
submitting this kind of stuff.
Paste of info file: http://ix.io/4Qo
Paste of core.txt file: http://ix.io/4Qp
Shawn, did you find workaround for the problem?
I've just upgraded to recent HEAD and see the same panic on closing
chrome. Switching back to r247601 just before Merge Capsicum overhaul
commit makes panic disappear.
~ # kgdb -n 1
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type show copying to see the conditions.
There is absolutely no warranty for GDB. Type show warranty for details.
This GDB was configured as amd64-marcel-freebsd...
Unread portion of the kernel message buffer:
VNASSERT failed
0xfe0196700760: tag none, type VBAD
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VV_NOSYNC|VI_DOOMED)
lock type zfs: UNLOCKED
panic: No vop_advlock(0xfe0196700760, 0xff823adb9908)
cpuid = 3
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff823adb9740
kdb_backtrace() at kdb_backtrace+0x39/frame 0xff823adb97f0
vpanic() at vpanic+0x127/frame 0xff823adb9830
kassert_panic() at kassert_panic+0x136/frame 0xff823adb98a0
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x92/frame 0xff823adb98d0
closef() at closef+0x9a/frame 0xff823adb9960
closefp() at closefp+0xa0/frame 0xff823adb99b0
amd64_syscall() at amd64_syscall+0x1f9/frame 0xff823adb9ab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xff823adb9ab0
--- syscall (6, FreeBSD ELF64, sys_close), rip = 0x80aeaaa8a, rsp =
0x7ebf3f38, rbp = 0x7ebf3f50 ---
[...]
(kgdb) fr 0
#0 doadump (textdump=1) at pcpu.h:231
231 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) up
#1 0x804f5827 in kern_reboot (howto=260) at
/freebsd-src/local/sys/kern/kern_shutdown.c:447
447 doadump(TRUE);
(kgdb)
#2 0x804f5d36 in vpanic (fmt=value optimized out, ap=value
optimized out)
at /freebsd-src/local/sys/kern/kern_shutdown.c:754
754 kern_reboot(bootopt);
(kgdb)
#3 0x804f5bc6 in kassert_panic (fmt=value optimized out)
at /freebsd-src/local/sys/kern/kern_shutdown.c:642
642 vpanic(fmt, ap);
(kgdb)
#4 0x80747aa2 in VOP_ADVLOCK_APV (vop=value optimized out,
a=0xff823adb9908)
at vnode_if.c:2522
2522VNASSERT(vop != NULL, a-a_vp, (No vop_advlock(%p, %p),
a-a_vp, a));
(kgdb)
#5 0x804b8eaa in closef (fp=0xfe014da8ccd0, td=0xfe0014aea920)
at vnode_if.h:1041
1041vnode_if.h: No such file or directory.
in vnode_if.h
(kgdb)
#6 0x804b7030 in closefp (fdp=0xfe001c8c4800, fd=value optimized
out, fp=0xfe014da8ccd0,
td=0xfe0014aea920, holdleaders=value optimized out)
at /freebsd-src/local/sys/kern/kern_descrip.c:1136
1136error = closef(fp, td);
(kgdb) p *fp
$5 = {f_data = 0xfe0196700760, f_ops = 0x80a477b8, f_cred =
0xfe0067907600,
f_vnode = 0xfe0196700760, f_type = 1, f_vnread_flags = 0, f_flag = 3,
f_count = 0, f_seqcount = 0,
f_nextoff = 16388, f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset
= 16388, f_label = 0x0}
(kgdb) p *fp
$6 = {f_data = 0xfe0196700760, f_ops = 0x80a477b8, f_cred =
0xfe0067907600,
f_vnode = 0xfe0196700760, f_type = 1, f_vnread_flags = 0, f_flag = 3,
f_count = 0, f_seqcount = 0,
f_nextoff = 16388, f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset
= 16388, f_label = 0x0}
(kgdb) p fp-f_vnode
$7 = (struct vnode *) 0xfe0196700760
(kgdb) p *fp-f_vnode
$8 = {v_tag = 0x807a3e35 none, v_op = 0x0, v_data = 0x0, v_mount =
0x0, v_nmntvnodes = {
tqe_next = 0xfe014fd95760, tqe_prev = 0xfe011d500958}, v_un =
{vu_mount = 0x0, vu_socket = 0x0,
vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev =
0x0}, v_cache_src = {
lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last =
0xfe01967007b0}, v_cache_dd = 0x0,
v_lock = {lock_object = {lo_name = 0x80dddbb1 zfs, lo_flags =
91881472, lo_data = 0,
lo_witness = 0x0}, lk_lock = 1, lk_exslpfail = 0, lk_timo = 51, lk_pri =
96}, v_interlock = {
lock_object = {lo_name = 0x807bfbb9 vnode interlock, lo_flags =
16908288, lo_data = 0,
lo_witness = 0x0}, mtx_lock = 6}, v_vnlock = 0xfe01967007c8,
v_actfreelist = {
tqe_next = 0xfe0031985b10, tqe_prev =
Re: Tmpfs panic in -current
On (27/06/2012 13:29), Kevin Lo wrote: Kevin Lo wrote: Konstantin Belousov wrote: On Tue, Jun 26, 2012 at 12:38:25PM +0800, Kevin Lo wrote: Konstantin Belousov wrote: On Mon, Jun 25, 2012 at 10:03:28AM +0800, Kevin Lo wrote: I've observed a panic in recent -current several times but I only have a picture of the backtrace: http://people.freebsd.org/~kevlo/panic_tmpfs.jpg Does this look at all familiar to anyone? Can you look up the line corresponding to tmpfs_reg_resize + 0x627 address in your kernel ? Sure. The screenshot looks strange. The instruction on which the kernel trapped is int 0x28 which should not appear in the compiled code. Kevin, do you have fuse modules loaded? It looks much like memmory corruption issue. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: GCC update for testing
On (17/05/2012 10:44), Pedro Giffuni wrote: Hi; I took a bunch of patches that were merged into the GCC 4.1 branch (under GPLv2) and prepared a patch for merging them into our base gcc. These are supposed to be bug fixes only. You can get the patch here: http://people.freebsd.org/~pfg/patches/patch-contrib-gcc And, for those really interested, the log is here: http://people.freebsd.org/~pfg/patches/gcc41-pr-log Could you check if patch fixes this issue: http://marc.info/?l=freebsd-hackersm=132348021530691w=2 I've disabled gcc from base locally since discovering it. Thanks, Gleb. It may be my imagination but the patch seems to be causing more warnings than usual and it breaks the kernel here: $ make cc -c -O2 -Os -pipe -fno-strict-aliasing -march=athlon64 -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I../../.. -I../../../contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-omit-frame-pointer -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror ../../../cam/scsi/scsi_sa.c cc1: warnings being treated as errors ../../../cam/scsi/scsi_sa.c: In function 'samount': ../../../cam/scsi/scsi_sa.c:1887: warning: 'comp_supported' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:1888: warning: 'write_protect' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:1887: warning: 'comp_enabled' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2728: warning: 'current_speed' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2728: note: 'current_speed' was declared here ../../../cam/scsi/scsi_sa.c:2727: warning: 'current_density' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2727: note: 'current_density' was declared here ../../../cam/scsi/scsi_sa.c:2725: warning: 'current_blocksize' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2725: note: 'current_blocksize' was declared here ../../../cam/scsi/scsi_sa.c:2728: warning: 'current_speed' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2728: note: 'current_speed' was declared here ../../../cam/scsi/scsi_sa.c:2725: warning: 'current_blocksize' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2725: note: 'current_blocksize' was declared here ../../../cam/scsi/scsi_sa.c:2728: warning: 'current_speed' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2728: note: 'current_speed' was declared here ../../../cam/scsi/scsi_sa.c:2725: warning: 'current_blocksize' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2725: note: 'current_blocksize' was declared here ../../../cam/scsi/scsi_sa.c:2728: warning: 'current_speed' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2728: note: 'current_speed' was declared here ../../../cam/scsi/scsi_sa.c:2725: warning: 'current_blocksize' may be used uninitialized in this function ../../../cam/scsi/scsi_sa.c:2725: note: 'current_blocksize' was declared here *** Error code 1 ... Other stuff I tested (Apache OpenOffice) builds fine. cheers, Pedro. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Intel GPU driver import
On (13/05/2012 00:39), Konstantin Belousov wrote: With r235375, all required VM support for new Intel GPU driver was committed into HEAD. There are still some things to improve and change, but now the all.14.9.patch does not touch anything outside agp or drm. This allows to start the process of importing the new Intel GPU driver into HEAD. I am writing this as initial head-up and to discuss some questions, for which I do have answers but would prefer to have additional feedback from people doing Xorg work. The patch as-is just replaces the Intel DRI1 bits with DRI2 driver. Patch added most of the KMS infrastructure into DRM core. Also, patch completely changed the locking model used by Intel driver. I made absolutely minimal efforts needed to keep other DRI1 drivers compilable. Despite that, I got several surpising reports that Radeon DRI1 still works. That said, for import I can (first choice) just apply the patch, replacing the Intel driver with new one. Or (second choice) I may create another directory, say sys/dev/drm2, and import _only_ Intel driver together with updated DRM core, there. The positive points to the second approach is that we still have older kernel drivers around. Also, I have more freedom in changing the DRM core, without fearing breakage in the DRI1 land. Since I do not really want to deal with Gen2-3 hardware, and VGA console does not work with new driver (yet), there are definite advantages. On the other hand, driver automatic loading will not work with dev/drm2 approach. New driver have to use different module name to co-exist with dri1 driver, so ddx driver cannot load new driver by old name. As result, users need to manually kldload new driver before starting Xorg. My own preference is to implement second choice and put the driver into dev/drm2. We had somewhat similar situation with new USB stack import. First imported as 'usb2' then renamed to 'usb'. Considering there will be no new DRI1 drivers in tree we could import drm2 into sys/dev/drm and move old implementation under sys/dev/drm_compat1. It will break POLA until automatic driver loading by X fixed. Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Status on X220
On (23/04/2012 08:16), Ganael LAPLANCHE wrote: On Fri, 20 Apr 2012 08:14:17 +0200, Lars Engels wrote Hi everyone, This is the same for my x200, but you can make it work: On 9.0-RELEASE you have to configure through device.hints(5), in CURRENT you can configure it on thy fly. See snd_hda(4) how to do this. FYI, this is what I had to put in /boot/devices.hint to have headset working properly on my x220 (on 10-CURRENT) : # Out : speaker + headphones # hint.hdac.0.cad0.nid31.config=as=1 hint.hdac.0.cad0.nid25.config=as=1 seq=15 # In : mic + external mic hint.hdac.0.cad0.nid35.config=as=2 hint.hdac.0.cad0.nid27.config=as=2 seq=15 Just for the record, it also works for ThinkPad T520. I've always been too lazy to read man page myself ;) Thanks! ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Using TMPFS for /tmp and /var/run?
On (04/04/2012 06:38), David Wolfskill wrote:
On Wed, Apr 04, 2012 at 12:50:35PM +0300, Gleb Kurtsou wrote:
...
tmpfs-32bit-size_max.patch.txt should fix the problem. I don't have i386
installations to test it myself.
Do you run PAE kernel? Could you try filling up /tmp at least to 10g.
...
After updating source to r233868, applying the patch, then updating,
here are the results of my testing so far (not using PAE).
Summary: as before, I believe that the patch didn't hurt anything, but
it also doesn't restrict the usable size of /tmp to the specified size
(from /etc/fstab):
I've checked on i386 and patch worked as expected, but it required
previous patch. I've combined both patches. Could you try it.
Thanks,
Gleb.
Script started on Wed Apr 4 06:23:25 2012
g1-227(10.0-C)[1] _do
uid=0(root) gid=0(wheel) groups=0(wheel),5(operator)
/dev/ada0s2a on / (ufs, local, soft-updates)
devfs on /dev (devfs, local, multilabel)
tmpfs on /tmp (tmpfs, local)
/dev/ada0s2d on /usr (ufs, local, soft-updates)
/dev/ada0s4e on /var (ufs, local, soft-updates)
/dev/ada0s4g on /common (ufs, local, soft-updates)
fdescfs on /dev/fd (fdescfs)
FreeBSD g1-227.catwhisker.org 10.0-CURRENT FreeBSD 10.0-CURRENT #2 233868M:
Wed Apr 4 06:02:25 PDT 2012
r...@g1-227.catwhisker.org:/usr/obj/usr/src/sys/CANARY i386
Wed Apr 4 06:23:27 PDT 2012
Removing old libraries
Please be sure no application still uses those libraries, else you
can not start such an application. Consult UPDATING for more
information regarding how to cope with the removal/revision bump
of a specific library.
Old libraries removed
Wed Apr 4 06:23:28 PDT 2012
g1-227(10.0-C)[2] exit
Script done on Wed Apr 4 06:23:29 2012
Script started on Wed Apr 4 06:23:35 2012
g1-227(10.0-C)[1] df -hi /tmp
FilesystemSizeUsed Avail Capacity iused ifree %iused Mounted on
tmpfs 23G 12k 23G 0% 17 2.1G0% /tmp
g1-227(10.0-C)[2] grep tmpfs /etc/fstab
# tmpfs /tmptmpfs rw,size=2147483648 0
0
tmpfs /tmptmpfs rw,size=8g 0 0
g1-227(10.0-C)[3] ls -lhT /bkp/tmp/test
-rw-r--r-- 1 david wheel 8.0G Mar 25 10:42:49 2012 /bkp/tmp/test
g1-227(10.0-C)[4] dd bs=1m if=!$ of=/tmp/test
dd bs=1m if=/bkp/tmp/test of=/tmp/test
8192+0 records in
8192+0 records out
8589934592 bytes transferred in 186.178099 secs (46138266 bytes/sec)
g1-227(10.0-C)[5] df -hi /tmp
FilesystemSizeUsed Avail Capacity iused ifree %iused Mounted on
tmpfs 23G8.0G 15G35% 18 2.1G0% /tmp
g1-227(10.0-C)[6] dd bs=1m if=/bkp/tmp/test of=/tmp/test1
8192+0 records in
8192+0 records out
8589934592 bytes transferred in 220.254916 secs (3868 bytes/sec)
g1-227(10.0-C)[7] df -hi /tmp
FilesystemSizeUsed Avail Capacity iused ifree %iused Mounted on
tmpfs 23G 16G7.0G70% 19 2.1G0% /tmp
g1-227(10.0-C)[8] exit
Script done on Wed Apr 4 06:33:12 2012
g1-227(10.0-C)[5]
Peace,
david
--
David H. Wolfskillda...@catwhisker.org
Depriving a girl or boy of an opportunity for education is evil.
See http://www.catwhisker.org/~david/publickey.gpg for my public key.
diff --git a/sys/fs/tmpfs/tmpfs.h b/sys/fs/tmpfs/tmpfs.h
index efa7c6d..3fc72ab 100644
--- a/sys/fs/tmpfs/tmpfs.h
+++ b/sys/fs/tmpfs/tmpfs.h
@@ -337,11 +337,10 @@ struct tmpfs_mount {
* system, set during mount time. This variable must never be
* used directly as it may be bigger than the current amount of
* free memory; in the extreme case, it will hold the SIZE_MAX
-* value. Instead, use the TMPFS_PAGES_MAX macro. */
+* value. */
size_t tm_pages_max;
- /* Number of pages in use by the file system. Cannot be bigger
-* than the value returned by TMPFS_PAGES_MAX in any case. */
+ /* Number of pages in use by the file system. */
size_t tm_pages_used;
/* Pointer to the node representing the root directory of this
@@ -486,58 +485,32 @@ int tmpfs_truncate(struct vnode *, off_t);
* Memory management stuff.
*/
-/* Amount of memory pages to reserve for the system (e.g., to not use by
+/*
+ * Amount of memory pages to reserve for the system (e.g., to not use by
* tmpfs).
- * XXX: Should this be tunable through sysctl, for instance? */
-#define TMPFS_PAGES_RESERVED (4 * 1024 * 1024 / PAGE_SIZE)
+ */
+#define TMPFS_PAGES_MINRESERVED(4 * 1024 * 1024 / PAGE_SIZE)
/*
- * Returns information about the number of available memory pages,
- * including physical and virtual ones.
- *
- * Remember to remove TMPFS_PAGES_RESERVED from the returned value to avoid
- * excessive memory usage.
- *
+ * Number of reserved swap pages should not be lower than
+ * swap_pager_almost_full high water mark
Re: Using TMPFS for /tmp and /var/run?
On (29/03/2012 21:49), O. Hartmann wrote:
Am 03/29/12 18:14, schrieb David Wolfskill:
On Thu, Mar 29, 2012 at 04:18:06PM +0200, O. Hartmann wrote:
I was wondering if there are some objections using TMPFS for /tmp and
/var/run.
...
My question is whether there are objections using TMPFS for bot /tmp/
and /var/run/ at this stage on FreeBSD 10.0-CURRENT/amd64?
I have no experience using tmpfs for /var/run, but I have been using it
for /tmp for some time (mostly in i386, though).
While I use it quite successfully on machines with a small number of
folks actively busy -- e.g., my desktop; my laptop; my home machines), I
encountered some issues when I tried to do so on machines that were
intended for significantly heavier use. Specifically:
* Compared to an md-resident /tmp, a tmpfs-resident /tmp has much less
flexibility for specifying the size. Per mdconfig(8), the former
uses:
-s size
Size of the memory disk. Size is the number of 512 byte
sectors
unless suffixed with a b, k, m, g, or t which denotes byte,
kilo-
byte, megabyte, gigabyte and terabyte respectively. Options -a
and -t swap are implied if not specified.
while the latter uses:
sizeSpecifies the total file system size in bytes. If zero (the
default) or a value larger than SIZE_MAX - PAGE_SIZE is given,
the available amount of memory (including main memory and swap
space) will be used.
In this configuration, I would have preferred to have specified
about 10GB for /tmp. I wouldn't mind if it spilled to swap space,
but I certaianly didn't want it using 10GB of RAM -- especially since
the machines only had 6GB RAM.
Nor did I especially want *all* of the swap space used for /tmp. I
would have allocated (say) 20GB for swap. I wouldn't mind if half of
that were used for /tmp -- but a reason I allocate so much swap is
that I've seen what happens when a machine runs out of swap, and it
wasn't pretty.
In any case, effective maximum usable size for tmpfs involves SIZE_MAX
(~4G) PAGE_SIZE (4K, in my case).
size_t is 64-bit on 64-bit archs.
* Even when I went ahead and created a tmpfs for /tmp, I'd get ENOSPC
whenever I tried to allocate anything on it -- until I dropped the
size specification to 2G (2**32). Well, 2GB for /tmp just wasn't at
all likely to be useful for my purposes in this case.
Are you using ZFS alongside tmpfs? It should be fixed in 9-STABLE.
So I continue to use tmpfs for /tmp for machines with fewer folks
logging in, but I'm a bit less enthusiastic about its use unless the
workload and other requirements are fairly carefully considered
beforehand.
Peace,
david
It seems there is only one switch which determines the size of the tmpfs
in question (size) and there is no convenient way to say what amount of
RAM is being used before using the swap space. I'd like to have at least
a knob determining the limit of RAM being used.
There is no way to force tmpfs to use given amount of RAM only. It's VM
subsystem that decides what pages to swap. Although some tweaking for VM
to prefer swapping tmpfs pages prior to process pages would be nice.
You could try the patch attached. It adds support for size option suffixes
(like 1g) and introduces swap limit (part of the older patch, not sure
if it's any use).
Patch is against 10-CURRENT.
Older version: https://github.com/glk/freebsd-head/commit/3bd8f7d
Thanks,
Gleb.
On the other hand - my view of those things is really naiv. I think
having tmpfs isn't even a benefit in terms of security, it should also
offer a speedy access to files kept in memory, doesn't it?
Linux is using TMPFS filesystems a lot for these purposes. How do they
overcome restrictions of the size or not flloding RAM and/or swap?
Regards,
Oliver
diff --git a/sys/fs/tmpfs/tmpfs.h b/sys/fs/tmpfs/tmpfs.h
index efa7c6d..3fc72ab 100644
--- a/sys/fs/tmpfs/tmpfs.h
+++ b/sys/fs/tmpfs/tmpfs.h
@@ -337,11 +337,10 @@ struct tmpfs_mount {
* system, set during mount time. This variable must never be
* used directly as it may be bigger than the current amount of
* free memory; in the extreme case, it will hold the SIZE_MAX
-* value. Instead, use the TMPFS_PAGES_MAX macro. */
+* value. */
size_t tm_pages_max;
- /* Number of pages in use by the file system. Cannot be bigger
-* than the value returned by TMPFS_PAGES_MAX in any case. */
+ /* Number of pages in use by the file system. */
size_t tm_pages_used;
/* Pointer to the node representing the root directory of this
@@ -486,58 +485,32 @@ int tmpfs_truncate(struct vnode *, off_t);
* Memory management stuff.
*/
-/* Amount of memory pages to reserve for
Re: Using TMPFS for /tmp and /var/run?
On (02/04/2012 06:26), David Wolfskill wrote:
On Mon, Apr 02, 2012 at 01:31:19PM +0300, Gleb Kurtsou wrote:
...
You could try the patch attached. It adds support for size option suffixes
(like 1g) and introduces swap limit (part of the older patch, not sure
if it's any use).
Patch is against 10-CURRENT.
Older version: https://github.com/glk/freebsd-head/commit/3bd8f7d
OK; here's a summary of what I found so far, now running:
FreeBSD g1-227.catwhisker.org 10.0-CURRENT FreeBSD 10.0-CURRENT #0 233772M:
Mon Apr 2 05:42:48 PDT 2012
r...@g1-227.catwhisker.org:/usr/obj/usr/src/sys/CANARY i386
* First, the patch applied cleanly (via patch -p1).
* Resulting sources build with no issues.
* Prior specification I had in /etc/fstab:
tmpfs/tmptmpfsrw,size=214748364800
worked same as before the patch; df -h /tmp reported a size of 2.0G.
* Changing the above to read:
tmpfs/tmptmpfsrw,size=2g00
also provided the same result, so the unit-specification code looks
as if it's working as expected.
* I have 20G specified for swap, and 4G RAM (and, as above, I'm running
i386). Changing the above tmpfs line in /etc/fstab to
tmpfs/tmptmpfsrw,size=8g00
(still) yields:
g1-227(10.0-C)[3] df -h /tmp
FilesystemSizeUsed Avail Capacity Mounted on
tmpfs 23G 12k 23G 0%/tmp
g1-227(10.0-C)[4]
tmpfs-32bit-size_max.patch.txt should fix the problem. I don't have i386
installations to test it myself.
Do you run PAE kernel? Could you try filling up /tmp at least to 10g.
(Yes, I'm using a whopping total of 12kB while running X. I know of
*very* few folks who use the window manager I prefer. :-})
I'll try exercising it a bit during the day at work report anything
noteworthy. But so far, I see no evidence of regression, and there is
some measure of usability improvement (IMO). So it's looking
encouraging. :-)
Peace,
david
--
David H. Wolfskillda...@catwhisker.org
Depriving a girl or boy of an opportunity for education is evil.
See http://www.catwhisker.org/~david/publickey.gpg for my public key.
commit 44f68235e23ab4bababeafe07d31e07feabb84ba
Author: Gleb Kurtsou gleb.kurt...@gmail.com
Date: Tue Apr 3 00:02:33 2012 +0300
tmpfs: Support file system sizes up to 4GB*PAGE_SIZE on 32 bit archs
diff --git a/sys/fs/tmpfs/tmpfs_vfsops.c b/sys/fs/tmpfs/tmpfs_vfsops.c
index 29f2ca4..6b3ecc0 100644
--- a/sys/fs/tmpfs/tmpfs_vfsops.c
+++ b/sys/fs/tmpfs/tmpfs_vfsops.c
@@ -233,17 +233,13 @@ tmpfs_mount(struct mount *mp)
* allowed to use, based on the maximum size the user passed in
* the mount structure. A value of zero is treated as if the
* maximum available space was requested. */
- if (size_max PAGE_SIZE || size_max SIZE_MAX - PAGE_SIZE)
+ if (size_max PAGE_SIZE || size_max UINT64_MAX - PAGE_SIZE ||
+ (SIZE_MAX UINT64_MAX size_max / PAGE_SIZE = SIZE_MAX))
pages = SIZE_MAX;
else
pages = howmany(size_max, PAGE_SIZE);
MPASS(pages 0);
- if (pages SIZE_MAX / PAGE_SIZE)
- size_max = pages * PAGE_SIZE;
- else
- size_max = SIZE_MAX;
-
if (nodes_max = 3) {
if (pages INT_MAX / nodes_per_page)
nodes_max = pages * nodes_per_page;
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: /usr/bin/tar creates invalid lib file
On (25/03/2012 10:53), Tim Kientzle wrote:
On Mar 25, 2012, at 5:53 AM, Boris Samorodov wrote:
On 24.03.2012 21:00, Tim Kientzle wrote:
On Mar 23, 2012, at 9:51 PM, Boris Samorodov wrote:
Can you send me the output of:
tar -cvf /tmp/test.tar
/usr/ports/devel/nspr/work/nspr-4.9/mozilla/nsprpub/build/dist/lib/../../pr/src/./libnspr4.so.1
(A tar archive containing only that one source file.)
This looks similar to a bug that we found in libarchive recently
I didn't think that bug impacted FreeBSD, but I may have been
wrong…. if it did, it will be obvious from the structure of the
created archive.
The following file is extracted after tarring:
-
% hd libnspr4.so.1
32 0a 30 0a 30 0a 32 34 31 39 37 31 0a 30 0a 00
|2.0.0.241971.0..|
0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
||
*
0200
-
The tar file itself attached (3KB in length).
Ugh. I'll probably need your help to diagnose this more precisely.
Here is the root problem: tar thinks this is a sparse file
with nothing in it.On FreeBSD, bsdtar now uses
lseek(SEEK_HOLE) to identify holes in the file. For
some reason, bsdtar is storing this file as just one big hole.
I experience a related issue. lseek(SEEK_HOLE) error checks are too
strict. Files are not added to archive if lseek(SEEK_HOLE) fails.
Ignoring lseek(SEEK_HOLE) at least in ENOTTY case would be preferable.
I run stacked file system (PEFS) on top of ZFS. ZFS reports
_PC_MIN_HOLE_SIZE, but ioctls to lower level fs disabled are in PEFS.
I work around it by properly handling _PC_MIN_HOLE_SIZE.
Test case:
% for i in a b c; do echo $i $i; done
% tar cf 1.tar a b c
tar: lseek(SEEK_HOLE) failed: Inappropriate ioctl for device
tar: lseek(SEEK_HOLE) failed: Inappropriate ioctl for device
tar: lseek(SEEK_HOLE) failed: Inappropriate ioctl for device
% ls -l 1.tar
-rw-r--r-- 1 gleb gleb 1024 Mar 26 00:28 1.tar
% tar tf 1.tar # -- no files in archive
%
truss log:
__acl_get_link(0x801c8c100,0x2,0x801d82000,0x0,0x1000,0x0) ERR#22 'Invalid
argument'
extattr_list_link(0x801c8c100,0x1,0x0,0x0,0x7fffc530,0x0) ERR#45 'Operation
not supported'
pathconf(a,0x15) = 512 (0x200)
open(a,O_NONBLOCK,03751270)= 5 (0x5)
lseek(5,0x0,0x3) ERR#25 'Inappropriate ioctl
for device'
close(5) = 0 (0x0)
tar: write(2,tar: ,5) = 5 (0x5)
Thanks,
Gleb.
There are a lot of things here that don't make sense:
* The extracted file should be all zero bytes. (The 2.0.0.241971.0. is the
sparse file map, it's not really part of the file.) How are you extracting
this?
* Can you run the tar command under truss or ktrace and look for calls to
lseek()? That would help verify that this is really a tar bug and not a
filesystem or kernel bug.
I'll spend some time today to see if I can reproduce the problem here.
Tim
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Idea for GEOM and policy based file encryption
On (21/03/2012 11:13), Vincent Hoffman wrote: On 21/03/2012 10:47, Andrey V. Elsukov wrote: On 21.03.2012 14:09, Victor Balada Diaz wrote: You would need to modify UFS, or maybe do something like CFS[1]. CFS works as an NFS server and you could modify it to only cipher the needed files. Also you could write a simple FS on FUSE, but last time i checked, our FUSE support had some problems. Yet another link: http://www.arg0.net/encfs or pefs FreeBSD wiki page: http://wiki.freebsd.org/PEFS blog: http://glebkurtsou.blogspot.com/search/label/pefs It's also in ports: http://www.freshports.org/sysutils/pefs-kmod/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Processes getting stuck in state tmpfs
On (08/03/2012 01:20), Bernhard Froehlich wrote: On 01.03.2012 23:25, Florian Smeets wrote: On 01.03.12 20:31, Gleb Kurtsou wrote: Could you test the patch attached. It's also available here as seperate commits: https://github.com/glk/freebsd-head/commits/tmpfs-rename The test that used to hang within a minute has now been running successfully for almost 2 hours. Looks good to me. I have also re-enabled tmpfs on redports.org main tinderbox with this patch applied and it runs smooth so far! Very good work. Thanks a lot! Thanks, Patch is under review now. And will be committed afterwards. FreeBSD tb3.bluelife.at 10.0-CURRENT FreeBSD 10.0-CURRENT #3: Wed Mar 7 23:47:56 CET 2012 r...@tb3.bluelife.at:/usr/obj/usr/src/sys/GENERIC amd64 -- Bernhard Froehlich http://www.bluelife.at/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Processes getting stuck in state tmpfs
Could you test the patch attached.
It's also available here as seperate commits:
https://github.com/glk/freebsd-head/commits/tmpfs-rename
Thanks,
Gleb.
On (23/02/2012 21:20), Florian Smeets wrote:
On 11.02.12 11:20, Gleb Kurtsou wrote:
On (10/02/2012 22:41), Florian Smeets wrote:
Hi,
if you set WRKDIRPREFIX to a tmpfs mountpoint and try to build audio/gsm
from ports one of the mv processes gets stuck in state tmpfs quite
often. Traces from a kernel with WITTNESS and DEBUG_VFS_LOCKS are
available here http://tb.smeets.im/~flo/tmpfs.txt
It's because of incorrect vnode locking order in tmpfs_rename. Issue is
known and tmpfs is not the only file system suffering from it (e.g. ext2).
There two ways of working around it in tree:
* UFS: try locking vnode, unlock all vnodes on failure, restart,
relookup vnodes needed.
* ZFS: introduce directory entry locks to guarantee fvp won't disappear,
fdvp can be safely traversed, etc. That won't be easy..
UFS-way would be a good temporal solution, but I think we should work on
improving VOP_RENAME() in a long run.
I'll try to prepare a patch in several days.
Hey Gleb,
did you get anywhere with this?
Thanks,
Florian
diff --git a/sys/fs/tmpfs/tmpfs_subr.c b/sys/fs/tmpfs/tmpfs_subr.c
index d2b2245..fe596aa 100644
--- a/sys/fs/tmpfs/tmpfs_subr.c
+++ b/sys/fs/tmpfs/tmpfs_subr.c
@@ -42,6 +42,7 @@ __FBSDID($FreeBSD$);
#include sys/proc.h
#include sys/stat.h
#include sys/systm.h
+#include sys/sysctl.h
#include sys/vnode.h
#include sys/vmmeter.h
@@ -56,6 +57,8 @@ __FBSDID($FreeBSD$);
#include fs/tmpfs/tmpfs_fifoops.h
#include fs/tmpfs/tmpfs_vnops.h
+SYSCTL_NODE(_vfs, OID_AUTO, tmpfs, CTLFLAG_RW, 0, tmpfs file system);
+
/* - */
/*
@@ -320,9 +323,11 @@ loop:
MPASS((node-tn_vpstate TMPFS_VNODE_DOOMED) == 0);
VI_LOCK(vp);
TMPFS_NODE_UNLOCK(node);
- vholdl(vp);
- (void) vget(vp, lkflag | LK_INTERLOCK | LK_RETRY, curthread);
- vdrop(vp);
+ error = vget(vp, lkflag | LK_INTERLOCK, curthread);
+ if (error != 0) {
+ vp = NULL;
+ goto out;
+ }
/*
* Make sure the vnode is still there after
@@ -420,11 +425,13 @@ unlock:
out:
*vpp = vp;
- MPASS(IFF(error == 0, *vpp != NULL VOP_ISLOCKED(*vpp)));
#ifdef INVARIANTS
- TMPFS_NODE_LOCK(node);
- MPASS(*vpp == node-tn_vnode);
- TMPFS_NODE_UNLOCK(node);
+ if (error == 0) {
+ MPASS(*vpp != NULL VOP_ISLOCKED(*vpp));
+ TMPFS_NODE_LOCK(node);
+ MPASS(*vpp == node-tn_vnode);
+ TMPFS_NODE_UNLOCK(node);
+ }
#endif
return error;
diff --git a/sys/fs/tmpfs/tmpfs_vnops.c b/sys/fs/tmpfs/tmpfs_vnops.c
index 93fea8b..dd54679 100644
--- a/sys/fs/tmpfs/tmpfs_vnops.c
+++ b/sys/fs/tmpfs/tmpfs_vnops.c
@@ -46,6 +46,7 @@ __FBSDID($FreeBSD$);
#include sys/sf_buf.h
#include sys/stat.h
#include sys/systm.h
+#include sys/sysctl.h
#include sys/unistd.h
#include sys/vnode.h
@@ -58,6 +59,13 @@ __FBSDID($FreeBSD$);
#include fs/tmpfs/tmpfs_vnops.h
#include fs/tmpfs/tmpfs.h
+SYSCTL_DECL(_vfs_tmpfs);
+
+static volatile int tmpfs_rename_restarts;
+SYSCTL_INT(_vfs_tmpfs, OID_AUTO, rename_restarts, CTLFLAG_RD,
+__DEVOLATILE(int *, tmpfs_rename_restarts), 0,
+Times rename had to restart due to lock contention);
+
/* - */
static int
@@ -920,6 +928,118 @@ out:
/* - */
static int
+tmpfs_rename_relock(struct vnode *fdvp, struct vnode **fvpp,
+struct vnode *tdvp, struct vnode **tvpp,
+struct componentname *fcnp, struct componentname *tcnp)
+{
+ struct vnode *nvp;
+ struct mount *mp;
+ struct tmpfs_dirent *de;
+ int error;
+
+ VOP_UNLOCK(tdvp, 0);
+ if (*tvpp != NULL *tvpp != tdvp)
+ VOP_UNLOCK(*tvpp, 0);
+ mp = fdvp-v_mount;
+
+relock:
+ atomic_add_int(tmpfs_rename_restarts, 1);
+ error = vn_lock(fdvp, LK_EXCLUSIVE);
+ if (error)
+ goto releout;
+ if (vn_lock(tdvp, LK_EXCLUSIVE | LK_NOWAIT) != 0) {
+ VOP_UNLOCK(fdvp, 0);
+ error = vn_lock(tdvp, LK_EXCLUSIVE);
+ if (error)
+ goto releout;
+ VOP_UNLOCK(tdvp, 0);
+ goto relock;
+ }
+ /*
+* Re-resolve fvp to be certain it still exists and fetch the
+* correct vnode.
+*/
+ de = tmpfs_dir_lookup(VP_TO_TMPFS_DIR(fdvp), NULL, fcnp);
+ if (de == NULL) {
+ VOP_UNLOCK(fdvp, 0);
+ VOP_UNLOCK(tdvp, 0);
+ if ((fcnp-cn_flags ISDOTDOT) != 0
Re: how to cross-build a single application ?
On (01/03/2012 18:52), Luigi Rizzo wrote: On Thu, Mar 01, 2012 at 12:24:13PM -0500, Alexander Kabaev wrote: On Thu, 1 Mar 2012 18:38:19 +0100 Luigi Rizzo ri...@iet.unipi.it wrote: What is the way to properly cross-build a single program (after having gone throught the 'toolchain' and possibly even a full 'buildworld') from the top-level directory of a FreeBSD source tree ? right now i do something like cd $SOURCE_ROOT make MAKEOBJDIRPREFIX=/my_obj_tree TARGET=amd64 buildworld but i seem to remember that there is a more efficient way when you want to rebuild only one program or one subtree. I think i have seen this question being answered from time to time but i can't find how now. any hints ? cheers luigi Something like this should work: cd src/ make buildenv TARGET=... cd usr.bin/blah make close: eval `make buildenvvars TARGET=... ` cd usr.bin/blah make There is tools/tinder.sh in src tree. thanks! luigi ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: devd based AUTOMOUNTER
On (18/02/2012 10:48), vermaden wrote:
Added a check if ntfs-3g is available, if not then mount_ntfs is used instead.
Added deleting of empty directories at ${MNTPREFIX}.
Added ${MNTPREFIX} to be set to /mnt or /media according to preference
#! /bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
MNTPREFIX=/media
LOG=/var/log/automount.log
STATE=/var/run/automount.state
DATEFMT=%Y-%m-%d %H:%M:%S
__create_mount_point() { # /* 1=DEV */
MNT=${MNTPREFIX}/$( basename ${1} )
mkdir -p ${MNT}
}
__state_lock() {
while [ -f ${STATE}.lock ]; do sleep 0.5; done
: ${STATE}.lock
}
Why not keep it stateless, unmounting by hand would kill integrity.
Usage of `file` is a big security concern.
I think geom config and list of mounted file systems should be
sufficient for collecting information at runtime. Although it may not be
that easy for disks without partitioning.
I'm using a python script for mounting/unmounting removable disks,
but having similar tool written in sh would be great.
https://github.com/glk/mmount
Thanks,
Gleb.
__state_unlock() {
rm ${STATE}.lock
}
__state_add() { # /* 1=DEV 2=PROVIDER 3=MNT */
__state_lock
grep -E ${3} ${STATE} 1 /dev/null 2 /dev/null {
__log ${1}:duplicated '${STATE}'
return 1
}
echo ${1} ${2} ${3} ${STATE}
__state_unlock
}
__state_remove() { # /* 1=MNT 2=STATE 3=LINE */
BSMNT=$( echo ${1} | sed 's/\//\\\//g' )
sed -i '' /${BSMNT}\$/d ${2}
}
__log() { # /* @=MESSAGE */
echo $( date +${DATEFMT} ) ${@} ${LOG}
}
case ${2} in
(attach)
for I in /dev/${1}*
do
case $( file -L -s ${I} | sed -E 's/label:\ \.*\//g' ) in
(*NTFS*)
dd ${I} count=1 2 /dev/null \
| strings \
| head -1 \
| grep -q NTFS {
__create_mount_point ${I}
which ntfs-3g 1 /dev/null 2 /dev/null {
ntfs-3g ${I} ${MNT} # /* sysutils/fusefs-ntfs */
} || {
mount_ntfs ${I} ${MNT}
}
__log ${I}:mount (ntfs)
}
;;
(*FAT*)
dd ${I} count=1 2 /dev/null \
| strings \
| grep -q FAT32 {
__create_mount_point ${I}
fsck_msdosfs -y ${I}
mount_msdosfs -o large -l -L pl_PL.ISO8859-2 -D cp852 ${I}
${MNT}
__log ${I}:mount (fat)
}
;;
(*ext2*)
__create_mount_point ${I}
fsck.ext2 -y ${I}
mount -t ext2fs ${I} ${MNT}
__log ${I}:mount (ext2)
;;
(*ext3*)
__create_mount_point ${I}
fsck.ext3 -y ${I}
mount -t ext2fs ${I} ${MNT}
__log ${I}:mount (ext3)
;;
(*ext4*)
__create_mount_point ${I}
fsck.ext4 -y ${I}
ext4fuse ${I} ${MNT} # /* sysutils/fusefs-ext4fuse */
__log ${I}:mount (ext4)
;;
(*Unix\ Fast\ File*)
__create_mount_point ${I}
fsck_ufs -y ${I}
mount ${I} ${MNT}
__log ${I}:mount (ufs)
;;
(*)
case $( dd ${I} count=1 2 /dev/null | strings | head -1 ) in
(EXFAT)
__create_mount_point ${I}
mount.exfat ${I} ${MNT} # /* sysutils/fusefs-exfat */
__log ${I}:mount (ufs)
;;
(*) continue ;;
esac
;;
esac
__state_add ${I} $( mount | grep -m 1 ${MNT} | awk '{printf $1}' ) \
${MNT} || continue
done
;;
(detach)
MOUNT=$( mount )
__state_lock
grep ${1} ${STATE} \
| while read DEV PROVIDER MNT
do
TARGET=$( echo ${MOUNT} | grep -E ^${PROVIDER} | awk '{print
$3}' )
[ -z ${TARGET} ] {
__state_remove ${MNT} ${STATE} ${LINE}
continue
}
umount -f ${TARGET}
unset TARGET
__state_remove ${MNT} ${STATE} ${LINE}
__log ${DEV}:umount
done
__state_unlock
__log /dev/${1}:detach
find ${MNTPREFIX} -type d -empty -delete
;;
esac
Not sure if you've looked at disktype in sysutils
but it may be useful to you.
I will get look into that, thanks ;)
Neat scripts!
Matt
Thanks mate.
Regards,
vermaden
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Processes getting stuck in state tmpfs
On (10/02/2012 22:41), Florian Smeets wrote: Hi, if you set WRKDIRPREFIX to a tmpfs mountpoint and try to build audio/gsm from ports one of the mv processes gets stuck in state tmpfs quite often. Traces from a kernel with WITTNESS and DEBUG_VFS_LOCKS are available here http://tb.smeets.im/~flo/tmpfs.txt It's because of incorrect vnode locking order in tmpfs_rename. Issue is known and tmpfs is not the only file system suffering from it (e.g. ext2). There two ways of working around it in tree: * UFS: try locking vnode, unlock all vnodes on failure, restart, relookup vnodes needed. * ZFS: introduce directory entry locks to guarantee fvp won't disappear, fdvp can be safely traversed, etc. That won't be easy.. UFS-way would be a good temporal solution, but I think we should work on improving VOP_RENAME() in a long run. I'll try to prepare a patch in several days. Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [patch] pam_exec: use program exit code instead of PAM_SYSTEM_ERR
On (24/01/2012 15:25), Jean-S?bastien P?dron wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
If the specified program exits with non-zero, current implementation
of pam_exec(8) logs this code and return PAM_SYSTEM_ERR. Therefore,
applications have no idea what went wrong with authentication.
Attached is a patch that changes the behaviour to always return the
program exit code as-is. This lets the program returns meaningful
informations to applications.
I also added a small paragraph explaining this to the man page.
I'm planning to commit this to -CURRENT (maybe in a week or two) and
merge it to 9 and 8 if there're no objections.
Please consider making it optional. It will break for generic
applications because pam_sm_chauthtok error codes are documented and
standardized. I'm not aware of any application that uses PAM error
constants as exit code.
Thanks,
Gleb.
Thanks for any comments!
- --
Jean-Sébastien Pédron
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.18 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8ev0IACgkQa+xGJsFYOlNjyACfSg6NgDLy+7VF6rEVV6yTINTd
rlgAoNpgWLvYBEL2DCejuPDz0yQRf5QY
=JEte
-END PGP SIGNATURE-
diff --git a/lib/libpam/modules/pam_exec/pam_exec.8
b/lib/libpam/modules/pam_exec/pam_exec.8
index 311d64c..c5d2404 100644
--- a/lib/libpam/modules/pam_exec/pam_exec.8
+++ b/lib/libpam/modules/pam_exec/pam_exec.8
@@ -32,7 +32,7 @@
.\
.\ $FreeBSD$
.\
-.Dd February 1, 2005
+.Dd January 24, 2012
.Dt PAM_EXEC 8
.Os
.Sh NAME
@@ -59,6 +59,12 @@ variables:
.Ev PAM_TTY ,
and
.Ev PAM_USER .
+.Pp
+The program exit code should be one of the codes defined in
+.Pa /usr/include/security/pam_constants.h
+under section XSSO 5.2. Authentication is successful if the return code is
+.Er PAM_SUCCESS
+(0), failed otherwise.
.Sh SEE ALSO
.Xr pam_get_item 3 ,
.Xr pam.conf 5 ,
diff --git a/lib/libpam/modules/pam_exec/pam_exec.c
b/lib/libpam/modules/pam_exec/pam_exec.c
index b7a870f..d497479 100644
--- a/lib/libpam/modules/pam_exec/pam_exec.c
+++ b/lib/libpam/modules/pam_exec/pam_exec.c
@@ -141,12 +141,7 @@ _pam_exec(pam_handle_t *pamh __unused, int flags
__unused,
openpam_log(PAM_LOG_ERROR, unknown status 0x%x, status);
return (PAM_SYSTEM_ERR);
}
- if (WEXITSTATUS(status) != 0) {
- openpam_log(PAM_LOG_ERROR, %s returned code %d,
- argv[0], WEXITSTATUS(status));
- return (PAM_SYSTEM_ERR);
- }
- return (PAM_SUCCESS);
+ return (WEXITSTATUS(status));
}
PAM_EXTERN int
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Freeze with 10.0 and VirtualBox {4.1.4|4.1.6|4.1.51r38464}
On (26/11/2011 14:44), Andriy Gapon wrote: on 26/11/2011 14:19 Gustau Pérez said the following: Starting Virtualbox in the console in headless mode allows to see what happens and get a dump of the panic. The messages I got were not the cause problem. The panic I was able to get shows this: http://pastebin.com/dHnB3Xh0 I can't get any further with core although I compiled virtualbox-ose-kmod with debug symbols (I used make config to enable them, because I think -DWITH_DEBUG does not work because kmk is used in the build process). Any help will be appreciated. vm_phys_alloc_contig implementation has been recently changed and now it seems to require that vm_page_queue_free_mtx is held. Using new vm_page_alloc_contig() may be a better option here. Can't help with patch, stuck with pre Nov 15 CURRENT myself. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SIOCGIFADDR broken on 9.0-RC1?
On (15/11/2011 18:10), GR wrote:
Hello list,
more insights since my last post. Here is a small code to trigger the bug
(end of email).
When you run it on 9.0-RC1, it gets an alias address instead of the main inet
address:
% ./get-ip re0
inet: 192.168.2.10
# Main address being 192.168.1.148
On 8.2-RELEASE, all goes well:
% ./get-ip re0
inet: PUBLIC_IP4
Is something broken, or a behaviour has changed since 8.2-RELEASE?
Your test case looks ok and works as expexted for me on 10-CURRENT, both
without aliases and after adding alias to interface. Perhaps it's the way
you add aliases (libdnet ?).
I've used:
ifconfing em0 alias OTHERIP
Thanks,
Gleb.
Best regards,
--8--
#include sys/types.h
#include sys/socket.h
#include stdio.h
#include stdlib.h
#include unistd.h
#include net/if.h
#include sys/sockio.h
#include string.h
#include netinet/in.h
#include arpa/inet.h
int
main(int argc, char *argv[])
{
int fd;
struct ifreq ifr;
const struct sockaddr_in *sa;
fd = socket(AF_INET, SOCK_DGRAM, 0);
if (fd 0) {
perror(socket);
exit(-1);
}
memset(ifr, 0, sizeof(struct ifreq));
strlcpy(ifr.ifr_name, argv[1], sizeof(ifr.ifr_name));
if (ioctl(fd, SIOCGIFADDR, ifr) == 0) {
sa = (const struct sockaddr_in *) ifr.ifr_addr;
printf(inet: %s\n, inet_ntoa(sa-sin_addr));
}
else {
perror(ioctl);
exit(-1);
}
exit(0);
}
--8--
--
^ ___ ___ http://www.GomoR.org/ -+
| / __ |__/Senior Security Engineer |
| \__/ | \ ---[ zsh$ alias psed='perl -pe ' ]---|
+-- Net::Frame = http://search.cpan.org/~gomor/ ---+
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HEADS UP: ports/ and 10.0-CURRENT
On (26/09/2011 23:03), Ade Lovett wrote: With the advent of the conversion of HEAD to 10.0-CURRENT and, as to be expected, ports/ is going to be essentially unusable for a while. The issue stems from configure scripts (to choose something completely at random) assuming that FreeBSD would never jump to a double-digit major version number, and as such, various regexps for freebsd1* (ie: FreeBSD 1.1.x) are now matching freebsd10. It's more exciting than that. FreeBSD = 10 is already seized by Apple :) http://www.google.com/codesearch#search/q=__FreeBSD__%5CW%2B10type=cs This is going to be some fairly fundamental breakage. However, until such time as 9.0-RELEASE is completely out of the door, with autotools hat on, I will _not_ be committing any changes to infrastructural ports to fix this. That is to say, until 9.0-R happens, and for some considerable period afterwards, ya'll can pretty much expect ports/ to be non-functional on HEAD. PRs mentioning this will be gleefully closed referencing this message. -aDe Reply-To set to me. Please honor it. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: fsid change of ZFS?
On (24/08/2011 21:34), Hiroki Sato wrote: Kostik Belousov kostik...@gmail.com wrote in 20110824082119.gj17...@deviant.kiev.zoral.com.ua: ko On Tue, Aug 23, 2011 at 11:23:03PM +0200, Pawel Jakub Dawidek wrote: ko On Tue, Aug 23, 2011 at 04:11:20PM -0400, Rick Macklem wrote: ko Here's the patch. (Hiroki could you please test this, thanks, rick.) ko ps: If the white space gets trashed, the same patch is at: ko http://people.freebsd.org/~rmacklem/fsid.patch ko ko The patch is fine by me. Thanks, Rick! ko ko Sorry, I am late. ko ko It seems that the probability of the collisions for the hash is quite high. ko Due to the fixup procedure, the resulting typenum will depend on the order ko of the module initialization, isn't it ? IMO, it makes the patch goal not ko met. I tried the following two experiments (the complete results are attached) to confirm the probability: 1. [fsidhash1.txt] well-known vfc_name and the names [a-z]fs (# of names is 36) with no fix-up recalculation. 2. [fsidhash2.txt] well-known vfc_name and the names [a-z][a-z]fs (# of names is 710) with no fix-up recalculation. There is no collision in the case 1. And when [a-z][a-z]fs are included the average number of the collided names in the same hash value is 4.43 (i.e. 160 different hash values are generated, the theoretical best number is (710 entries / 256 buckets) = 2.77). Could you run the same test with fnv_32_buf()? Collision rate is likely to be lower. At least, vfc_names we currently have in our kernel code have no collision, fortunately. As you noticed [a-z][a-z]fs is an impractical data set and these results cannot explain the characteristics for all possible and practical vfc_names, so whether this hash is reasonable or not depends on how we think of them. Comments or other better idea? -- Hiroki ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: chromium port causing massive I/O faults
On (27/07/2011 00:48), Alexander Best wrote: On Mon Jul 25 11, Matthias Andree wrote: Am 25.07.2011 09:21, schrieb Alexander Best: On Mon Jul 25 11, Adrian Chadd wrote: Is it perhaps doing disk IO using mmap? how can i check, whether that's the case or not? Use truss(1) for instance. However, unless there are *practical* problems, a high number of page faults is not an indication for problems. Although it may sound scary, page faults are a feature of the memory management. unfortunately truss(1) is crashing chromium :( i opened up a new thread reagarding this issue on freebsd-current@. Could you try ktrace? It works for me another thing i noticed is the increase in system calls caused by chromium. let's have a look at hub.freebsd.org: uptime = 149 days and 'vmstat -s' reports: 2168697753 cpu context switches 2266220366 device interrupts 2902880931 software interrupts 3779075897 traps 902107847 system calls now on my box: uptime = 2 days and 'vmstat -s' reports: 1155995386 cpu context switches 164577882 device interrupts 189456976 software interrupts 137007580 traps 2178434582 system calls About 2.5k syscalls with chrome + a lot of other stuff running. 1.5k without chrome. Looks like there is a lot of clock_gettime and gettimeofday syscalls. ~ % kdump -m 1 -f ktrace.out | grep 'CALL .*gettime' | wc -l 24343 ~ % kdump -E -m 1 -f ktrace.out | grep 'CALL .*gettime' | tail -20 12747 chrome 15.077376 CALL gettimeofday(0x7f7f9630,0x7f7f9640) 12747 chrome 15.077396 CALL clock_gettime(0x4,0x7fbfb6f0) 12747 chrome 15.077497 CALL gettimeofday(0x7fbfb650,0x7fbfb660) 12747 chrome 15.077609 CALL gettimeofday(0x7fbfb650,0x7fbfb660) 12747 chrome 15.077723 CALL gettimeofday(0x7fbfb650,0) 12747 chrome 15.077845 CALL clock_gettime(0,0x7fbfb2b0) 12747 chrome 15.078337 CALL clock_gettime(0x4,0x7f9fa630) 12747 chrome 15.078544 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.078587 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.078632 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.078674 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.082803 CALL gettimeofday(0x7edd3630,0x7edd3640) 12747 chrome 15.084644 CALL gettimeofday(0x7fbfb650,0x7fbfb660) 12747 chrome 15.084746 CALL clock_gettime(0x4,0x7fbfb670) 12747 chrome 15.084815 CALL clock_gettime(0x4,0x7fbfb670) 12747 chrome 15.086620 CALL gettimeofday(0x7efd4650,0x7efd4660) 12747 chrome 15.086736 CALL clock_gettime(0x4,0x7efd4670) 12747 chrome 15.086815 CALL clock_gettime(0x4,0x7efd4670) 12747 chrome 15.098315 CALL gettimeofday(0x7fffafe0,0x7fffaff0) 12747 chrome 15.098680 CALL clock_gettime(0x4,0x7fffb250) Some work was done by kib@ to create a kernel page strong current time and other misc info to eliminate gettimeofday kind syscalls. Bits of it were commited but I'm not sure if it was finished. But anyway calling gettimeofday hundreds of times per second is a chrome bug. FreeBSD 9.0-CURRENT #2 r224003+777e962: Thu Jul 14 13:04:55 EEST 2011 chromium-11.0.696.57_1 i ran the following command twice. first time without running chromium and the second time with chromium running: otaku% vmstat -s|grep system calls; sleep 1; vmstat -s|grep system calls 2178187850 system calls 2178189739 system calls otaku% vmstat -s|grep system calls; sleep 1; vmstat -s|grep system calls 2177998835 system calls 2178022003 system calls so it's 2k/sec vs. 23k/sec cheers. alex ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: chromium port causing massive I/O faults
On (27/07/2011 09:18), Alexander Best wrote: On Wed Jul 27 11, Gleb Kurtsou wrote: On (27/07/2011 00:48), Alexander Best wrote: On Mon Jul 25 11, Matthias Andree wrote: Am 25.07.2011 09:21, schrieb Alexander Best: On Mon Jul 25 11, Adrian Chadd wrote: Is it perhaps doing disk IO using mmap? how can i check, whether that's the case or not? Use truss(1) for instance. However, unless there are *practical* problems, a high number of page faults is not an indication for problems. Although it may sound scary, page faults are a feature of the memory management. unfortunately truss(1) is crashing chromium :( i opened up a new thread reagarding this issue on freebsd-current@. Could you try ktrace? It works for me another thing i noticed is the increase in system calls caused by chromium. let's have a look at hub.freebsd.org: uptime = 149 days and 'vmstat -s' reports: 2168697753 cpu context switches 2266220366 device interrupts 2902880931 software interrupts 3779075897 traps 902107847 system calls now on my box: uptime = 2 days and 'vmstat -s' reports: 1155995386 cpu context switches 164577882 device interrupts 189456976 software interrupts 137007580 traps 2178434582 system calls About 2.5k syscalls with chrome + a lot of other stuff running. 1.5k without chrome. Looks like there is a lot of clock_gettime and gettimeofday syscalls. ~ % kdump -m 1 -f ktrace.out | grep 'CALL .*gettime' | wc -l 24343 ~ % kdump -E -m 1 -f ktrace.out | grep 'CALL .*gettime' | tail -20 12747 chrome 15.077376 CALL gettimeofday(0x7f7f9630,0x7f7f9640) 12747 chrome 15.077396 CALL clock_gettime(0x4,0x7fbfb6f0) 12747 chrome 15.077497 CALL gettimeofday(0x7fbfb650,0x7fbfb660) 12747 chrome 15.077609 CALL gettimeofday(0x7fbfb650,0x7fbfb660) 12747 chrome 15.077723 CALL gettimeofday(0x7fbfb650,0) 12747 chrome 15.077845 CALL clock_gettime(0,0x7fbfb2b0) 12747 chrome 15.078337 CALL clock_gettime(0x4,0x7f9fa630) 12747 chrome 15.078544 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.078587 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.078632 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.078674 CALL clock_gettime(0x4,0x7f9fa650) 12747 chrome 15.082803 CALL gettimeofday(0x7edd3630,0x7edd3640) 12747 chrome 15.084644 CALL gettimeofday(0x7fbfb650,0x7fbfb660) 12747 chrome 15.084746 CALL clock_gettime(0x4,0x7fbfb670) 12747 chrome 15.084815 CALL clock_gettime(0x4,0x7fbfb670) 12747 chrome 15.086620 CALL gettimeofday(0x7efd4650,0x7efd4660) 12747 chrome 15.086736 CALL clock_gettime(0x4,0x7efd4670) 12747 chrome 15.086815 CALL clock_gettime(0x4,0x7efd4670) 12747 chrome 15.098315 CALL gettimeofday(0x7fffafe0,0x7fffaff0) 12747 chrome 15.098680 CALL clock_gettime(0x4,0x7fffb250) Some work was done by kib@ to create a kernel page strong current time and other misc info to eliminate gettimeofday kind syscalls. Bits of it were commited but I'm not sure if it was finished. But anyway calling gettimeofday hundreds of times per second is a chrome bug. *lol* i did exactly the same measurements, you did. :) here are my results: otaku% kdump|grep CALL mmap|wc 7242896 58468 otaku% kdump -s|grep CALL clock_gettime|wc 49545 198180 2772674 otaku% kdump -s|grep CALL linux_clock_gettime|wc 40185 160740 2491298 otaku% kdump -s|grep CALL linux_gettimeofday|wc 21670 86680 1278530 otaku% kdump -s|grep CALL gettimeofday|wc 8173 32692 525053 otaku% kdump -s|grep CALL linux_sys_futex|wc 6191 24764 548800 I suppose linux_* stuff comes from flashplugin. Clearly flash generates more gettime syscalls than chrome itself. Unfortunately the only way to fix this mess in a linux centric world is to implement syscall free gettimeofday with linux ABI support. syscall-free gettimeofday discussion: http://freebsd.1045724.n5.nabble.com/fast-syscall-free-gettimeofday-td4488301.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Heavy I/O blocks FreeBSD box for several seconds
On (11/07/2011 16:36), m...@freebsd.org wrote: On Mon, Jul 11, 2011 at 4:00 PM, Ali Mashtizadeh mashtiza...@gmail.com wrote: Maybe someone can setup something like reviewboard [1] for developers to use. This may also help folks who want to keep abreast of the current work in a particular subsystem or get involved into the development process more. At my company we use reviews and it seems to help the catch some bugs and help new engineers ramp up faster. [1] http://www.reviewboard.org/ FreeBSD development is completely open; anyone can sign up for the svn-src-* mailing list they are interested in, including svn-src-head@. Code reviews are plenty as well; just check the list archives for discussion of bugs, poor design choices and unintended effects. But most reviews are silent and after-the-fact by looking at the list mail. It's a system that seems to be working just fine for the FreeBSD project so far. This isn't a job for most anyone; it's a volunteer project and so anything that raises the barrier to getting work done for the project should be looked at with skepticism. I agree with everything said above and think that it's not reviews that's missing. By review I don't mean something like getting ok to commit reply from N developers before committing. svn-src@ works great for it, commits keep getting reverted :) Review is a time consuming process that also requires certain level of expertise. Volunteer project can hardly afford it. Having a project adopted way of sharing work in progress will be a step forward. Yes, I'm aware of perforce, it's to hard to use and wasn't designed to share and test ideas. I think guthub can be a very good candidate (but AFAIK it won't allow hosting of FreeBSD repo for not paid accounts). I'm not suggesting switching to git as VCS, but using github UI for communication and tracking not yet commited or work in progress changes. In ideal world developers will merge patches from each other increasing chance of a good code to survive and get commited later. Currently we have patches hosted at people.freebsd.org, as attachments on maillists and PRs -- almost all stale or outdated. Key difference of github is that original patch author will be aware of you using it, potentially updating and improving it. Others can continue supporting the patch if original author abandons it, etc. Sending patches is too complicated and counterproductive comparing to github. Thanks, Gleb. Is there a specific deficit that you want to address? Thanks, matthew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Thoughts on TMPFS no longer being considered highly experimental
On (23/06/2011 20:44), Olivier Smedts wrote: 2011/6/23 Alexander V. Chernikov melif...@ipfw.ru: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Jacob wrote: I gave up on using it after a brief try earlier this year. I can't remember the details, but it did lock up my amd64 system. On Thu, 23 Jun 2011, David O'Brien wrote: Does anyone object to this patch? David Wolfskill and I have run TMPFS on a number of machines for two years with no problems. I may have missed something, but I'm not aware of any serious PRs on TMPFS either. There was some issues with sendfile(2) and mmap(2) causing kernel hangs in some cases. vim triggers such hangs for me. However, those problems were fixed and MFCed (afair). I'm using tmpfs on several machines in production without any problems. Maybe being _highly_ experimental for nearly 4 years is enough? :) I think there are still problems with high wired memory consumers like ZFS. I've got 0-sized tmpfs with 8GB RAM + ZFS with 4GB ARC + 4GB swap. There is a patch to make tmpfs memory management more strict (more aggressive), and set default partition size to half of all memory. http://marc.info/?l=freebsd-fsm=129747362722933w=2 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Starting with svn commit r216333 current freezes
On (18/12/2010 10:14), Christian Gusenbauer wrote: Hi! With commit r216333 to pmap.c my PC (i386 32 bit) freezes within a few seconds when X (with nvidia-driver 256.53) starts. I already recompiled and reinstalled the nvidia driver, but this didn't change anything. I also tried the latest nvidia-driver 260.19.29 but without luck, too :-(. By chance I saw a panic message vm_page_unwire: page 0x... wire count is zero, but no crash dump was generated. It works fine for me on amd64. % uname -a FreeBSD tops 9.0-CURRENT FreeBSD 9.0-CURRENT #9 r216475+2eff56c: Thu Dec 16 08:21:48 EET 2010 r...@tops:/usr/obj/freebsd-src/local/sys/TOPS amd64 pciconf: vgap...@pci0:1:0:0: class=0x03 card=0x04411028 chip=0x0a7510de rev=0xa2 hdr=0x00 vendor = 'NVIDIA Corporation' class = display subclass = VGA Xorg log: (--) PCI:*(0:1:0:0) 10de:0a75:1028:0441 nVidia Corporation GT218 [GeForce 310M] rev 162, Mem @ 0xfa00/16777216, 0xc000/268435456, 0xd000/33554432, I/O @ 0xe000/128, BIOS @ 0x/65536 (II) LoadModule: glx (II) Loading /usr/local/lib/xorg/modules/extensions/libglx.so (II) Module glx: vendor=NVIDIA Corporation compiled for 4.0.2, module version = 1.0.0 Module class: X.Org Server Extension (II) NVIDIA GLX Module 256.53 Fri Aug 27 20:49:58 PDT 2010 Any clues? Thanks, Christian. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: tmpfs out of space (ZFS related?)
On (21/11/2010 22:22), Ivan Voras wrote: I got a curious error today while starting PostgreSQL, complaining about out of space errno while creating lock file on /tmp. /tmp on this machine is mounted as tmpfs and indeed, here is the statistic: biggie:/# df -i Filesystem1M-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/mfid0s1a 9912 5193 392657% 306079 1012831 23% / devfs 0 0 0 100% 0 0 100% /dev fdescfs 0 0 0 100% 4 110920% /dev/fd tmpfs 0 0 0 100% 9 0 100% /tmp tank 376044 0 376044 0% 4 7701383470% /tank tank/ports 376658 614 376044 0% 145919 7701383470% /usr/ports tank/mysql 37607329 376044 0% 102 7701383470% /var/db/mysql tank/pgdata90400469 24425 376044 6%1047 7701383470% /tank/pgdata90 On the other hand, top reports this: last pid: 79667; load averages: 0.08, 0.68, 0.77up 1+09:12:13 00:11:33 44 processes: 1 running, 43 sleeping CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 13M Active, 46M Inact, 15G Wired, 232K Cache, 1458M Buf, 8358M Free Swap: 1024M Total, 1024M Free Note the 8358M free report. That's probably integer overflow, could you try specifying max size as mount option. There was a series of integer overflows in tmpfs. Used and available are 0 in df output. On my system I have: % df Filesystem SizeUsed Avail Capacity Mounted on tmpfs 4.3G4.0K4.3G 0%/usr/obj/ports Also note, that tmpfs does somewhat weired thing on constantly recalculating available space based on free memory/swap (not sure if specifying max size would fix it). Did you try double copy elimination patch by k...@? http://lists.freebsd.org/pipermail/freebsd-fs/2010-June/008786.html It doesn't apply cleanly to current any more, but fixing it shouldn't be hard. The patch wouldn't fix this particular issue but looks very promising. Thanks, Gleb The server has ZFS and was doing IO intensive database work on it; the 8 GB free memory comes from PostgreSQL being restarted and freeing the memory (but failing to start again...). Starting PostgreSQL gets me this message: Nov 22 00:18:24 biggie postgres[79696]: [1-1] FATAL: could not write lock file /tmp/.s.PGSQL.5432.lock: No space left on device This is 8-STABLE amd64. Running touch /tmp/abc works, and creates a file. Running echo abc /tmp/abc doesn't return an error but *doesn't write anything to the file*, just creates a directory entry. The status doesn't change over time, i.e. df on tmpfs always shows 0 free. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Fixing and importing the fusefs kernel module - any VFS-savvy takers?
On (28/10/2010 13:49), Ivan Voras wrote: Hello, After a discussion in arch@, it looks like there are many developers interested in having fusefs in the tree but no VFS experts with the time to fix the remaining bugs and basically make it stable enough to commit to the base tree. Fusefs is the Linux-developed userland filesystem interface which is fairly popular in the wild, especially with the sshfs module which allows mounting of generic ssh/sftp directories in a very easy way. Among other filesystems developed for fuse there are some cluster file systems, some crypto file systems and helper file systems used by Gnome and such environments. The initial FreeBSD reimplementation of the kernel module (the userland utlities and libraries don't require complicated porting) was was developed in one of the very early Google Summer of Code projects (2005) and is now in a bit unusual situation: 1) it *is* popular, as reports about its breakage arrive pretty soon after it breaks (i.e. due to mainline kernel changes) 2) it is currently available as a port but it is practically unmaintained. The source code archive is from 2008 and the port contains a dozen patches to be applied to it to make it work on recent systems 3) it is also not exactly rock stable, though this has improved with the above patches; Currently, with sshfs, it is good enough to: - survive blogbench runs - survive fsx runs with arguments -W -R -L, i.e. no mmaped operations, no file size altering / truncate operations There have been claims it also corrupts kernel memory. Basically, this is a call for help in working on fusefs. There are several developers and users willing to do testing and such but no available developers with their hands in the guts of VFS to squash the buried bugs. Fusefs might be especially relevant to desktop users and as such to PC-BSD developers, so I'm cc-ing Kris in case he has a comment. Is anyone interested? Hi Ivan, I didn't reply to thread on @current as it seemed to me that decision was to keep it as it is until someone steps in to maintain it. I'd agree that sshfs is most wanted feature, but fuse_sshfs implementation is broken at best. It doesn't even have notion on inode numbers. It returns all directory entries with d_file=0, the same way st_ino=0. To make it actually work (dirent's with d_file=0 considered empty placeholders by FreeBSD VFS and libc) librefuse fills it with arbitrary numbers. To make long story short stuff like 'cd ..' works for you only because your sh and/or filemanager keeps full path on its own. Lots of other things using VOP_VPTOCNP are also broken. In this particular case puffs_sshfs looks much more promising, although it's explicitly marked as incomplete and buggy. The same applies to vast majority of fuse filesystems. Ignoring it is probably the easiest way to solve the problem, but I'd expect future userlevel filesystem implementation to comply to our VFS. Absence of mmap support is a real show stopper. It's also broken in puffs, and doesn't pass fsx tests. (I've once started implementing it but lost interest in userlevel filesystems after digging deeper into it.) On the other hand adding it to fuse shouldn't be very hard, it's just a question of free time. To sum it up. My personal opinion is that we'd better go with puffs-style approach. Implement userspace-kernel protocol that is as much close to our VFS as possible, and implement proper wrapper like librefuse (which is ok, but looks more like sketch than ready to use library). What I don't like about puffs is that is basically ignores locking serializing request from kernel. I'd suggest we ask a person with skills in network filesystems: locking, caching and all related issues, and improve existing puffs infrastructure accordingly (that includes inode numbers problem I've mentioned before). NFS4 might have interesting ideas. Probably Rick Macklem could express his opinion on this regard. Besides as far as I know OpenAFS has user-kernel interface and implements filesystem at userspace. Sun or Apple have reimplemented fuse in their own, although initial version was based on FreeBSD fuse port. It might be worth checking before moving any further with fuse. References: http://permalink.gmane.org/gmane.os.freebsd.architechture/13623 http://fuse.sourceforge.net/ http://fuse4bsd.creo.hu/ http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/fusefs-kmod/ http://old.nabble.com/forum/Search.jtp?forum=6572local=yquery=fusefs http://old.nabble.com/forum/Search.jtp?forum=6610local=yquery=fusefs ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Fixing and importing the fusefs kernel module - any VFS-savvy takers?
On (28/10/2010 22:24), Ivan Voras wrote: On 28 October 2010 16:15, Gleb Kurtsou gleb.kurt...@gmail.com wrote: I'd agree that sshfs is most wanted feature, but fuse_sshfs implementation is broken at best. It doesn't even have notion on inode numbers. It returns all directory entries with d_file=0, the same way st_ino=0. To make it actually work (dirent's with d_file=0 considered empty placeholders by FreeBSD VFS and libc) librefuse fills it with arbitrary numbers. To make long story short stuff like 'cd ..' works for you only because your sh and/or filemanager keeps full path on its own. Lots of other things using VOP_VPTOCNP are also broken. In this particular case puffs_sshfs looks much more promising, although it's explicitly marked as incomplete and buggy. The same applies to vast majority of fuse filesystems. Ignoring it is probably the easiest way to solve the problem, but I'd expect future userlevel filesystem implementation to comply to our VFS. For these fuse-sshfs problems, how many are the problem of sshfs (the userland code), the FUSE API (because it's designed for Linux) and the fuse4bsd kernel module (because it's unfinished and buggy)? These are sshfs problems, and the real problem is that user level filesystems are of much lower quality than kernel level. Writing good filesystems in userspace shouldn't be much easier than writing kernel one (not counting fancy language of choice and ntfs-3g-like use of binary drivers). All the kernel restrictions and requirements are still there nor puffs, nor fuse do black magic for you. Absence of mmap support is a real show stopper. It's also broken in puffs, and doesn't pass fsx tests. (I've once started implementing it but lost interest in userlevel filesystems after digging deeper into it.) On the other hand adding it to fuse shouldn't be very hard, it's just a question of free time. To sum it up. My personal opinion is that we'd better go with puffs-style approach. Implement userspace-kernel protocol that is as much close to our VFS as possible, and implement proper wrapper like librefuse (which is ok, but looks more like sketch than ready to use library). What I don't like about puffs is that is basically ignores locking serializing request from kernel. I'm trying to avoid dispersal of effort. Basically, I won't be convinced to support puffs until someone who knows (possibly you if you are up to it) demonstrates that the refuse API is stable enough and practically 100% compatible with FUSE - No-no, I'm not convincing you to choose puffs. It's too immature. I just like the approach. some of their file systems look fit for serious use and 99% isn't very much when talking about OS stability. That's why I'm so sceptical about fuse, puffs, and entire concept in general. Is fuse really that stable on linux? Do people use it on production servers? On the other hand, if a developer suddenly appears and says I will make puffs+refuse work I will support him completely and I will stop crusading for fuse because having puffs is better than nothing. :) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: pam_pefs setup (Re: RFC: pefs - stacked cryptographic filesystem)
On (08/09/2010 11:25), Daniel O'Connor wrote: On 08/09/2010, at 3:22, Gleb Kurtsou wrote: Please note that your home directory has to be mounted, I mount it in /etc/rc.local, but don't add any keys. pam_pefs adds the key. Also note that it has to be exactly your home directory (/home/gleb in my case), to prevent possible attacks. And keychain database has to be created, so that pam_pefs knows how to verify the key. Have you considered something similar to pam_mount? (http://pam-mount.sourceforge.net/) ie pam_pefs could mount your home directory itself and unmount it on logout. I knew about pam_mount before starting pam_pefs and my intent was to keep pam_pefs as simple as possible. Unlike some other stacked cryptographic filesystems, pefs can be mounted in read-only mode without providing a key. pam_mount can be used together with pam_pefs to mount/unmount filesystem on login/logout if needed. pam_mount is more generic then pam_pefs. At the moment pam_pefs doesn't remove key on logout because it is a bit tricky if there are several keys installed. I'm also against the idea of keeping keys installed by current session during the session to remove them on logout. Key chains for different sessions may overlap. I'd suggest to use pam_mount to unmount filesystem on logout in such scenario. Thanks, Gleb. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: pefs - stacked cryptographic filesystem
On (07/09/2010 14:28), Robert Watson wrote: On Mon, 6 Sep 2010, Gleb Kurtsou wrote: I would like to ask for feedback on a kernel level stacked cryptographic filesystem. It has started as Summer Of Code'2009 project and matured a lot since then. I've recently added support for sparse files and switched to XTS encryption mode. I've been using it to encrypt my home directory for almost a year already, and use fsx, dbench and blogbench for testing. So it should be fairly stable. Tested on top of ZFS, UFS and tmpfs on amd64 and i386; both 9-CURRENT and 8-STABLE supported. Please email me separately if you're willing to help testing on big endian machine, XTS code doesn't look endian correct. At this point all of the project goals complete and I'd like it to get wider coverage in terms of tests and reviews and hope to see it commited to HEAD soon. Hi Gleb: This sounds like really exciting work! Do you have much in the way of formal documentation of your crypto design at this point? I'd like to point some of the local crypto gurus at Cambridge at it to do some analysis of your approach. However, as they rightly point out, reverse engineering crypto from code is rather a high barrier of entry for a crypto review, so detailed documentation of the approach and a formal format description would be much prefered :-). Hello Robert, I've updated my older blog post on pefs crypto design, it's not formal but hope it helps. If there is anything else you might need, I'd be happy to help: http://glebkurtsou.blogspot.com/2009/09/pefs-crypto-primitives.html Thanks, Gleb Thanks, Robert Installation instructions: 1a. Clone git repository: # git clone git://github.com/glk/pefs.git pefs # cd pefs 1b. Or download latest snapshot from github: http://github.com/glk/pefs/archives/master 2. Build and install: # make obj all # make install 3. Mount pefs filesystem: # pefs mount ~/Private ~/Private 4. Enter passphrase: # pefs addkey ~/Private 5. Test it and report back. There is also a man page available. 6. Example how to save your key in keychain database. pefs has to be mounted and key specified to make fs writable, create keychain with single entry (keychain -Z option): # pefs addchain -Z ~/Private Don't encrypt .pefs.db: # mv ~/Private/.pefs.db /tmp # umount ~/Private # mv /tmp/.pefs.db ~/Private # pefs mount ~/Private ~/Private Use -c option to verify key is in database # pefs addkey -c ~/Private 7. You can setup pam_pefs (not compiled by default) to add key to home directory and authenticate against keychain database on login, e.g. by adding the following line to /etc/pam.d/system before pam_unix.so: authsufficient pam_pefs.so try_first_pass The following is a list of its most important features: * Kernel level file system, no user level daemons needed. Transparently runs on top of existing file systems. * Random per file tweak value used for encryption, which guaranties different cipher texts for the same encrypted files. * Saves metadata only in encrypted file name, but not in file itself. * Supports arbitrary number of keys per file system, default directory key, mixing files encrypted with different keys in same directory. * Allows defining key chains, can be used to add/delete several keys by specifying only master key. * Uses modern cryptographic algorithms: AES and Camellia in XTS mode, PKCS#5v2 and HKDF for key generation. Github repository: http://github.com/glk/pefs More details on my blog: http://glebkurtsou.blogspot.com/search/label/pefs Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
pam_pefs setup (Re: RFC: pefs - stacked cryptographic filesystem)
On (07/09/2010 16:32), Thomas Vogt wrote: [...] I've an issue with pam_pefs: === lib/libpam/modules/pam_pefs (install) install -C -o root -g wheel -m 444 libpam_pefs.a /usr/lib install -C -o root -g wheel -m 444 libpam_pefs_p.a /usr/lib install -o root -g wheel -m 444 pam_pefs.8.gz /usr/share/man/man8 I do not see any pam_pefs.so which makes login not possible if pam.d/system is modified as mentioned in your description: auth sufficient pam_pefs.so try_first_pass Sorry, I don't quite understand you here. Don't hesitate contacting me again if didn't understand you correctly. I've also missed one more line, which actually adds the key: session optionalpam_pefs.so Setup I've posted makes possible to login using pefs key or standard pam_unix.so password. Here is my /etc/pam.d/system file: # auth authsufficient pam_opie.so no_warn no_fake_prompts authrequisite pam_opieaccess.so no_warn allow_local authsufficient pam_pefs.so try_first_pass authrequiredpam_unix.so no_warn try_first_pass nullok # account account requiredpam_login_access.so account requiredpam_unix.so # session session optionalpam_pefs.so session requiredpam_lastlog.so no_fail # password passwordrequiredpam_unix.so no_warn try_first_pass I have stronger password for pefs, while traditional password is weaker and easier to type. I use pefs password to login only the first time and add key my home directory. Please note that your home directory has to be mounted, I mount it in /etc/rc.local, but don't add any keys. pam_pefs adds the key. Also note that it has to be exactly your home directory (/home/gleb in my case), to prevent possible attacks. And keychain database has to be created, so that pam_pefs knows how to verify the key. Details on how to create it available in my original email. That's rather inconvenient procedure, but you need to do it just once, it's so complicated because pefs is read-only if no key specified, but database should not be encrypted to make it accessible by pam_pefs: 3. Mount pefs filesystem: # pefs mount /home/ME /home/ME 4. Enter passphrase: # pefs addkey /home/ME # pefs addchain -Z /home/ME Don't encrypt .pefs.db: # mv ~/Private/.pefs.db /tmp # umount ~/Private # mv /tmp/.pefs.db /home/ME # pefs mount /home/ME /home/ME Use -c option to verify key is in database # pefs addkey -c /home/ME I'll try to make it easier, I didn't actually expect anyone to try it, and just mentioned it without providing instructions not to write long setup procedure. You can also try adding debug option to pam_pefs.so config if something goes wrong. I don't remember details but pefs/Makefile contains the following comment by me: # Should be built from sources tree # SUBDIR+= lib/libpam/modules/pam_pefs But if you are able to build it, it should be fine. Thanks, Gleb. Regards, Thomas ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: pefs - stacked cryptographic filesystem
Thanks to Peter Holm and Thomas Vogt for finding several bugs: * Compilation with DIAGNOSTIC option * Vnode reference and lock leak in pefs_rename() I've uploaded new version to test: http://github.com/downloads/glk/pefs/pefs-2010-09-07.tar.gz Github repository is also updated. Also note, that if you have extra debugging options like DEBUG_LOCKS in your kernel config pefs module has to be build with same options. I set KERNBUILDDIR to my kernel build directory to make it work: # uname -v FreeBSD 9.0-CURRENT #25 r212049+d758796: Tue Aug 31 22:09:45 EEST 2010 r...@tops:/usr/obj/freebsd-src/local/sys/TOPS # export KERNBUILDDIR=/usr/obj/freebsd-src/local/sys/TOPS # cd pefs/sys/modules/pefs # make clean # make make install Thanks, Gleb. On (06/09/2010 21:38), Gleb Kurtsou wrote: Hello, I would like to ask for feedback on a kernel level stacked cryptographic filesystem. It has started as Summer Of Code'2009 project and matured a lot since then. I've recently added support for sparse files and switched to XTS encryption mode. I've been using it to encrypt my home directory for almost a year already, and use fsx, dbench and blogbench for testing. So it should be fairly stable. Tested on top of ZFS, UFS and tmpfs on amd64 and i386; both 9-CURRENT and 8-STABLE supported. Please email me separately if you're willing to help testing on big endian machine, XTS code doesn't look endian correct. At this point all of the project goals complete and I'd like it to get wider coverage in terms of tests and reviews and hope to see it commited to HEAD soon. Installation instructions: 1a. Clone git repository: # git clone git://github.com/glk/pefs.git pefs # cd pefs 1b. Or download latest snapshot from github: http://github.com/glk/pefs/archives/master 2. Build and install: # make obj all # make install 3. Mount pefs filesystem: # pefs mount ~/Private ~/Private 4. Enter passphrase: # pefs addkey ~/Private 5. Test it and report back. There is also a man page available. 6. Example how to save your key in keychain database. pefs has to be mounted and key specified to make fs writable, create keychain with single entry (keychain -Z option): # pefs addchain -Z ~/Private Don't encrypt .pefs.db: # mv ~/Private/.pefs.db /tmp # umount ~/Private # mv /tmp/.pefs.db ~/Private # pefs mount ~/Private ~/Private Use -c option to verify key is in database # pefs addkey -c ~/Private 7. You can setup pam_pefs (not compiled by default) to add key to home directory and authenticate against keychain database on login, e.g. by adding the following line to /etc/pam.d/system before pam_unix.so: auth sufficient pam_pefs.so try_first_pass The following is a list of its most important features: * Kernel level file system, no user level daemons needed. Transparently runs on top of existing file systems. * Random per file tweak value used for encryption, which guaranties different cipher texts for the same encrypted files. * Saves metadata only in encrypted file name, but not in file itself. * Supports arbitrary number of keys per file system, default directory key, mixing files encrypted with different keys in same directory. * Allows defining key chains, can be used to add/delete several keys by specifying only master key. * Uses modern cryptographic algorithms: AES and Camellia in XTS mode, PKCS#5v2 and HKDF for key generation. Github repository: http://github.com/glk/pefs More details on my blog: http://glebkurtsou.blogspot.com/search/label/pefs Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: pefs - stacked cryptographic filesystem
On (07/09/2010 10:57), Kevin Oberman wrote: On Mon, 6 Sep 2010, Gleb Kurtsou wrote: I would like to ask for feedback on a kernel level stacked cryptographic filesystem. It has started as Summer Of Code'2009 project and matured a lot since then. I've recently added support for sparse files and switched to XTS encryption mode. I've been using it to encrypt my home directory for almost a year already, and use fsx, dbench and blogbench for testing. So it should be fairly stable. Tested on top of ZFS, UFS and tmpfs on amd64 and i386; both 9-CURRENT and 8-STABLE supported. Please email me separately if you're willing to help testing on big endian machine, XTS code doesn't look endian correct. At this point all of the project goals complete and I'd like it to get wider coverage in terms of tests and reviews and hope to see it commited to HEAD soon. I've got to ask a probably dumb question...how is this better then geli encrypted objects? I've used them for sometime with excellent results. Or does it provide functionality that geli does not? If geli works for you I'd suggest you continue using geli, geli design is order of magnitude simpler due to being block device but not a filesystem. Besides geli provides data authentication, which is very important, and something that can't be easily implemented in stacked filesystem. Pros of stacked filesystem approach: * You don't have to create separate filesystem (use separate partition), just encrypt part of existing filesystem * It's easier to create/manage snapshots/backups (imho) * Ability to use multiple keys, each directory can have its own key, files with different keys can be mixed in one directory, etc. You don't have to create another filesystem and claim that you don't know password for this one if you're asked to provide it, in case you have something to hide ;) Cons: * pefs maximum file name length is ~180 bytes, but not 255 * Stacked filesystem is likely to be slower due to extra overhead I don't really know what makes one better then other, it has only on thing in common - encryption - everything else is different. It depends on how you intend to use it. I was using geli myself, the only problem I had was that at some point I realized that initial partition size was too small :) But that more or less applies to both approaches. E.g. non-standard example where stacked filesystem may be preferred is to use it for encrypted crash dumps: if dump available on dumpdev mount /var/crash as pefs; ask user for password, or automatically add random one and let user to save it later after boot. Crash dumps are encrypted without extra cost of setting up partition, geli, etc. Thanks, Gleb. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: ober...@es.netPhone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: pefs - stacked cryptographic filesystem
On (07/09/2010 17:04), Ivan Voras wrote: On 09/06/10 20:38, Gleb Kurtsou wrote: Tested on top of ZFS, UFS and tmpfs on amd64 and i386; both 9-CURRENT and 8-STABLE supported. You probably didn't test it, but I've tried pefs on top of ext2fs (I use ext2fs to share data between OSes) and it quickly panicked. I've tried it once, it paniced. msdosfs is known to work, but I'd say it shouldn't :) I'll look at it tomorrow, thanks for pointing it. If you are going to use it note that lower filesystem should be case sensitive, the same applies to ZFS. Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: pefs - stacked cryptographic filesystem
On (07/09/2010 16:27), Ivan Voras wrote: On 09/06/10 20:38, Gleb Kurtsou wrote: Hello, I would like to ask for feedback on a kernel level stacked cryptographic filesystem. It has started as Summer Of Code'2009 project and matured a lot since then. I've recently added support for sparse files and switched to XTS encryption mode. I've tried it and so far it works :) 3. Mount pefs filesystem: # pefs mount ~/Private ~/Private I see you've used the same example in the man page. Maybe it would be better for educational purposes to use two separate directories, e.g. ~/Private and ~/Decrypted to avoid confusion by new users (of course not all examples need to use this). Actually I've used the same directory solely for educational purposes -- there is just one directory, it's either encrypted or not. Users should think of it as of nullfs on steroids, but that doesn't sound acceptable for man page. Seriously, any help on documentation is more than welcome. I'm not good at writing it, and there is too much to document. We could discuss all the details privately to write a decent man page, if you have time to help. 6. Example how to save your key in keychain database. This is probably in line with what rwatson said (and would be covered by the same document): can you describe what keychains actually do? Yes. I was thinking about 6 and 7 as of feature list, so no documentation or instructions. Keychains are mentioned in man page, but once again documentation is far from being good. Help wanted. Most important fact is that keychains have nothing to do with filesystem, it's solely userspace utility concept. Chain is a series of keys (keychain db entries). Each db entry consists of two keys: parent key and child key. Child key may be zero, i.e. end of chain marker, (created by addchain -Z). Keychain database itself is a dictionary of the following form: db[parent_key] = child_key Consider the following database: db[k0] = 0 (zero child key) db[k1] = k2 db[k2] = k3 db[k3] = 0 k0 is special. Its child key is zero (end of chain). Chain would consist only of one key: k0 If user enters k1, the following chain can be retrieved from the database: k1 k2 k3. All three keys are then added to filesystem. In case of k2 chain is k2 k3. All entries stored encrypted in a way that child entry can be decrypted only by parent key. Using key chains one can emulate access levels. (Don't miss 'pefs randomchain' command which was invented especially to make some fun for those trying to look at your data) 7. You can setup pam_pefs (not compiled by default) to add key to home directory and authenticate against keychain database on login, e.g. by adding the following line to /etc/pam.d/system before pam_unix.so: authsufficient pam_pefs.so try_first_pass So, this would bypass passwd and let the user in if his password authenticates against the keychain database in his home directory? Exactly, that's the way I use it. More detailed description available here: http://marc.info/?l=freebsd-currentm=128388197901390w=2 Will it automagically pefs-mount his home directory? No, not mounting pefs is intentional. It automagically adds keys to already mounted pefs filesystem. * Uses modern cryptographic algorithms: AES and Camellia in XTS mode, PKCS#5v2 and HKDF for key generation. I do have an request: since you are already using kernel crypto support, it would be simple to just throw Blowfish in :) If for nothing else, consider it a gift to those who are fond of Blowfish's large key sizes (upto 448 bits). XTS requires 128 bit block cipher. One can use Blowfish to encrypt two 64 bit subblocks, but one would need to mix these blocks somehow, that what XTS does :) A bit tricky to implement. Adding serpent or twofish should be easy. Actually, it would probably be seen as a reflection of consistency to implement the same algorithms that geli(8) implements. geli doesn't implement XTS yet - if your XTS code proves to be stable it would be a good thing to include it as standard and then use it from geli. The problem is 128 bit block in XTS, in my opinion hacking XTS to add support for algorithms with other block sizes is a very bad idea. There are more than enough modern algorithms that can be used instead. Not sure how far reflection of consistency extends but pefs is very likely not to support modes other then XTS, because for now it's the only widespread standard mode of operation with arbitrary sector size and arbitrary offset encryption. I see you've copied SHA2 code to the pefs code. What is wrong with just using the kernel's SHA2 implementation? That was accidental, or there was a reason I've already forgotten about. Anyway it's just a copy from kernel, my intention was to make it easier to test by users. There's hmac which is one-to-one copy from geli. I've extracted it because now both geli and pefs use same code. Thanks, Gleb
RFC: pefs - stacked cryptographic filesystem
Hello, I would like to ask for feedback on a kernel level stacked cryptographic filesystem. It has started as Summer Of Code'2009 project and matured a lot since then. I've recently added support for sparse files and switched to XTS encryption mode. I've been using it to encrypt my home directory for almost a year already, and use fsx, dbench and blogbench for testing. So it should be fairly stable. Tested on top of ZFS, UFS and tmpfs on amd64 and i386; both 9-CURRENT and 8-STABLE supported. Please email me separately if you're willing to help testing on big endian machine, XTS code doesn't look endian correct. At this point all of the project goals complete and I'd like it to get wider coverage in terms of tests and reviews and hope to see it commited to HEAD soon. Installation instructions: 1a. Clone git repository: # git clone git://github.com/glk/pefs.git pefs # cd pefs 1b. Or download latest snapshot from github: http://github.com/glk/pefs/archives/master 2. Build and install: # make obj all # make install 3. Mount pefs filesystem: # pefs mount ~/Private ~/Private 4. Enter passphrase: # pefs addkey ~/Private 5. Test it and report back. There is also a man page available. 6. Example how to save your key in keychain database. pefs has to be mounted and key specified to make fs writable, create keychain with single entry (keychain -Z option): # pefs addchain -Z ~/Private Don't encrypt .pefs.db: # mv ~/Private/.pefs.db /tmp # umount ~/Private # mv /tmp/.pefs.db ~/Private # pefs mount ~/Private ~/Private Use -c option to verify key is in database # pefs addkey -c ~/Private 7. You can setup pam_pefs (not compiled by default) to add key to home directory and authenticate against keychain database on login, e.g. by adding the following line to /etc/pam.d/system before pam_unix.so: authsufficient pam_pefs.so try_first_pass The following is a list of its most important features: * Kernel level file system, no user level daemons needed. Transparently runs on top of existing file systems. * Random per file tweak value used for encryption, which guaranties different cipher texts for the same encrypted files. * Saves metadata only in encrypted file name, but not in file itself. * Supports arbitrary number of keys per file system, default directory key, mixing files encrypted with different keys in same directory. * Allows defining key chains, can be used to add/delete several keys by specifying only master key. * Uses modern cryptographic algorithms: AES and Camellia in XTS mode, PKCS#5v2 and HKDF for key generation. Github repository: http://github.com/glk/pefs More details on my blog: http://glebkurtsou.blogspot.com/search/label/pefs Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: pefs - stacked cryptographic filesystem
Sorry for replying to myself, I've realized I put wrong download link: http://github.com/downloads/glk/pefs/pefs-2010-09-06.tar.gz On (06/09/2010 21:38), Gleb Kurtsou wrote: Hello, I would like to ask for feedback on a kernel level stacked cryptographic filesystem. It has started as Summer Of Code'2009 project and matured a lot since then. I've recently added support for sparse files and switched to XTS encryption mode. I've been using it to encrypt my home directory for almost a year already, and use fsx, dbench and blogbench for testing. So it should be fairly stable. Tested on top of ZFS, UFS and tmpfs on amd64 and i386; both 9-CURRENT and 8-STABLE supported. Please email me separately if you're willing to help testing on big endian machine, XTS code doesn't look endian correct. At this point all of the project goals complete and I'd like it to get wider coverage in terms of tests and reviews and hope to see it commited to HEAD soon. Installation instructions: 1a. Clone git repository: # git clone git://github.com/glk/pefs.git pefs # cd pefs 1b. Or download latest snapshot from github: http://github.com/glk/pefs/archives/master Or use direct download link: http://github.com/downloads/glk/pefs/pefs-2010-09-06.tar.gz 2. Build and install: # make obj all # make install 3. Mount pefs filesystem: # pefs mount ~/Private ~/Private 4. Enter passphrase: # pefs addkey ~/Private 5. Test it and report back. There is also a man page available. 6. Example how to save your key in keychain database. pefs has to be mounted and key specified to make fs writable, create keychain with single entry (keychain -Z option): # pefs addchain -Z ~/Private Don't encrypt .pefs.db: # mv ~/Private/.pefs.db /tmp # umount ~/Private # mv /tmp/.pefs.db ~/Private # pefs mount ~/Private ~/Private Use -c option to verify key is in database # pefs addkey -c ~/Private 7. You can setup pam_pefs (not compiled by default) to add key to home directory and authenticate against keychain database on login, e.g. by adding the following line to /etc/pam.d/system before pam_unix.so: auth sufficient pam_pefs.so try_first_pass The following is a list of its most important features: * Kernel level file system, no user level daemons needed. Transparently runs on top of existing file systems. * Random per file tweak value used for encryption, which guaranties different cipher texts for the same encrypted files. * Saves metadata only in encrypted file name, but not in file itself. * Supports arbitrary number of keys per file system, default directory key, mixing files encrypted with different keys in same directory. * Allows defining key chains, can be used to add/delete several keys by specifying only master key. * Uses modern cryptographic algorithms: AES and Camellia in XTS mode, PKCS#5v2 and HKDF for key generation. Github repository: http://github.com/glk/pefs More details on my blog: http://glebkurtsou.blogspot.com/search/label/pefs Thanks, Gleb. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [CFT] BSDL iconv in base system
On (15/06/2010 02:13), Gabor Kovesdan wrote: Hello Folks, during the last summer, Google generously founded my Summer of Code project, which was providing a BSD-licensed iconv implementation for FreeBSD. I'm proud to announce that the work has been completed and a patch is available to add it to the base system. The results of this work are: - The Citrus implementation has been ported from NetBSD. - Some utilities have been added. There is a conversion table generator, which can compare conversion tables to reference data generated by GNU libiconv. This helps ensuring conversion compatibility. - UTF-16 surrogate support and some endianness issues have been fixed. - The rather chaotic Makefiles to build metadata have been refactored and cleaned up, now it is easy to read and it is also easier to add support for new encodings. - A bunch of new encodings and encoding aliases have been added. - Support for 1-2, 1-3 and 1-4 mappings, which is needed for transliterating with flying accents as GNU does, like u. - Lots of warnings have been fixed, the major part of the code is now WARNS=6 clean. - New section 1 and section 5 manual pages have been added. - Some GNU-specific calls have been implemented: iconvlist(), iconvctl(), iconv_canonicalize(), iconv_open_into() - Support for GNU's //IGNORE suffix has been added. - The - argument for stdin is now recognized in iconv(1) as per POSIX. - The Big5 conversion module has been fixed. - The iconv.h header files is supposed to be compatible with the GNU version, i.e. sources should build with base iconv.h and GNU libiconv. I've just did a very quick test and it seems ports can safely link to GNU libiconv, there's no conflict. - Various cleanups and style(9) fixes. - A bachelor thesis written in Hungarian language: http://www.kovesdan.org/files/bsc_iconv.pdf The rather big patch (42,5M) is available here: http://www.kovesdan.org/patches/iconv_base_integrate.diff Any comments, suggestions or bugreports are very welcome. Awesome! Thanks for working on it. Are there any plans to resurrect/finish multibyte collation support GSoC'2008 project: http://wiki.freebsd.org/KonradJankowski/Collation And are you aware of any plans on adding utf8-aware regex? I think NetBSD has already imported one: http://blog.netbsd.org/tnf/entry/efficient_wide_character_regular_expressions Thanks, Gleb. -- Gabor Kovesdan FreeBSD Volunteer EMAIL:ga...@freebsd.org .:|:.ga...@kovesdan.org WEB:http://people.FreeBSD.org/~gabor .:|:.http://kovesdan.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
