Re: WineConf Agenda
On Mon, Apr 04, 2005 at 04:19:36PM +1000, Andrew Tridgell wrote: Personally I think that keeping a full alternative dcache in userspace isn't the right approach. I think we could bear the hit of something that keeps O(number_of_directories) memory, but not O(number_of_files). The lack of a sensible way to auto-shrink the user space dcache in low memory situations is just one of the many problems of a full userspace dcache. But using your status bits, one can compress such a cache significantly. That is, using the fact that most files on an Unix FS are not mixed case, we can avoid storing O(number_of_files). We would then need to store all files only in r/w[1] directories with mixed case filesystems[2]. So we would be trading a little bit of memory for speed. Not a bad tradeoff these days. -- Dimi. 1. We can avoid doing so for directories that are writable by root only, those are not likely to change. 2. Essentially, those are under /home. Which means that a first order approximation would be a simple cache that works blindly on directories that match a simple regexp pattern. For most cases, that would simply be '/home/.*', and I think that would get us 90% there.
Re: WineConf Agenda
I should also say that if the WineConf agenda doesn't have enough room for us to complete our Wine-Samba discussions, then I'd like to invite any Wine developers who would like to continue the discussion to come to the SambaXP conference which takes place directly after WineConf just a few hours north of Stuttgart in Germany. See http://www.sambaxp.org/ for more details. I will be in Germany from the 28th of April until the evening of the 7th of May, so there should be plenty of time to have some in-depth discussions with anyone who is interested both at WineConf, at SambaXP and even after the end of SambaXP. The Wine and Samba projects have a lot in common, so I really hope that this will be the beginning of a strong relationship between us. Cheers, Tridge
Re: WineConf Agenda
Dimitrie, Speaking of which, both Samba and Wine suffer from the lack of support of cases-insensitivness in the kernel. Now, I'm not suggesting that we should push for that (I'm aware of the past discussions on LKML, and I must agree with Linus), but we should push for *something* that's acceptable and helps us solve the negative lookups a bit faster than a full directory listing every time. I have a proposed solution for that which needs no kernel modifications. I have been meaning to write this up properly, so if you like I can do that for WineConf. Here is the (probably inadequate) rough description: The key is to look at the relative probability of different case combinations of names created using posix interfaces. Names created using Win32 style interfaces (via Wine or Samba) cover all different case combinations, but names created by unix users tend (to a very large degree) to use a much more restricted set of case combinations. So, proposed solution is: 1) store the full case preserved name in a xattr 2) programs like Samba and Wine will lookup this xattr element and return that case preserved name in directory listings 3) keep a shared memory cache bitmap indexed by directory inode. It would consume memory of 4 bits per directory plus one microsecond timestamp per directory when full (note, it does not consume memory per-filename, only per directory). This keeps the memory usage low. 4) the 4 bits would indicate the state of that directory. Default state of a directory is 0, meaning unknown (that state is implied if an entry for that directory is not present in the shared cache) 5) other states would mean things like: i) some filename in this directory may have a uppercase character in the leading position only. ii) some filename in this directory might have a uppercase character in any position iii) some filename in this directory may have a leading set of uppercase characters iv) all names in this directory are lowercase 6) when storing a name in the posix filesystem, we store the lowercase name, plus store the case preserved name in the xattr 7) when needing to look for a name in a directory, we check the shared bitmap. If it says unknown then we have to scan the directory as usual 8) when scanning the directory, we update the global bitmap if needed with anything we find. For example, if we only come across names that in the posix world are all lowercase then we set that state. 9) when we look for a name and the global cache indicates that (for example) there may be a file with 1 uppercase char in the leading position then we need to call stat() at most twice. As most filesystems these days use hash based directories, this stat() call costs us O(log N), which is a heck of a lot better than O(N) for a full scan. 10) cache coherence is done by the microsecond timestamp on the directory. This isn't perfect, but is the best we can do without kernel help. We might possibly be able to convince the kernel guys to give us a better coherence method (such as a in-memory revision count per directory). The end result of this plan is this: - for directories that are accessed only by Wine/Samba, and not by posix programs, we will operate very fast (ie. O(log N)) and completely accurately. We will be case insensitive and case preserving. - for directories that are accessed only by posix systems, there is no overhead - for mixed directories, Win32 names will appear lowercase to posix programs, but will be case preserving to Wine/Samba. Wine/Samba will be fast as long as posix programs don't create names like MyFileNameIsReallyCool. It doesn't matter if Wine/Samba create names like that, only if posix does. Worst case is what we have now (full scan if we have silly posix users). This all relies on the fact that posix programs tend to quite rarely create filenames with lots of upper/lowercase. You get some like Mail, and Desktop, but we can cope with those easily using bits as above. We just need to judge at what stage doing several stat() calls pays off versus just doing a scan. That will require some tuning (it might be useful to keep an estimate of the directory size in the cache to guide this heuristic). Obviously there are many details I am glossing over (locking and permissions on the global cache for example). I am confident these can be solved quite easily. Cheers, Tridge
Re: WineConf Agenda
Steven, Finally, if we have time, I'd like to discuss cooperation on IDL files and MSRPC interfaces. Yes I would like to discuss this as well. Eirc Kohl has made quite a lot of changes to WIDL for ReactOS to support our MSRPC and PLug and Pray implementation and I would like to see these changes merged back in for Wine and Samba so we can adopt a common IDL compiler. My take on the way we could cooperate on IDL is that the most effective route is by sharing IDL files, and sharing knowledge. I think that sharing the actual IDL compiler is much more problematic technically. The reason for this is that Wine really needs RPC/IDL support with compatibility at the object code level, so that existing Win32 code that calls the low level marshalling APIs continues to work. This puts a strong restriction on the approach your compiler needs to take, including needing to use the format string table based approach. Please correct me if I am wrong about this. The Samba project doesn't need object code compatibility, we need only wire compatibility. While we could implement wire compatibility via object code compatibility, it might significantly restrict some of the more interesting things we are doing with IDL at the moment. We have taken the notion of IDL for RPC interfaces and expanded it to be used for all sorts of regular marshalling/unmarshalling tasks, including NBT (and WINS), DGRAM packets and even for on-disk format of xattrs for extended NTFS filesystem attributes. To do this, we have extended IDL in our compiler in quite a number of ways, allowing for constructs that MIDL does not handle. We would be very reluctant to give up those extensions as they have proved to be very valuable, and it might prove difficult to keep those extensions in some cases if we adopted the table based IDL compiler approach. While I am certainly open to discussing ways in which this might change, I suspect that for the moment the needs of our two compilers will keep them separate. What we can do, however, is cooperate on the IDL files themselves and on test suites and knowledge about how the associated machinery works (such as the many RPC auth, sign and seal methods). To this end we have put the following note in our IDL directory: The IDL files in this directory are made available by the Samba Team under the following license: Permission to use, copy, modify, and distribute these interface definitions for any purpose is hereby granted without fee. This work is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. It is our hope that this is sufficient for the Wine developers to be happy about using our IDL files in Wine, or at least grabbing any bits that are useful. Our collection of IDL files is not complete, but it is growing and already includes quite a lot of useful stuff. We also have quite extensive test suites for most of the IDL we have written. Another thing we should discuss is the methods by which we develop new IDL files that are compatible with the interfaces that Microsoft uses in MSRPC. We have become quite proficient in that area, and it might be useful to have some discussions with some interested Wine developers on how that is done. I'm sure we can learn things from each other in that area. Cheers, Tridge
Re: WineConf Agenda
Dimi, - for mixed directories, Win32 names will appear lowercase to posix programs, but will be case preserving to Wine/Samba. Wine/Samba This is something I don't really like: to change the filename's case behind user's back. From my POV, it's not acceptable. I know I'd be pissed if that happened to me. It only changes the name from the point of view of posix. The name is completely preserved from the point of view of Wine/Samba. I can still understand this annoying some people, but I don't think it would annoy the majority of Samba users. I think we'd be much better off if we relied on some kernel feature. The current mechanism works just fine, if you want the same behaviour, but faster, you need such and such in the kernel. It's a much simpler (and cleaner) case to make then to change such fundamental behaviour. yes, but you will run up against huge objections from the kernel community. The key sticking point is that filenames can no longer be treated as bags-of-bytes. You have to start understanding character sets in the kernel in a very fundamental way to implement case insensitivity. I tried to argue that the kernel should start to understand UTF-8 and was shot down with extreme prejudice. The kernel guys know this means great pain for us, but they are unwilling to budge. It might be that NFSv4 will force this issue, but it might not. They might end up with the same sort of solutions we implement now. That is why I came up with my proposed solution. The speed of the current scan directory system is just way too slow and doing it right from our point of view (making the filesystem case insensitive) appears not to be an option. My proposal is a attempt to salvage something workable. In Samba we would make this an option. It would be there to solve the speed problem and if users don't like the consequences (like losing the case preservation from the point of view of posix apps) then they can disable it. Cheers, Tridge
Re: WineConf Agenda
Dimitrie O. Paun wrote: On Sun, Apr 03, 2005 at 08:57:58PM +1000, Andrew Tridgell wrote: In each case I will be trying to encourage methods which can store the full NTFS semantics, rather than limiting ourselves to only the things that fit natually in posix filesystems. I'm guessing we will have some lively discussion on whether this is a worthwhile aim :-) Speaking of which, both Samba and Wine suffer from the lack of support of cases-insensitivness in the kernel. Now, I'm not suggesting that we should push for that (I'm aware of the past discussions on LKML, and I must agree with Linus), but we should push for *something* that's acceptable and helps us solve the negative lookups a bit faster than a full directory listing every time. Linux 2.6 has an API to monitor directory changes right? We could have our own directory list hash in RAM. //Jakob
Re: WineConf Agenda
Dimi, I understand that, but I'd certainly fall into the minority of Samba users then. I share my /home/dimi directory via Samba to my laptop, there's no way I want my filenames modified by the system. Existing filenames, and names that are crated by posix don't get modified of course. By you are right that the case of names created on windows would not be seen by posix. Of course, a simple mod to /bin/ls or a LD_PRELOAD that hooks readdir() and some other calls could make posix apps see the case preserved name. Whether that is a good idea is debatable. In Wine (and I would have thought in Samba too) we try to integrate as much as possible the two (Win32 and POSIX) environments. A solution that looks good only from a Wine/Samba standpoint fails on this criteria. Obviously, we want it to look good from both perspectives simultaneously. yes, that is the ideal, it just depends how much you are willing to pay for this in performance. When you are waiting many minutes for something that should take a second or two you start to notice the time. I forget now the outcome of that discussion, whatever happened to Linus' proposal for that bit? It hasn't been implemented yet as far as I know. Read the lkml thread for the exact proposal. Cheers, Tridge
Re: WineConf Agenda
On Mon, 2005-04-04 at 16:19 +1000, Andrew Tridgell wrote: Unfortunately it doesn't really help with case-insensitivity, as it is a non-blocking async interface, so the listeners get told way too late. Plus you have to have every directory open that you want to monitor, which would get silly. I think inotify does not require you to have an fd open for each directory, that was the motivation behind its design: GNOME needed a way to monitor removable media without blocking its unmount. The rest of your points about maintaining a userspace dcache seem to make sense, though maybe trading memory for syscalls in the meantime would be worth doing (until the kernel provides more support). thanks -mike
Re: WineConf Agenda
Brian, I think a full agenda would be about 11 items. If you would like to present something let me know - there's definitely space available. I'd like to present something on the way Samba4 stores the extra NTFS meta-data in posix filesystems (streams, NT ACLs, DOS attributes, extra timestamps etc) and open up a discussion on whether we could come to an agreement with the wine project on how this should be done. I can provide a demo of the implementation we currently have in the posix NTVFS backend in Samba4. I'd similarly like to discuss cooperating on share modes, oplocks and byte range locks. In each case I will be trying to encourage methods which can store the full NTFS semantics, rather than limiting ourselves to only the things that fit natually in posix filesystems. I'm guessing we will have some lively discussion on whether this is a worthwhile aim :-) Finally, if we have time, I'd like to discuss cooperation on IDL files and MSRPC interfaces. btw, I'm arriving in Stuttgart early on the 28th, so if anyone from the wine or Samba community wants to meet up early then please let me know. I'm sure we could find a corner where we can chat about windows interoperability. Cheers, Tridge
Re: WineConf Agenda
On Sun, Apr 03, 2005 at 08:57:58PM +1000, Andrew Tridgell wrote: In each case I will be trying to encourage methods which can store the full NTFS semantics, rather than limiting ourselves to only the things that fit natually in posix filesystems. I'm guessing we will have some lively discussion on whether this is a worthwhile aim :-) Speaking of which, both Samba and Wine suffer from the lack of support of cases-insensitivness in the kernel. Now, I'm not suggesting that we should push for that (I'm aware of the past discussions on LKML, and I must agree with Linus), but we should push for *something* that's acceptable and helps us solve the negative lookups a bit faster than a full directory listing every time. -- Dimi.
Re: WineConf Agenda
Dimitrie O. Paun wrote: On Sun, Apr 03, 2005 at 08:57:58PM +1000, Andrew Tridgell wrote: In each case I will be trying to encourage methods which can store the full NTFS semantics, rather than limiting ourselves to only the things that fit natually in posix filesystems. I'm guessing we will have some lively discussion on whether this is a worthwhile aim :-) Speaking of which, both Samba and Wine suffer from the lack of support of cases-insensitivness in the kernel. Now, I'm not suggesting that we should push for that (I'm aware of the past discussions on LKML, and I must agree with Linus), but we should push for *something* that's acceptable and helps us solve the negative lookups a bit faster than a full directory listing every time. For the curious, the best link to the past discussion I could find is http://marc.theaimsgroup.com/?l=linux-kernelm=107700108430044w=2 - Dan -- Trying to get a job as a c++ developer? See http://kegel.com/academy/getting-hired.html
Re: WineConf Agenda
On Mon, Apr 04, 2005 at 11:08:42AM +1000, Andrew Tridgell wrote: I have a proposed solution for that which needs no kernel modifications. I have been meaning to write this up properly, so if you like I can do that for WineConf. I think it would be very good to have such a discution. It seems to me this is one performance hotpoint for both projects. - for mixed directories, Win32 names will appear lowercase to posix programs, but will be case preserving to Wine/Samba. Wine/Samba This is something I don't really like: to change the filename's case behind user's back. From my POV, it's not acceptable. I know I'd be pissed if that happened to me. I think we'd be much better off if we relied on some kernel feature. The current mechanism works just fine, if you want the same behaviour, but faster, you need such and such in the kernel. It's a much simpler (and cleaner) case to make then to change such fundamental behaviour. -- Dimi.
Re: WineConf Agenda
On Mon, Apr 04, 2005 at 11:51:02AM +1000, Andrew Tridgell wrote: It only changes the name from the point of view of posix. The name is completely preserved from the point of view of Wine/Samba. I can still understand this annoying some people, but I don't think it would annoy the majority of Samba users. I understand that, but I'd certainly fall into the minority of Samba users then. I share my /home/dimi directory via Samba to my laptop, there's no way I want my filenames modified by the system. In Wine (and I would have thought in Samba too) we try to integrate as much as possible the two (Win32 and POSIX) environments. A solution that looks good only from a Wine/Samba standpoint fails on this criteria. Obviously, we want it to look good from both perspectives simultaneously. yes, but you will run up against huge objections from the kernel community. The key sticking point is that filenames can no longer be treated as bags-of-bytes. Right. And I can understand why. But I was hoping we can get some form of support from the kernel along the lines of the bit that Linus suggested that would allow us to efficiently do the case insensitivity in user space. I think that the problem boils down to being able to maintain the state of directories in userspace in a coherent manner. If we don't get support from that in the kernel, it naturally degenerates to reading the directory every single time. If we do get support however, we shoudln't need to reread most of the time, which would be good enough. I forget now the outcome of that discussion, whatever happened to Linus' proposal for that bit? -- Dimi.
Re: WineConf Agenda
Brian Vincent wrote: I think a full agenda would be about 11 items. If you would like to present something let me know - there's definitely space available. If you've been wondering, There's a neat project I'd like to do if I could only get a few more people interested, well, this is the perfect time to bring it up. If there's something strategic to Wine development, this is a great time to discuss it. I don't know if I can find the time to prepare a proper presentation, but if I do, I'd like talk about and get some input from you on the subject of testing. 1) I want the conformance tests done faster. (I have a clear idea how.) 2) What's almost never been brought up on wine-devel is the unit testing in the XP sense. 1 has been discussed on wine-devel quite a bit, so I'll skip it in this mail. regarding 2: CXTest is like application testing. This is good, high-level testing. The conformance tests test the Windows API. But in the case of a complicated API (and Windows is complicated) maybe we need to test the smaller units behind the API. So, the conformance tests often are somewhere between application testing and unit testing. Can we have an infrastructure for running unit tests of small units of code private to the implementation? I.e. these tests, we could call them private tests, would not run on Windows, but during build of Wine. Would it be acceptable to just include extra test code in the implementation c file, surrounded by #ifdef COMPILE_PRIVATE_TESTS ? regards, Jakob
RE: WineConf Agenda
Title: RE: WineConf Agenda In presenting the commercial value of wine and why we chose to use it, etc. How much time do I have for the presentation? -Original Message- From: Jakob Eriksson [mailto:[EMAIL PROTECTED]] Sent: Thursday, March 31, 2005 11:20 PM To: [EMAIL PROTECTED] Cc: Wine Devel; [EMAIL PROTECTED] Subject: Re: WineConf Agenda Brian Vincent wrote: I think a full agenda would be about 11 items. If you would like to present something let me know - there's definitely space available. If you've been wondering, There's a neat project I'd like to do if I could only get a few more people interested, well, this is the perfect time to bring it up. If there's something strategic to Wine development, this is a great time to discuss it. I don't know if I can find the time to prepare a proper presentation, but if I do, I'd like talk about and get some input from you on the subject of testing. 1) I want the conformance tests done faster. (I have a clear idea how.) 2) What's almost never been brought up on wine-devel is the unit testing in the XP sense. 1 has been discussed on wine-devel quite a bit, so I'll skip it in this mail. regarding 2: CXTest is like application testing. This is good, high-level testing. The conformance tests test the Windows API. But in the case of a complicated API (and Windows is complicated) maybe we need to test the smaller units behind the API. So, the conformance tests often are somewhere between application testing and unit testing. Can we have an infrastructure for running unit tests of small units of code private to the implementation? I.e. these tests, we could call them private tests, would not run on Windows, but during build of Wine. Would it be acceptable to just include extra test code in the implementation c file, surrounded by #ifdef COMPILE_PRIVATE_TESTS ? regards, Jakob
Re: WineConf Agenda
On Tue, Mar 29, 2005 at 10:31:33PM -0700, Brian Vincent wrote: I'm going to start this off with a huge apology - there's a very real chance I've asked someone to do a presentation and it's not on this list. If so, you need to email me to remind me. If you were definitely planning on presenting something because I emailed you, let me know - there's lots of space available. I know this only affects 2 or 3 people, but I tossed around a few ideas and didn't want to leave any loose ends. (Any of those items would make nice topics.) This is the initial agenda for WineConf. I hope to add a few more items, but Jeremy suggested I pull the trigger and announce it. I agree - there's enough here to fill two days and leave a lot of time for people to get together in small groups. Only the first two items are in any formal order: Alexandre - Keynote (He said he's reusing last year's slides.) Dimi - to do list / status update Steven Edwards and the ReactOS crew Charles W Stevenson - Winelib / Winelib porting / and a commercial perspective Andrew Barlett / Samba team? - Samba 4, authentication, lots of intelligent ideas Juan Lang - Wine's RPC + Samba or something like that :) Jeremy White - CXTest Jason Edmeades - DirectX 9 / wined3d / eye candy I think a full agenda would be about 11 items. If you would like to present something let me know - there's definitely space available. I could give an introduction into reverse engineering, including an IDA Pro introduction (if there is interest.) Ciao, Marcus
Re: WineConf Agenda
Hi, On Fri, 2005-04-01 at 02:20, Jakob Eriksson wrote: 2) What's almost never been brought up on wine-devel is the unit testing in the XP sense. The problem is you need to write a stub interface for all of the functions the function or library you are testing due to all the cross-calls win32 makes. You end up spending a huge ammount of time just writting stubs. In the ReactOS CIS system Casper has implemented support for this and while its a good idea it a rather large beast. I think he developed a system to automagicly generate the stubs needed but then you end up having two sets of tests to test the same functions. Not to mention I don't see how it can work on Linux anyway. Thanks Steven
WineConf Agenda
I'm going to start this off with a huge apology - there's a very real chance I've asked someone to do a presentation and it's not on this list. If so, you need to email me to remind me. If you were definitely planning on presenting something because I emailed you, let me know - there's lots of space available. I know this only affects 2 or 3 people, but I tossed around a few ideas and didn't want to leave any loose ends. (Any of those items would make nice topics.) This is the initial agenda for WineConf. I hope to add a few more items, but Jeremy suggested I pull the trigger and announce it. I agree - there's enough here to fill two days and leave a lot of time for people to get together in small groups. Only the first two items are in any formal order: Alexandre - Keynote (He said he's reusing last year's slides.) Dimi - to do list / status update Steven Edwards and the ReactOS crew Charles W Stevenson - Winelib / Winelib porting / and a commercial perspective Andrew Barlett / Samba team? - Samba 4, authentication, lots of intelligent ideas Juan Lang - Wine's RPC + Samba or something like that :) Jeremy White - CXTest Jason Edmeades - DirectX 9 / wined3d / eye candy I think a full agenda would be about 11 items. If you would like to present something let me know - there's definitely space available. If you've been wondering, There's a neat project I'd like to do if I could only get a few more people interested, well, this is the perfect time to bring it up. If there's something strategic to Wine development, this is a great time to discuss it. For example, we've had mixed feelings regarding revision control and its been suggested to bring it up as a topic. Would anyone find it useful to discuss subversion vs. arch vs. cvs? We could probably get Mike Hearn to show off how he manages patches with SVK and then let him arm wrestle Alexandre about whether arch is better. Friday night we will have an informal dinner after meeting up at the hotel. The conference will kick off Saturday morning on campus around 8am. Lunch will be provided on Saturday and Sunday as well as dinner on Saturday. There will definitely be a lot of time for informal discussion. For the Samba guys - I think there's a great opportunity to share some knowledge to prevent anyone from reinventing the wheel. If there's any topic you'd find interesting, let me know. If you'd just like to hear a specific topic from the Wine folks, we can arrange that. For travel arrangements, please join [EMAIL PROTECTED] ( http://www.winehq.org/mailman/listinfo/wineconf ) For instance, I'm actually flying into Frankfurt on April 23rd and it'd be cool to hang out with anyone arriving early. If you haven't made reservations at a hotel, don't worry too much. There's still rooms available as Mr. Schmid described here: http://www.winehq.com/hypermail/wine-devel/2005/03/1119.html If you haven't signed up on the RSVP list, please do so. It can be found on the WineConf page: http://www.winehq.com/site/wineconf Lastly, if you have any questions / comments / criticisms let me know. -Brian PS. Dimi owes me a beer from last year, so we're all obligated to go out Friday night. I'm looking forward to seeing everyone in Stuttgart!
Wineconf Agenda
Brian Vincent wrote: I PS - still looking for ideas for the agenda, if you have any, let me know. Also let me know if you'd like to present something. Apart from all of Wine, I'm always interested in the conformance testing. I believe it's crucial in speeding up Wines' development. For each bug found, it is often a good idea to write an automatic regression test. Maybe we can extend on the winetest.exe infrastructure. I would like to see testing done in realtime - we now have a turnaround of maybe 3-4 days for a test developer like myself. I do a tiny, tiny patch, and then wait 3-4 days to see how it behaves on 6 platforms. What I would like to see: a cluster of Windows machines with different versions, all waiting eagerly for a small test.exe to appear. A dispatcher receives tests and then distributes it to waiting Windows machines. As soon as machine has run a test, it is rebooted. Preferrably from a clean vmware image. Mockup example: [EMAIL PROTECTED]:~/src/wine/dlls/kernel/tests$ make testgrid_file i586-mingw32msvc-gcc -c -I. -I. -I../../../include -I../../../include -D_REENTRANT -fPIC -Wall -pipe -mpreferred-stack-boundary=2 -fno-strict-aliasing -gstabs+ -Wpointer-arith -g -O2 -o file.cross.o file.c file.c:1: warning: -fPIC ignored for target (all code is position independent) i586-mingw32msvc-gcc alloc.cross.o atom.cross.o change.cross.o codepage.cross.o comm.cross.o console.cross.o directory.cross.o drive.cross.o environ.cross.o file.cross.o format_msg.cross.o generated.cross.o heap.cross.o locale.cross.o module.cross.o mailslot.cross.o path.cross.o pipe.cross.o process.cross.o profile.cross.o thread.cross.o time.cross.o timer.cross.o virtual.cross.o testlist.cross.o -o kernel32_crosstest.exe -lkernel32 i586-mingw32msvc-strip kernel32_crosstest.exe upx kernel32_crosstest.exe gpg --sign kernel32_crosstest.exe kernel32_crosstest.sig tar -cf kernel32_crosstest.tar kernel32_crosstest.exe kernel32_crosstest.sig wget http://testgrid.winehq.org/gridtest.cgi --post-file=kernel32_crosstest.tar --output-document=testgrid_file.txt [EMAIL PROTECTED]:~/src/wine/dlls/kernel/tests$ [EMAIL PROTECTED]:~/src/wine/dlls/kernel/tests$ [EMAIL PROTECTED]:~/src/wine/dlls/kernel/tests$ cat testgrid_file.txt Windows 2000 begin: file.c:1384: Test failed: DeleteFileA: error 3 file: 487732 tests executed, 0 marked as todo, 1 failure. End Windows 2000. Windows 98 begin: file.c:804: Test failed: MoveFileA: unexpected error 3 file.c:1384: Test failed: DeleteFileA: error 3 file: 487732 tests executed, 0 marked as todo, 2 failures. End Windows 98. Windows XP begin: file: 487732 tests executed, 0 marked as todo, 0 failures. End Windows XP. Windows NT4 begin: file: 487732 tests executed, 0 marked as todo, 0 failures. End Windows NT4. [EMAIL PROTECTED]:~/src/wine/dlls/kernel/tests$ [EMAIL PROTECTED]:~/src/wine/dlls/kernel/tests$ [EMAIL PROTECTED]:~/src/wine/dlls/kernel/tests$ regards, Jakob
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
Paul Millar wrote: On Thursday 17 March 2005 10:33, Jakob Eriksson wrote: Apart from all of Wine, I'm always interested in the conformance testing. I believe it's crucial in speeding up Wines' development. For each bug found, it is often a good idea to write an automatic regression test. Yes, although that's a rather weary job, which no one enjoys doing :^/ I do! Maybe I have a condition, but I really love doing it! :-) There's three issues here: Yes. o fast(er) update of CVS (or whatever filestore we're using). This would need either: * a super-charged Alexandre ;) or * a separate CVS tree in which developers can edit the wine/dlls/*/tests/ directory, This makes sense. Winetest could actually be a separate project. I'm not saying it should be, mind you, just that conformance testing is interesting not only for Wine, but for Windows developers in general. So a separate CVS tree makes a great deal of sense, IMHO. Wine could import snapshots of the tests for its' conformance testing. This could speed up test development considerably, I imagine. or * give up on centralised/distributed testing architecture and switch to a personal testing environment. o fast(er) build of winetest.exe I originally argued for async. winetests and went as far as implementing this as part of WRT, so in principles this is already done. WRT worked based on the email notification of CVS updates. Builds, with minor changes, doesn't take long (using ccache), so you're probably looking 10-20 minutes turn around, with whatever delay the test-clients introduce. Though, without fixing the first issue, this doesn't help us much. o fast(er) running, through vmware platforms. Sure, this can be done, but its a distributed model, so everyone can chip in. Shouldn't be too difficult to achieve this. Sounds like fun, doesn't it? Test servers could register in the cluster worldwide. (Although I originally imagined a very centralized solution with a big server running vmware images.) Just my 2-c worth :) Are you coming to wineconf? regards, Jakob
wineconf agenda
Brian Vincent wrote: PS - still looking for ideas for the agenda, if you have any, let me know. Also let me know if you'd like to present something. Will anyone demo CXtest? I think it would be great. http://www.cxtest.org/ regards, Jakob
Enhancing winetest infrastructure [WAS: Wineconf agenda]
On Thursday 17 March 2005 10:33, Jakob Eriksson wrote: Apart from all of Wine, I'm always interested in the conformance testing. I believe it's crucial in speeding up Wines' development. For each bug found, it is often a good idea to write an automatic regression test. Yes, although that's a rather weary job, which no one enjoys doing :^/ Maybe we can extend on the winetest.exe infrastructure. I would like to see testing done in realtime [...] There's three issues here: o fast(er) update of CVS (or whatever filestore we're using). This would need either: * a super-charged Alexandre ;) or * a separate CVS tree in which developers can edit the wine/dlls/*/tests/ directory, or * give up on centralised/distributed testing architecture and switch to a personal testing environment. o fast(er) build of winetest.exe I originally argued for async. winetests and went as far as implementing this as part of WRT, so in principles this is already done. WRT worked based on the email notification of CVS updates. Builds, with minor changes, doesn't take long (using ccache), so you're probably looking 10-20 minutes turn around, with whatever delay the test-clients introduce. Though, without fixing the first issue, this doesn't help us much. o fast(er) running, through vmware platforms. Sure, this can be done, but its a distributed model, so everyone can chip in. Shouldn't be too difficult to achieve this. Just my 2-c worth :) Paul. pgpaeq8nAuEBd.pgp Description: PGP signature
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
Jakob Eriksson wrote: Maybe one could script something up so that a commit to tests CVS would not be *possible* without a confirmed test pass on Windows 95, NT, 2000 and XP. That would be very hard to do and mostly pointless for drivers. Ivan.
Re: wineconf agenda
Will anyone demo CXtest? I think it would be great. http://www.cxtest.org/ I think that's one of those topics that Brian is holding in reserve; we'll be more than happy to demo it. Jer
Re: wineconf agenda
Hi, On Thu, Mar 17, 2005 at 08:35:25AM -0600, Jeremy White wrote: Will anyone demo CXtest? I think it would be great. http://www.cxtest.org/ I think that's one of those topics that Brian is holding in reserve; we'll be more than happy to demo it. Nice stuff! Will you have decided by then whether to call it CXtest, Cxtest, CxTest or CXTEST? ;-) (YES, I've seen all of those!) Andreas
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
On Thursday 17 March 2005 11:54, Jakob Eriksson wrote: Maybe one could script something up so that a commit to tests CVS would not be *possible* without a confirmed test pass on Windows 95, NT, 2000 and XP. That'd be the best thing since sliced bread. CVS supports doing some server-side validation tests before accepting a commit, so it could be done; but I don't think this would be nice. CVS is a mechanism for sharing code: its not really a testing framework. There would be potential issues with CVS locking, timeouts (e.g. what happens if the XP machine dies?), but the real issue is it would just gets in the way of developers by making the CVS server fragile. For the testing framework, I'd say what we have just now is fine. It lives outside (and on top of) CVS. Having broken tests is OK, provided they're fixed within a suitable time-scale [*]. (just my 2c-worth again) Cheers, Paul [*] -- Of course, what is the suitable time-scale? who's willing to make sure things get fixed? pgpaMsk6vte8H.pgp Description: PGP signature
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
Ivan Leo Puoti wrote: Jakob Eriksson wrote: Maybe one could script something up so that a commit to tests CVS would not be *possible* without a confirmed test pass on Windows 95, NT, 2000 and XP. That would be very hard to do and mostly pointless for drivers. Sometimes hard is worthwile. At least to me. regards, Jakob
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
On Thursday 17 March 2005 11:51, Jakob Eriksson wrote: Paul Millar wrote: [writing tests]'s a rather weary job, which no one enjoys doing :^/ I do! Maybe I have a condition, but I really love doing it! :-) Long may that continue! [...] [separate CVS tree] This makes sense. Winetest could actually be a separate project. I'm not saying it should be, mind you, just that conformance testing is interesting not only for Wine, but for Windows developers in general. So a separate CVS tree makes a great deal of sense, IMHO. Wine could import snapshots of the tests for its' conformance testing. This could speed up test development considerably, I imagine. I'd tread careful here: this isn't a panacea. Yes, its fairly easy to *say* OK, we just merge in the new tests, but would merging the snapshots really be a trivial job? If it is non-trivial, then Alexandre would front at least some of it and I suspect making more work for Alexandre is the last thing anyone (including Alexandre :^) would want. On top of that, there would be many other issues, e.g. How the diffs between the two trees were presented? How the changes in wine would be merged back into the winetest tree? What happens to patches that Alexandre rejects (coz they're just wrong)? Ultimately, its Alexandre's call (and I think I can guess his answer). [...] Are you coming to wineconf? All things being equal, yes. Cheers, Paul. pgpJPmyTe9YX6.pgp Description: PGP signature
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
Paul Millar wrote: On Thursday 17 March 2005 11:51, Jakob Eriksson wrote: Paul Millar wrote: [writing tests]'s a rather weary job, which no one enjoys doing :^/ I do! Maybe I have a condition, but I really love doing it! :-) Long may that continue! Thanks, I hope so too! :-) Only my day job and things like planning a marriage stops me from digging deep into Wine testing. [..] I'd tread careful here: this isn't a panacea. Yes, its fairly easy to *say* OK, we just merge in the new tests, but would merging the snapshots really be a trivial job? If it is non-trivial, then Alexandre would front at least some of it and I suspect making more work for Alexandre is the last thing anyone (including Alexandre :^) would want. Ouch. Of course, should have thought of that. Maybe we need a patch penguin? regards, Jakob
Re: wineconf agenda
A bit OT, well kinda. I'm trying to stabilize offscreen rendering in DirectX 9, I know Jason was thinking of doing a demo. Is there going to be another wine release before wine conf so I can merge DirectX and do you still plan to do a demo Jason, if so what are you time-lines for something stable? for anyone who's interested Offscreen rendering is hitting state blocks quite a bit, since stateblocks is next on my list of patches to merge with wine I thought it would be a good idea to do some work upfront instead of sending in lots of patches that change things around a lot. ICQ 169222899 MSN [EMAIL PROTECTED] pgp0l8VqiuhSL.pgp Description: PGP signature
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
On Thu, 17 Mar 2005, Jakob Eriksson wrote: Ouch. Of course, should have thought of that. Maybe we need a patch penguin? CVS actually handles 'vendor branches' fairly nicely. It should be possible for Jakob (or whoever else decides to be the 'test penguin') to maintain his own 'testing' CVS tree, with rapid development, etc, and periodically resync his own tree to canonical Wine CVS (using the vendor branch functionality) and then create large-ish 'tests-only' chunks from that to throw at Alexandre once in a while. There really shouldn't be much reason for Alexandre to reject patches that touch tests only; after all, if the tests pass on windows, they should pass on wine, no matter how evil they look. (Well, within reason.) That's MHO, at least. If I understand correctly, the primary reason for the 'testing' CVS is just to manage distribution of proposed tests to a server farm of test-runners; which is slightly different from the purpose of the mainline CVS tree. [Also -- a decoupled 'testing' CVS like I describe above can be implemented by the motivated folks w/o Alexandre's involvement at all, which permits judgements to be postponed until we've got some evidence of usefulness.] In this vein -- where *is* the current testing infrastructre located? I'm pretty new to Wine, and I couldn't find any links from winehq. [These should probably be added, or made more visible if they do exist, especially if the goal is to encourage test submission with patch submission.] --scott LIONIZER class struggle SCRANGER ESGAIN overthrow GRALLSPICE UKUSA tonight EZLN Sudan NSA KMFLUSH PBPRIME mail drop Leitrim IDEA AELADLE ( http://cscott.net/ )
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
Paul Millar wrote: On Thursday 17 March 2005 11:54, Jakob Eriksson wrote: Maybe one could script something up so that a commit to tests CVS would not be *possible* without a confirmed test pass on Windows 95, NT, 2000 and XP. That'd be the best thing since sliced bread. CVS supports doing some server-side validation tests before accepting a commit, so it could be done; but I don't think this would be nice. CVS is a mechanism for sharing code: its not really a testing framework. I didn't mean exactly on the CVS level. When Alexandre commits any patch, he first checks that the code still passes regression tests. I want something similar for the test patches. Maybe like this: the testing dispatcher signs a working patch* with GPG. (Or no GPG. Just set a flag somewhere. Details are not important.) Alexandre will see this flag when saving a patch from [EMAIL PROTECTED] and know that the patch is OK as far as the test grid is concerned. It could be as non-intrusive as this: the test dispatcher monitors the [EMAIL PROTECTED] for patches. As soon as it sees a patch it recognises and knows it has tested, it sends a mail to wine-patches akin to: The patch with CRC32 so-and-so posted by him or her, named so-and-so is hereby verified by me, the Wine Regression Grid Tester. ** or The patch with CRC32 so-and-so posted by him or her, named so-and-so failed under the following versions of Windows; bla bla blah, with the following error message: blah blah bla some more. Truthfully yours, the Wine Regression Grid Tester. ** Then it's up to Alexandre if he wants to commit a test which the grid tester has rejected, or for which there is no confirmation. If you don't like the idea of a program spamming wine-patches, it could be separate list, or a webpage with a copy of wine-patches, with different messages colour-codes updated as they get tested by the grid tester. * A working patch is a patch that has been tested and found working on Win 95, 98, ME, NT4, 2000 and XP. ** Could we call it WineGrind? :-) For the testing framework, I'd say what we have just now is fine. It lives outside (and on top of) CVS. Having broken tests is OK, provided they're fixed within a suitable time-scale [*]. Actually, I think having broken tests is not OK. It not only goes against my zealotry for Extreme Programming, it's also very annoying when I have _no_clue_ how to fix a broken test and the author is missing or don't want to touch the code with a ten foot stick. (just my 2c-worth again) Cheers, Paul [*] -- Of course, what is the suitable time-scale? who's willing to make sure things get fixed? Exactly, I feel rage everytime I see those red and yellow boxes at http://test.winehq.org/data/ ;-) regards, Jakob
Re: Enhancing winetest infrastructure [WAS: Wineconf agenda]
C. Scott Ananian wrote: On Thu, 17 Mar 2005, Jakob Eriksson wrote: Ouch. Of course, should have thought of that. Maybe we need a patch penguin? CVS actually handles 'vendor branches' fairly nicely. It should be possible for Jakob (or whoever else decides to be the 'test penguin') to maintain his own 'testing' CVS tree, with rapid development, etc, and periodically resync his own tree to canonical Wine CVS (using the vendor branch functionality) and then create large-ish 'tests-only' chunks from that to throw at Alexandre once in a while. There really shouldn't be much reason for Alexandre to reject patches that touch tests only; after all, if the tests pass on windows, they should pass on wine, no matter how evil they look. (Well, within reason.) That's MHO, at least. If I understand correctly, the primary reason for the 'testing' CVS is just to manage distribution of proposed tests to a server farm of test-runners; which is slightly different from the purpose of the mainline CVS tree. [Also -- a decoupled 'testing' CVS like I describe above can be implemented by the motivated folks w/o Alexandre's involvement at all, which permits judgements to be postponed until we've got some evidence of usefulness.] Yes. And I think I can implement most of even the more elaborate schemes without initially disturbing Alexandre or anyone else. As you say, until we get more evidence of usefulness. In this vein -- where *is* the current testing infrastructre located? I'm pretty new to Wine, and I couldn't find any links from winehq. [These should probably be added, or made more visible if they do exist, especially if the goal is to encourage test submission with patch submission.] http://test.winehq.org/data/ regards, Jakob
WineConf agenda ideas
Over the the past week or so I've harassed some folks about the agenda for WineConf. At this point there's a lot that remains to be filled in, but I need some feedback. Also, I need more ideas for presentations. Does anyone want to present something? It'd be great if someone would volunteer. I think what we're shooting for is a presentation followed by discussion all contained within about one hour. In total we'll have 10 - 12 different topics. Here's the items that are definitely going to be on the agenda: WRS intro Keynote - Alexandre Road to .9 and 1.0 - Dimi ReactOS update - all the ReactOS guys Wine / networking / Samba issues - Juan Lang We've also got some other items, but I need feedback as to whether any of you find this stuff interesting. (So reply and tell me if any of these items sound interesting.) Windows authentication - Andrew Bartlett (um.. haven't asked him yet though) Something else Samba related? - Samba guys Commercial port of a Winelib app and using Wine - an enterprise user CXTest and regression testing - someone from CodeWeavers Version control - Mike Hearn wearing a fire retardant suit DCOM work - Mike or Rob DirectX update - ? Also, if you need to make hotel reservations, there's just a few weeks left to do that. Check out the WineConf page for details: http://www.winehq.org/site/wineconf As people begin to figure out travel plans, the wineconf mailing list will pick up. How to join: http://www.winehq.org/mailman/listinfo/wineconf Let me know if you have any questions. -Brian
