Re: [ANN] TransactionKit, Lockless Multi-Reader, Multi-Writer Transaction Capable Hash Tables

2008-04-23 Thread Daniel DeCovnick


On Apr 22, 2008, at 7:48 PM, John Engelhart wrote:






*applause*
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to [EMAIL PROTECTED]


[ANN] TransactionKit, Lockless Multi-Reader, Multi-Writer Transaction Capable Hash Tables

2008-04-22 Thread John Engelhart

All,

I've recently released the first version of TransactionKit, which is  
made of two main components: The core library, and a Foundation  
compatibility API layer.


Homepage: http://transactionkit.sourceforge.net/
Documentation: http://transactionkit.sourceforge.net/Documentation/index.html
SVN trunk is available at: 
http://transactionkit.svn.sourceforge.net/svnroot/transactionkit/trunk

TransactionKit is made available under a 3-clause BSD License.

As an aside, and begging the karma gods for forgiveness, I'm currently  
looking for a job.  I obviously would like to find work doing Cocoa  
programming, but I also have extensive experience in networking (ala  
sr. backbone engineer at a top tier 1 provider).  Physically in the  
greater Toronto (CA) metro area, US citizen, legal to work in both CA  
and US.  Working remotely has a certain appeal. :)


The core library is mostly C based, with a bit of Objective-C in the  
appropriate places for Objective-C compatibility.  The Objective-C  
parts in the core library are optional, and can be easily stripped  
away leaving just a C based core.


The Foundation API compatibility layer replicates the C based (not the  
newer 10.5 Objective-C based) NSHashTable and NSMapTable, along with a  
subclass / re-implementation of NSDictionary and NSMutableDictionary.


The development environment has been Mac OS X 10.5 on a G4 PPC system,  
though I expect it should work effortlessly on any 10.5 system.  The  
only thing I can think of off the top of my head that would prevent  
10.4 use is the fact that the Dictionary clones implement  
NSFastEnumeration, other than that I can't think of any 10.5 specific  
features it makes use of.  10.5 Garbage Collection is not supported  
nor are their plans to- I have simply not been able to get 10.5's GC  
system to work reliably under the grueling punishment of the synthetic  
stress tests that TransactionKit places on the proper accounting of  
resources by the memory allocation system.  This has not been from a  
lack of trying, I've just found the 10.5 GC system to be buggy and  
unreliable.  Examples include: compiler bugs that don't always insert  
write barriers (ala a typdefed struct assignment, such as  
MyExampleType = *initExampleType), or that the functions  
objc_atomicCompareAndSwapGlobalBarrier (and friends) were essentially  
no-ops until 10.5.2 (specifically, objc4-371.1).  Personally, I've  
found the 10.5 GC system extremely non-intuitive and ultimately  
impossible to correctly use in practice.  I think the section on "The  
Costs of Precise Pointer Identification" in http://www.hpl.hp.com/personal/Hans_Boehm/gc/conservative.html 
 summarize many of the objections and problems I've encountered.


Full disclosure: If you're not familiar with the implications of  
"lockless" and "multithreading" combined in the same sentence, you  
should know that this library attempts to deal with some notoriously  
difficult to get right problems.  The common methodology in  
multithreading programming is to employ a mutual exclusion lock around  
a data structure to prevent simultaneous modifications by multiple  
threads at the same time.  TransactionKit uses a novel, experimental  
means to allow lockless modifications by multiple threads  
concurrently.  While some concurrent multiple writer data structures  
do exist, most are generally academic research grade problems /  
solutions.  TransactionKit is definitely a prototype and experimental  
in nature at this time.


- There are almost certainly bugs. -  While certainly unintentional, I  
think its proper to set your expectations now.  I think for most  
'simple' things, things should be mostly bug free.  Corner cases and  
the complicated nuances that crop up during multithreading concurrent  
use is where most of the bugs probably are, especially in dealing with  
the concept of "when" things happens relative to transaction start,  
commit, and rollback times.


While the ultimate goal is to have a highly portable C "core" and an  
Objective-C layer built on top of that, for right now it realistically  
only works on Mac OS X.  This is probably OK considering the  
audience :).  At this point, the C library is largely undocumented,  
but it really isn't that complicated: create a table, free a table,  
insert, get, and remove a key, along with begin, commit, and rollback  
a transaction.  Thats about it, really, and won't be further covered  
here.  The rest of this is regarding the Objective-C portion.


The Objective-C portion, specifically the Dictionary clones, are  
essentially straight, method for method, re-implementations of their  
foundation Dictionary counterparts.  There are two classes,  
TKDictionary and TKMutableDictionary. TKDictionary is a subclass of  
NSDictionary, and TKMutableDictionary is a subclass of TKDictionary.   
They "should" be drop in replacements for their Foundation  
equivalents, and interoperate invisibly with each other.  Wh