The original idea of the i86 segmented architecture was that OSs would 
be written that created zillions of bite-sized threads, each with its 
own small address space. Nobody wrote an OS to take advantage of that, 
and instead the ridiculous "far pointer" was invented to desegment the 
deliberately segmented address space. But some of the original 
threads-are-good/processes-are-bad thinking still remained in the OS.

----- Original Message ----- 
From: "Beau Wilkinson" <b...@mtllc.us>
To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org>
Sent: Tuesday, September 22, 2009 11:41 AM
Subject: Re: [sqlite] cygwin and sqlite


> >The point I was trying to make is that the majority of commands 
> >cygwin
>>supports don't have to do any forking. They could just be method
>>calls. Pipes would be parameters going in and return values coming
>>out, all running in a single process. If fork() is specifically 
>>called
>>in a script, then the slower approach would have to be used, but 
>>most
>>of the time it would not.
>
> Yes, in theory that's true, but Cygwin strives to be a lightweight 
> connecting layer between the Unix-like facilities of Windows and the 
> wealth of Unix code available. Wherever these "Unix-like" features 
> turn out to not actually not be so "Unix-like," the model breaks 
> down (e.g. fork()), and Cygwin has to be larger and more intrusive. 
> There are many other architectures in use out there. The many 
> Unix-like tools that run natively on Windows are, in some sense, an 
> example of another POSIX / Windows hybridization archietcture. But 
> this is not the architecture of Cygwin.
>
> Incidentally, I think the fundamental problem is the asinine way 
> Windows treats processes - vs. - threads. Microsoft seems to 
> encourage multithreading, not just by making processes relatively 
> heavy, but also in their documentation and examples and in the 
> overall direction of their R&D efforts.
>
> My opinion is that processes should be lightweight, because they 
> offer a much more predictable and useful way to effect parallel 
> processing than threads. I always try to use processes over threads 
> when I detect the need for parallelism, even in Windows. Other 
> Windows developers consider my process-spawning to be awkward, or 
> even "a hack," but as someone with Unix experience this seems 
> natural, and the end results are typically more deterministic.
>
> ----- Original Message -----
> From: "Pavel Ivanov" <paiva...@gmail.com>
> To: "General Discussion of SQLite Database" 
> <sqlite-users@sqlite.org>
> Sent: Tuesday, September 22, 2009 6:58 AM
> Subject: Re: [sqlite] cygwin and sqlite
>
>
>>> But after all these years I wonder why they don't fix the fork
>>> problem?
>>> MacOS runs on Intel processors. Windows runs on Intel processors.
>>> Surely
>>> they could learn how it *should* be done by studying things like
>>> the
>>> Open Source Java code?
>>
>> You seem to forget the basics. It's not the processor who makes
>> fork()
>> possible, it's OS. Unix kernel implemented fork() and Unix kernel
>> implemented process management in the way that makes implementation
>> of
>> fork() very quick and easy. Windows didn't implement fork() and it
>> implemented process management in the way making fork() impossible.
>> I'd say it's a superior achievement on cygwin side that they were
>> able
>> to implement fork() somehow at all. Just a simple fact: you execute
>> some code that uses memory in some way then you call fork() on Unix
>> and you already have 2 absolutely different processes that can
>> access
>> the same data in memory. On windows there's no way to start a new
>> process so that it can access the same data as first process unless
>> you thought about that beforehand, placed all your data into shared
>> memory (which is a lot harder to work with, btw) and made another
>> process to read the same shared memory. Also there's no way to 
>> start
>> new process on Windows so that it executes the same code as first
>> process from the point where second process was started...
>> So I'd better not complained but tried to understand the roots of
>> the problem...
>>
>> And if you want the comparable speed of scripts on both platforms I
>> suggest you to look into perl (or python, whatever you prefer). The
>> both have native implementations and their speed should be
>> comparable
>> to each other. That said of course if you don't start a lot of
>> processes from scripts and don't try to run command line
>> utilities...
>>
>> Pavel
>>
>> On Mon, Sep 21, 2009 at 10:29 PM, John <jhy...@earthlink.net> 
>> wrote:
>>> Pavel Ivanov wrote:
>>>>> MacBook Mac OS X 10.5.8
>>>>> 2 GHz Intel Core Duo
>>>>> 1 GB memory:
>>>>> 17 minutes 46 seconds.
>>>>>
>>>>> IBM ThinkPad
>>>>> Windows XP (latest patches)
>>>>> 1.70 GHz, 512 MB memory:
>>>>> 6 hours 25 minutes 57 seconds
>>>>
>>>> Windows is very slow in starting new processes if compared to any
>>>> Unix
>>>> system (especially if compared Windows + 512 MB and Unix + 1 GB).
>>>> In
>>>> cygwin starting new processes even slower because for some reason
>>>> emulating fork() involves starting 2 processes one of which dies
>>>> immediately. And bash scripts use processes a lot especially with
>>>> Unix
>>>> paradigm when for each small action you start new program (like
>>>> sed,
>>>> awk, test, true and whole lot of others). Thus bash scripts on
>>>> cygwin
>>>> will be slow unavoidably.
>>>> But I'm digressing. This is subject for some other mailing list.
>>>> :)
>>>
>>> I'm blacklisted apparently on the cygwin mailing list for when a
>>> couple of years ago I complained rather unflatteringly about how
>>> slow
>>> it was/is when I was writing a simple expenses program (that works
>>> in
>>> seconds on my Mac). I forgot about that. A badge of honor in my
>>> opinion.
>>>
>>> But after all these years I wonder why they don't fix the fork
>>> problem?
>>> MacOS runs on Intel processors. Windows runs on Intel processors.
>>> Surely
>>> they could learn how it *should* be done by studying things like
>>> the
>>> Open Source Java code?
>>>
>>> It looks like I won't be able to distribute my stopgap cygwin code
>>> on
>>> Windows. I need to start speed reading my Java/Swing books I guess
>>> in
>>> my quest for my program to write once, run anywhere.
>>>
>>>> Pavel
>>>>
>>>> On Fri, Sep 18, 2009 at 3:26 AM, John <jhy...@earthlink.net>
>>>> wrote:
>>>>> Pavel Ivanov wrote:
>>>>>>> At least I think that is what you suggest, and think it just
>>>>>>> may work! But I could be wrong!
>>>>>> Yes, that's exactly what I suggest.
>>>>>>
>>>>>> Pavel
>>>>> It worked! Fortunately I had already parameterized SQLITE3 as a
>>>>> preference variable so I could have the same scripts run easily
>>>>> on Mac
>>>>> OS and Windows. There are dozens of sqlite3 calls throughout the
>>>>> scripts.
>>>>>
>>>>> My whole set of scripts that process raw data and load the
>>>>> database by
>>>>> reading text files seem to work.
>>>>>
>>>>> cygwin is as slow as I recall, however. I was writing expense
>>>>> scripts a
>>>>> few years ago and abandoned it for MacOS Unix. I moved 100% to
>>>>> Mac OS.
>>>>> (except for this project which I want to work on Mac, linux, and
>>>>> Windows; my next goal is recoding it in Java with its Swing GUI,
>>>>> but I'm
>>>>> just learning Java and Swing, but I'm on my way...).
>>>>>
>>>>> Observed elapsed times on my two notebook computers for the same
>>>>> scripts
>>>>> to load the database (using sqlite3 calls and lots of sed and 
>>>>> awk
>>>>> processing of thousands of lines of input data):
>>>>>
>>>>> MacBook Mac OS X 10.5.8
>>>>> 2 GHz Intel Core Duo
>>>>> 1 GB memory:
>>>>> 17 minutes 46 seconds.
>>>>>
>>>>> IBM ThinkPad
>>>>> Windows XP (latest patches)
>>>>> 1.70 GHz, 512 MB memory:
>>>>> 6 hours 25 minutes 57 seconds
>>>>>
>>>>> Fortunately, sqlite .dump and restoring from the resultant sql
>>>>> will be
>>>>> able to be used for most of the heavy lifting when I'm done.
>>>>> Changes to
>>>>> the data will come in small increments over time from then on. 
>>>>> My
>>>>> dumpit
>>>>> and restoreit scripts each take only seconds on both platforms
>>>>> for the
>>>>> full set of current data.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>> On Thu, Sep 17, 2009 at 1:18 PM, John <jhy...@earthlink.net>
>>>>>> wrote:
>>>>>>> Pavel Ivanov wrote:
>>>>>>>>> I'd rather avoid building sqlite3 under cygwin. I would like
>>>>>>>>> to keep as much as possible in native code, compromising 
>>>>>>>>> only
>>>>>>>>> on cygwin to run my scripts.
>>>>>>>> And this is root of your problem. Using mix of cygwin-native
>>>>>>>> applications with windows-native applications will always 
>>>>>>>> have
>>>>>>>> such
>>>>>>>> problem.
>>>>>>>>
>>>>>>>>> When installing cygwin, you it offers you the choice to
>>>>>>>>> switch
>>>>>>>>> to default text file type to DOS (\r\n). Should I try that?
>>>>>>>> Don't do that. This mode of operation is not supported much
>>>>>>>> and not
>>>>>>>> recommended by cygwin developers and it reportedly will
>>>>>>>> significantly
>>>>>>>> slow down cygwin's operation.
>>>>>>>>
>>>>>>>>> So I guess my question here is, do any sqlite users here
>>>>>>>>> have experience fixing this on Windows for Unix cygwin
>>>>>>>>> script calls?
>>>>>>>> The major suggestion here: write some "windows native code
>>>>>>>> launcher"
>>>>>>>> that will be used for running all non-cygwin applications
>>>>>>>> (this can be
>>>>>>>> just function in the script). It will do nothing on unix
>>>>>>>> platforms
>>>>>>>> (select your own preferred way of distinguishing it) and it
>>>>>>>> will
>>>>>>>> always strip off '\r' from output of running application on
>>>>>>>> windows
>>>>>>>> (you can use sed for that). And there's nothing else you can
>>>>>>>> do about
>>>>>>>> it.
>>>>>>> This sounds like a great idea. I can have all sqlite3.exe 
>>>>>>> calls
>>>>>>> "intercepted" by another script call like:
>>>>>>>
>>>>>>> NumPar=`WINDOWSCALL Program Arguments`
>>>>>>>
>>>>>>> WINDOWSCALL is the launcher that calls Program sqlite3.exe
>>>>>>> with its arguments and strips off any trailing \r's
>>>>>>> and returns that string to the caller through stdout,
>>>>>>> as to NumPar here. WINDOWSCALL can do nothing on Unix/MacOS,
>>>>>>> and fix the string on Windows.
>>>>>>>
>>>>>>> At least I think that is what you suggest, and think it just
>>>>>>> may work! But I could be wrong!
>>>>>>>
>>>>>>> Thanks! I'll try coding it.
>>>>>>>
>>>>>>>> Pavel
>>>>>>>>
>>>>>>>> On Thu, Sep 17, 2009 at 12:26 PM, John <jhy...@earthlink.net>
>>>>>>>> wrote:
>>>>>>>>> I am writing some Unix scripts on Mac OS X that use
>>>>>>>>> sqlite3. Since the program could be useful to those
>>>>>>>>> on Windows, I figured I'd see if they worked under
>>>>>>>>> cygwin.
>>>>>>>>>
>>>>>>>>> A lot of it works, but calling sqlite3.exe from
>>>>>>>>> cygwin and returning a string with the value
>>>>>>>>> returned from the database seems to attach a
>>>>>>>>> "\r" that expr doesn't remove.
>>>>>>>>>
>>>>>>>>> That is:
>>>>>>>>>
>>>>>>>>> NumPar=`sqlite3.exe ${DATABASE} "SELECT NumPar FROM 
>>>>>>>>> citations
>>>>>>>>> WHERE
>>>>>>>>> X='Key' ;"`
>>>>>>>>>
>>>>>>>>> NumPar comes back as: "12\r"
>>>>>>>>>
>>>>>>>>> and
>>>>>>>>>
>>>>>>>>> NumPar=`expr ${NumPar}`
>>>>>>>>>
>>>>>>>>> doesn't convert it to integer, as the subsequent test fails
>>>>>>>>> because
>>>>>>>>> of NumPar being non-integer (it isn't complaining about N,
>>>>>>>>> that
>>>>>>>>> is integer in the code):
>>>>>>>>>
>>>>>>>>> if [ ${N} -le ${NumPar} ]
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> I can fix this case by:
>>>>>>>>>
>>>>>>>>> NumPar=`printf '%s' "${NumPar}" | sed 's/[^0-9]//g'`
>>>>>>>>>
>>>>>>>>> but then other scripts fail later, presumably because of
>>>>>>>>> strings with \r on them. (I suppose I can use sed to
>>>>>>>>> always remove \r's on every one of these calls, but
>>>>>>>>> that seems pretty kludgy, especially since "clean"
>>>>>>>>> Mac OS X handles all this "properly" without that.
>>>>>>>>> I'm hoping to find an elegant solution.
>>>>>>>>>
>>>>>>>>> I'd rather avoid building sqlite3 under cygwin. I would like
>>>>>>>>> to keep as much as possible in native code, compromising 
>>>>>>>>> only
>>>>>>>>> on cygwin to run my scripts.
>>>>>>>>>
>>>>>>>>> When installing cygwin, you it offers you the choice to
>>>>>>>>> switch
>>>>>>>>> to default text file type to DOS (\r\n). Should I try that?
>>>>>>>>> My pretty serious objection to that would be that any users
>>>>>>>>> already using cygwin with the "correct" default settings
>>>>>>>>> would
>>>>>>>>> not be able to use the scripts anyway.
>>>>>>>>>
>>>>>>>>> So I guess my question here is, do any sqlite users here
>>>>>>>>> have experience fixing this on Windows for Unix cygwin
>>>>>>>>> script calls?
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> _______________________________________________
>>>>>>>>> sqlite-users mailing list
>>>>>>>>> sqlite-users@sqlite.org
>>>>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> sqlite-users mailing list
>>>>>>>> sqlite-users@sqlite.org
>>>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> sqlite-users mailing list
>>>>>>> sqlite-users@sqlite.org
>>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>>
>>>>>> _______________________________________________
>>>>>> sqlite-users mailing list
>>>>>> sqlite-users@sqlite.org
>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>
>>>>> _______________________________________________
>>>>> sqlite-users mailing list
>>>>> sqlite-users@sqlite.org
>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>
>>>> _______________________________________________
>>>> sqlite-users mailing list
>>>> sqlite-users@sqlite.org
>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>
>>> _______________________________________________
>>> sqlite-users mailing list
>>> sqlite-users@sqlite.org
>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>
>> _______________________________________________
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> The information contained in this e-mail is privileged and 
> confidential information intended only for the use of the individual 
> or entity named.  If you are not the intended recipient, or the 
> employee or agent responsible for delivering this message to the 
> intended recipient, you are hereby notified that any disclosure, 
> dissemination, distribution, or copying of this communication is 
> strictly prohibited.  If you have received this e-mail in error, 
> please immediately notify the sender and delete any copies from your 
> system.
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users 

_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Reply via email to