Wow!!!! Jim, you're asking to rewrite the whole code of cygwin. You
want all sorts of programs (like sed, awk, expr etc) to suddenly
become a part of one program and the code of bash to be rewritten from
scratch so that it never forks itself into new process but still
implements all features that it has (I don't even sure if it's
possible). Sorry, but that's not what cygwin is about. It's about
taking sources of all programs from Unix and compiling them without
changes (or with minimal changes) so that they work on Windows and so
that they use all Unix principles (like standard structure of file
system directories e.g.).
With your requirements you'll need some other project. But I don't
know any that supports bash scripts from Unix as is, without
changes...
Pavel
On Tue, Sep 22, 2009 at 1:01 PM, Jim Showalter <j...@jimandlisa.com> wrote:
> The point I was trying to make is that the majority of commands cygwin
> supports don't have to do any forking. They could just be method
> calls. Pipes would be parameters going in and return values coming
> out, all running in a single process. If fork() is specifically called
> in a script, then the slower approach would have to be used, but most
> of the time it would not.
>
> ----- Original Message -----
> From: "Pavel Ivanov" <paiva...@gmail.com>
> To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org>
> Sent: Tuesday, September 22, 2009 6:58 AM
> Subject: Re: [sqlite] cygwin and sqlite
>
>
>>> But after all these years I wonder why they don't fix the fork
>>> problem?
>>> MacOS runs on Intel processors. Windows runs on Intel processors.
>>> Surely
>>> they could learn how it *should* be done by studying things like
>>> the
>>> Open Source Java code?
>>
>> You seem to forget the basics. It's not the processor who makes
>> fork()
>> possible, it's OS. Unix kernel implemented fork() and Unix kernel
>> implemented process management in the way that makes implementation
>> of
>> fork() very quick and easy. Windows didn't implement fork() and it
>> implemented process management in the way making fork() impossible.
>> I'd say it's a superior achievement on cygwin side that they were
>> able
>> to implement fork() somehow at all. Just a simple fact: you execute
>> some code that uses memory in some way then you call fork() on Unix
>> and you already have 2 absolutely different processes that can
>> access
>> the same data in memory. On windows there's no way to start a new
>> process so that it can access the same data as first process unless
>> you thought about that beforehand, placed all your data into shared
>> memory (which is a lot harder to work with, btw) and made another
>> process to read the same shared memory. Also there's no way to start
>> new process on Windows so that it executes the same code as first
>> process from the point where second process was started...
>> So I'd better not complained but tried to understand the roots of
>> the problem...
>>
>> And if you want the comparable speed of scripts on both platforms I
>> suggest you to look into perl (or python, whatever you prefer). The
>> both have native implementations and their speed should be
>> comparable
>> to each other. That said of course if you don't start a lot of
>> processes from scripts and don't try to run command line
>> utilities...
>>
>> Pavel
>>
>> On Mon, Sep 21, 2009 at 10:29 PM, John <jhy...@earthlink.net> wrote:
>>> Pavel Ivanov wrote:
>>>>> MacBook Mac OS X 10.5.8
>>>>> 2 GHz Intel Core Duo
>>>>> 1 GB memory:
>>>>> 17 minutes 46 seconds.
>>>>>
>>>>> IBM ThinkPad
>>>>> Windows XP (latest patches)
>>>>> 1.70 GHz, 512 MB memory:
>>>>> 6 hours 25 minutes 57 seconds
>>>>
>>>> Windows is very slow in starting new processes if compared to any
>>>> Unix
>>>> system (especially if compared Windows + 512 MB and Unix + 1 GB).
>>>> In
>>>> cygwin starting new processes even slower because for some reason
>>>> emulating fork() involves starting 2 processes one of which dies
>>>> immediately. And bash scripts use processes a lot especially with
>>>> Unix
>>>> paradigm when for each small action you start new program (like
>>>> sed,
>>>> awk, test, true and whole lot of others). Thus bash scripts on
>>>> cygwin
>>>> will be slow unavoidably.
>>>> But I'm digressing. This is subject for some other mailing list.
>>>> :)
>>>
>>> I'm blacklisted apparently on the cygwin mailing list for when a
>>> couple of years ago I complained rather unflatteringly about how
>>> slow
>>> it was/is when I was writing a simple expenses program (that works
>>> in
>>> seconds on my Mac). I forgot about that. A badge of honor in my
>>> opinion.
>>>
>>> But after all these years I wonder why they don't fix the fork
>>> problem?
>>> MacOS runs on Intel processors. Windows runs on Intel processors.
>>> Surely
>>> they could learn how it *should* be done by studying things like
>>> the
>>> Open Source Java code?
>>>
>>> It looks like I won't be able to distribute my stopgap cygwin code
>>> on
>>> Windows. I need to start speed reading my Java/Swing books I guess
>>> in
>>> my quest for my program to write once, run anywhere.
>>>
>>>> Pavel
>>>>
>>>> On Fri, Sep 18, 2009 at 3:26 AM, John <jhy...@earthlink.net>
>>>> wrote:
>>>>> Pavel Ivanov wrote:
>>>>>>> At least I think that is what you suggest, and think it just
>>>>>>> may work! But I could be wrong!
>>>>>> Yes, that's exactly what I suggest.
>>>>>>
>>>>>> Pavel
>>>>> It worked! Fortunately I had already parameterized SQLITE3 as a
>>>>> preference variable so I could have the same scripts run easily
>>>>> on Mac
>>>>> OS and Windows. There are dozens of sqlite3 calls throughout the
>>>>> scripts.
>>>>>
>>>>> My whole set of scripts that process raw data and load the
>>>>> database by
>>>>> reading text files seem to work.
>>>>>
>>>>> cygwin is as slow as I recall, however. I was writing expense
>>>>> scripts a
>>>>> few years ago and abandoned it for MacOS Unix. I moved 100% to
>>>>> Mac OS.
>>>>> (except for this project which I want to work on Mac, linux, and
>>>>> Windows; my next goal is recoding it in Java with its Swing GUI,
>>>>> but I'm
>>>>> just learning Java and Swing, but I'm on my way...).
>>>>>
>>>>> Observed elapsed times on my two notebook computers for the same
>>>>> scripts
>>>>> to load the database (using sqlite3 calls and lots of sed and awk
>>>>> processing of thousands of lines of input data):
>>>>>
>>>>> MacBook Mac OS X 10.5.8
>>>>> 2 GHz Intel Core Duo
>>>>> 1 GB memory:
>>>>> 17 minutes 46 seconds.
>>>>>
>>>>> IBM ThinkPad
>>>>> Windows XP (latest patches)
>>>>> 1.70 GHz, 512 MB memory:
>>>>> 6 hours 25 minutes 57 seconds
>>>>>
>>>>> Fortunately, sqlite .dump and restoring from the resultant sql
>>>>> will be
>>>>> able to be used for most of the heavy lifting when I'm done.
>>>>> Changes to
>>>>> the data will come in small increments over time from then on. My
>>>>> dumpit
>>>>> and restoreit scripts each take only seconds on both platforms
>>>>> for the
>>>>> full set of current data.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>> On Thu, Sep 17, 2009 at 1:18 PM, John <jhy...@earthlink.net>
>>>>>> wrote:
>>>>>>> Pavel Ivanov wrote:
>>>>>>>>> I'd rather avoid building sqlite3 under cygwin. I would like
>>>>>>>>> to keep as much as possible in native code, compromising only
>>>>>>>>> on cygwin to run my scripts.
>>>>>>>> And this is root of your problem. Using mix of cygwin-native
>>>>>>>> applications with windows-native applications will always have
>>>>>>>> such
>>>>>>>> problem.
>>>>>>>>
>>>>>>>>> When installing cygwin, you it offers you the choice to
>>>>>>>>> switch
>>>>>>>>> to default text file type to DOS (\r\n). Should I try that?
>>>>>>>> Don't do that. This mode of operation is not supported much
>>>>>>>> and not
>>>>>>>> recommended by cygwin developers and it reportedly will
>>>>>>>> significantly
>>>>>>>> slow down cygwin's operation.
>>>>>>>>
>>>>>>>>> So I guess my question here is, do any sqlite users here
>>>>>>>>> have experience fixing this on Windows for Unix cygwin
>>>>>>>>> script calls?
>>>>>>>> The major suggestion here: write some "windows native code
>>>>>>>> launcher"
>>>>>>>> that will be used for running all non-cygwin applications
>>>>>>>> (this can be
>>>>>>>> just function in the script). It will do nothing on unix
>>>>>>>> platforms
>>>>>>>> (select your own preferred way of distinguishing it) and it
>>>>>>>> will
>>>>>>>> always strip off '\r' from output of running application on
>>>>>>>> windows
>>>>>>>> (you can use sed for that). And there's nothing else you can
>>>>>>>> do about
>>>>>>>> it.
>>>>>>> This sounds like a great idea. I can have all sqlite3.exe calls
>>>>>>> "intercepted" by another script call like:
>>>>>>>
>>>>>>> NumPar=`WINDOWSCALL Program Arguments`
>>>>>>>
>>>>>>> WINDOWSCALL is the launcher that calls Program sqlite3.exe
>>>>>>> with its arguments and strips off any trailing \r's
>>>>>>> and returns that string to the caller through stdout,
>>>>>>> as to NumPar here. WINDOWSCALL can do nothing on Unix/MacOS,
>>>>>>> and fix the string on Windows.
>>>>>>>
>>>>>>> At least I think that is what you suggest, and think it just
>>>>>>> may work! But I could be wrong!
>>>>>>>
>>>>>>> Thanks! I'll try coding it.
>>>>>>>
>>>>>>>> Pavel
>>>>>>>>
>>>>>>>> On Thu, Sep 17, 2009 at 12:26 PM, John <jhy...@earthlink.net>
>>>>>>>> wrote:
>>>>>>>>> I am writing some Unix scripts on Mac OS X that use
>>>>>>>>> sqlite3. Since the program could be useful to those
>>>>>>>>> on Windows, I figured I'd see if they worked under
>>>>>>>>> cygwin.
>>>>>>>>>
>>>>>>>>> A lot of it works, but calling sqlite3.exe from
>>>>>>>>> cygwin and returning a string with the value
>>>>>>>>> returned from the database seems to attach a
>>>>>>>>> "\r" that expr doesn't remove.
>>>>>>>>>
>>>>>>>>> That is:
>>>>>>>>>
>>>>>>>>> NumPar=`sqlite3.exe ${DATABASE} "SELECT NumPar FROM citations
>>>>>>>>> WHERE
>>>>>>>>> X='Key' ;"`
>>>>>>>>>
>>>>>>>>> NumPar comes back as: "12\r"
>>>>>>>>>
>>>>>>>>> and
>>>>>>>>>
>>>>>>>>> NumPar=`expr ${NumPar}`
>>>>>>>>>
>>>>>>>>> doesn't convert it to integer, as the subsequent test fails
>>>>>>>>> because
>>>>>>>>> of NumPar being non-integer (it isn't complaining about N,
>>>>>>>>> that
>>>>>>>>> is integer in the code):
>>>>>>>>>
>>>>>>>>> if [ ${N} -le ${NumPar} ]
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> I can fix this case by:
>>>>>>>>>
>>>>>>>>> NumPar=`printf '%s' "${NumPar}" | sed 's/[^0-9]//g'`
>>>>>>>>>
>>>>>>>>> but then other scripts fail later, presumably because of
>>>>>>>>> strings with \r on them. (I suppose I can use sed to
>>>>>>>>> always remove \r's on every one of these calls, but
>>>>>>>>> that seems pretty kludgy, especially since "clean"
>>>>>>>>> Mac OS X handles all this "properly" without that.
>>>>>>>>> I'm hoping to find an elegant solution.
>>>>>>>>>
>>>>>>>>> I'd rather avoid building sqlite3 under cygwin. I would like
>>>>>>>>> to keep as much as possible in native code, compromising only
>>>>>>>>> on cygwin to run my scripts.
>>>>>>>>>
>>>>>>>>> When installing cygwin, you it offers you the choice to
>>>>>>>>> switch
>>>>>>>>> to default text file type to DOS (\r\n). Should I try that?
>>>>>>>>> My pretty serious objection to that would be that any users
>>>>>>>>> already using cygwin with the "correct" default settings
>>>>>>>>> would
>>>>>>>>> not be able to use the scripts anyway.
>>>>>>>>>
>>>>>>>>> So I guess my question here is, do any sqlite users here
>>>>>>>>> have experience fixing this on Windows for Unix cygwin
>>>>>>>>> script calls?
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> _______________________________________________
>>>>>>>>> sqlite-users mailing list
>>>>>>>>> sqlite-users@sqlite.org
>>>>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> sqlite-users mailing list
>>>>>>>> sqlite-users@sqlite.org
>>>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> sqlite-users mailing list
>>>>>>> sqlite-users@sqlite.org
>>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>>
>>>>>> _______________________________________________
>>>>>> sqlite-users mailing list
>>>>>> sqlite-users@sqlite.org
>>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>>
>>>>> _______________________________________________
>>>>> sqlite-users mailing list
>>>>> sqlite-users@sqlite.org
>>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>>
>>>> _______________________________________________
>>>> sqlite-users mailing list
>>>> sqlite-users@sqlite.org
>>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>>
>>> _______________________________________________
>>> sqlite-users mailing list
>>> sqlite-users@sqlite.org
>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>
>> _______________________________________________
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
