Re: [Ilugc] Some shell scripting
> I was reading the OpenBSD install script for something and I found > that the simple pdksh script > itself was somewhat ...complex. Slackware Linux installer and package management tools and the plethora of rc scripts on most Linux distros do still use shell scripts. Most modern Linux distros however use systemd or upstart instead of old rc-style shell scripts for performance and scalability reasons. The installers of most modern Linux distros are either based on Python or Ruby scripts for maintainability and scalability reasons. > So if you want to write cool software shell scripting is not a bad idea. That was back in the '90s. Today, shell scripting is a *very* bad idea if portability, maintainability and performance are concerned. Shell scripting is not recommended for anything greater than a 10 line script. For anything bigger and complex than that - you should use Python, Ruby or any other modern scripting language instead. > The advantage shell scripts have over perl scripts is that with shell > you have all the UNIX toolkit programs > along with even C code you write for use. That's why Larry Wall created perl so that you do not have to depend on 'C code' (binary executable programs) of your underlying system. Try porting a sophisticated shell script from Solaris to OpenBSD or Linux and you'll understand why shell scripting is a bad idea for writing medium-to-large scale programs. Shell and old generation Perl emphasize glue-scripting (gluing together other programs written largely in C/C++) mainly because they lacked features on their own. Today, things are different. Modern scripting languages (Python, Ruby and also Perl) are full-featured and glue-scripting today can be fully avoided except for a few one-liners. -- Chandrashekar Babu. http://www.chandrashekar.info/ ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] Some shell scripting
On Wed, 2012-12-26 at 10:54 +0530, Chandrashekar Babu wrote: > Most modern Linux distros however use systemd or upstart > instead of old rc-style shell scripts for performance and > scalability reasons. I differ, If you consider Gentoo is a modern distro, then your statement is wrong. Gentoo as well as its childs still use openRC, not systemd/upstart. > That was back in the '90s. Today, shell scripting is a *very* bad idea > if portability, maintainability and performance are concerned. Shell > scripting is not recommended for anything greater than a 10 line > script. For anything bigger and complex than that - you should > use Python, Ruby or any other modern scripting language instead. I still differ, People who want performance don't consider Python/Ruby. A careful design can make scalability easier by writing it in C and do a glue-scripting, rather than Ruby/Python. I'm not an architect who can talk about scalability/performance of a big or enterprise level application, just an amateur programmer, but in my opinion, whatever complex the problem, a well defined split-up and writing tools for each split can scale whatever level we want. Want a proof? just look at git. It doesn't suck!! because it was not written in Python/Ruby, its a bunch of C programs which can be used as commands in bash (as well as in any language). what you say about this? with your so called Python/Ruby scalability? Just my opinion, it doesn't mean, all applications in the world should be written in C. but, I'm saying beautiful applications are all written in C. Thanks, Mohan R ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] Some shell scripting
On Thu, Dec 27, 2012 at 2:01 AM, Mohan R wrote: > > I still differ, People who want performance don't consider Python/Ruby. > A careful design can make scalability easier by writing it in C and do an > glue-scripting, rather than Ruby/Python. > When we talk performance, let's talk about what exactly we mean by it. Most applications are never bottlenecked on the CPU and are most of the time waiting for I/O. In such cases, the so-called "overhead" of running a dynamic language like perl/python/ruby far outweighs the drawbacks. Writing good code in C is probably twice as hard as writing good code in perl (and probably 10x as hard as writing good code in python / ruby). By "good", I mean, maintainable, readable and modular code. That said, when it comes to shell there is one problem I've faced a lot: Since shell scripts leverage external utilities (like cat/sort/cut/etc.,) that can be realized only after a 'fork()' and since most people today prefer to run stuff on VMs, the performance of fork system call on certain VM hosts is unpredictable and bad. Sometimes it may take several seconds for a seemingly harmless process like cat to instantiate. > > I'm not an architect who can talk about scalability/performance of a big > or enterprise level application, just an amateur programmer, but in my > opinion, whatever complex the problem, a well defined split-up and > writing tools for each split can scale whatever level we want. > > Want a proof? just look at git. It doesn't suck!! because it was not > written in Python/Ruby, its a bunch of C programs which can be used as > commands in bash (as well as in any language). what you say about this? > with your so called Python/Ruby scalability? > git is a fairly limited and well defined problem area that involves mostly string manipulation. Also, its open source (and open source software quality _usually_ is better than enterprise grade software, IMHO). Consider something more complex and event driven, like your typical enterprise grade HTTP application that is under a constant pressure of meeting deadlines and people whose, shall we say, social conditioning is not necessarily wired for the most elegant solutions but instead for those that just work and those that give them recognition in their social circle (read: "career"). Cheers, -Suraj -- Career Gear - Industry Driven Talent Factory http://careergear.in/ ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] Some shell scripting
Hi, > > I differ, If you consider Gentoo is a modern distro, then your statement > is wrong. Gentoo as well as its childs still use openRC, not > systemd/upstart. I used the word "Most"; not "All". Gentoo, Slackware and a couple of other distros do not use systemd or upstart yet. Having said that, on Gentoo today, systemd is just an 'emerge' away. > I still differ, People who want performance don't consider Python/Ruby. > A careful design can make scalability easier by writing it in C and do a > glue-scripting, rather than Ruby/Python. Wrong. People who want performance prefer Perl/Python/Ruby or the likes in comparison to C programs + Shell. It is obvious that you have no first hand exposure at maintaining a 25,000+ line shell script (which I did in the late 90s) - so you have no clue on what I'm talking about. As an exercise, try writing a Linux installer and package manager with features and capability of Redhat's installer (anaconda) and their package manager (RPM/YUM) using just shell scripts and existing bare-bones Linux tools (tar, cpio, bzip2, etc). You'll understand immediately why people chose Python/Ruby over shell :-) > > I'm not an architect who can talk about scalability/performance of a big > or enterprise level application, just an amateur programmer, but in my > opinion, whatever complex the problem, a well defined split-up and > writing tools for each split can scale whatever level we want. I am a technology mentor - and I train architects on how software must not be built in the first place. I can talk at length about scalability modelling. In the 80s and early 90s, traditional UNIX paradigms emphasized on splitting complex problems into smaller problem sets that could be addressed by carefully written tools and utilities in C and then glued-up together using the shell (just like LEGO(tm) building blocks). This worked well during at that time. But the problem with shell is in its architectural design - it was never meant for solving complex problems - the overhead of forking multiple process (to run different programs), its lack of built-in functions for rich data-types (hashes, objects, resizable vector-based arrays) and algorithms (map, reduce, sort, search), and its extensive dependency on external tools and commands (like cut, grep, sed, etc) for trivial tasks makes it slower, unintuitive and unattractive for solving "complex" problems. Today, we have a better design - rather can creating hundreds and thousands of binary programs using C, we have most performance-critical and low-level functions written in C and bundled into libraries. And we use smart programming languages like Python/Ruby/Perl that wrap these libraries as extension modules. That's how Python/Ruby/Perl perform better and are more scalable. For example, a python program made up of Python + C library extension is a lot faster than a shell script made up of Shell + C program binary (as there is no fork-dup-exec involved in the former). > Want a proof? just look at git. It doesn't suck!! Git doesn't suck as long as you are happy with what it provides. There's very little scope for customizability here. Every time I tried implementing a workflow radically different from what git provides, I end up yelling out what Git really meant - "Goddamn IdioT" (as per Linus Torvalds, the creator) and move on to mercurial. > because it was not > written in Python/Ruby, its a bunch of C programs which can be used as > commands in bash (as well as in any language). And that's why it is rigid, non-flexible and expects you to learn umpteen commands to manage your source code eventually. > what you say about this? with your so called Python/Ruby scalability? Mercurial is written fully on Python and it is way more scalable and fully customizable than Git. For example, writing trigger rules for mercurial expects you to only know python and mercurial library interface. In comparison, git expects you to know umpteen commands, shell and a plethora of not-so-portable utilities like inotify on Linux - that's not what you call scalable. Mercurial has proven to be much faster than git for large-scale commits and I have personally benchmarked both myself on server deployments. For your reference, look up the following link: http://draketo.de/proj/hg-vs-git-server/test-results.html Ubuntu community folks will tell you how bazaar is as fast as (or better than) git - and that's written in Python too. The beauty of well-designed software lies in its extensibility to solve problems pertaining to the user domain. Lesson for you: merely writing an "application" in C does not make it any faster. In most cases, it is counter-productive as it does not emphasize reusability. Good software design principles revolve around using C/C++ to address problems related to system domain - that could be extended as libraries by programs written in Python/Ruby/Perl/Haskell/Java or any
Re: [Ilugc] Some shell scripting
On Thu, 2012-12-27 at 10:07 +0530, Chandrashekar Babu wrote: >http://draketo.de/proj/hg-vs-git-server/test-results.html Kind of a response benchmarking which proved that hg is better. Just one question? did the test used python+Subprocess.call() for all the git call's? I thought libgit.a is there (or better http://libgit2.github.com/) which prevents fork-exec. However, the test is talking about gc. So, my point doesn't count as a valid argument. But I'm just wondering the test is comparing things which are not in the same level, If the entire test performed by doing python+hg-with-api and python +git-via-subprocess.call()/bash+git-commands, then I'll say, the test is biased towards hg. Well, you can ask "the test is all about garbage collection, what the hell you are talking about?" well, may be git has some problem with gc, but I'm saying there is something fishy the way the test has been performed. > Lesson for you: merely writing an "application" in C does not make it any > faster. In most cases, it is counter-productive as it does not emphasize > reusability. Good software design principles revolve around using C/C++ > to address problems related to system domain - that could be extended as > libraries by programs written in Python/Ruby/Perl/Haskell/Java or any > high-level language that allows application developers to focus on problems > related to user domain. > And yes, get this out of your mind that C is all about "high performance". > Performance, Scalability, Maturity and Maintainability depends *not* on your > choice of programming language - but rather the software design and > architecture. Aahh!! the same old (or modern) advice. Common pundits, I'm not saying C is the only way and I really say python/javascript is way better on text processing, no doubt on that, its my personal experience. But saying C and shell-scripting doesn't belongs to the modern programming is utter shit. > As another exercise for you, try solving the algorithm problems on > http://projecteuler.net/problems using C (with standard C libraries of > course). Well, I'm solving my own day-to-day problems with C http://github.com/mohan43u/ the same problems can be solvable much easier in other languages, but whats the point of solving my problems in easier way? I'm not a corporate with deadlines. Programming is fun after all :) > Then, try looking up solutions for the same problems solved > using Python, Haskell, Ruby, Perl (and more languages if you wish) > and now prove to us that C programs are beautiful than > Python/Ruby/Haskell programs ;-) Well, beautiful, depends on one's view. I would say thorn is beautiful than the rose (this is a counter post right? obviously I have to post like this. sorry for wasting you time). Thanks, Mohan R ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
