The author has been writing games since the Atari 800, and has been doing Erlang since '99. So, I don't think it is a lack of understanding of the technology.
On Nov 10, 2007 11:56 AM, David Pollak <[EMAIL PROTECTED]> wrote: > > > > Mike Pence wrote: > > Somewhat related, I found this interesting: > > > > Admitting that functional programming can be awkward: > > http://prog21.dadgum.com/3.html > > Interesting reading, but IMHO, dead wrong. > > The example he described would be ideal for Erlang and Erlang processes > (Actors). One can encapsulate (a la the promise of OO programming, but > in my experience rarely achieved) the behaviors of the the meta parts of > the system (Ants, world, etc.) in separate processes that message each > other. > > I've recently been building a complex system that describes a bunch of > collaborative user interactions and the results on those interactions on > other users and the system as a whole. I modeled it in about 200 lines > of Scala Actor code (very much like Erlang). It would have taken a lot > more time and LoC to model using Java constructs. > > Yes, FP is a different way of thinking, but once one groks the new way, > it makes it easier to solve problems. > > > > > > > > > > > On Nov 10, 2007 10:08 AM, mojo.talantikite <[EMAIL PROTECTED]> wrote: > >> Thanks for all the answers, I really appreciate them. Sometime later > >> today or tomorrow I'd like to write up some questions that pertain to > >> more of what I'm trying to do, but for now that gives me some pretty > >> good ideas as to how I could use Erlang/ErlyWeb. > >> > >> Overall, what do you feel are Erlang/ErlyWeb's particular strengths > >> when it comes to web development? Of course you can pretty much get > >> any language/framework to do what you need it to do, but how does this > >> particular combination yield itself to solving (or simplifying) > >> particular web development problems? > >> > >> Thanks again, the answers were very helpful. > >> > >> best, > >> Mojo > >> > >> On Nov 9, 6:25 am, Al <[EMAIL PROTECTED]> wrote: > >>> On Nov 9, 9:08 am, "Yariv Sadan" <[EMAIL PROTECTED]> wrote: > >>> > >>> > >>> > >>>> On Nov 8, 2007 12:23 PM, mojo.talantikite <[EMAIL PROTECTED]> wrote: > >>>>> I'm in the very early stages of founding a startup, and currently > >>>>> we're in talks about what type of technology to be using. As there > >>>>> aren't too many people on board now, and none besides myself that are > >>>>> technical, there is a pretty blank slate for development as of now. > >>>>> After playing around with Erlang for the past couple of days I'm > >>>>> starting to think about the pros/cons of using Erlang. > >>>>> Anyways, I have questions pertaining to how Erlang/ErlyWeb addresses > >>>>> certain needs and issues of scaling. I thought it might be more useful > >>>>> for a greater number of people if instead of speaking solely about our > >>>>> needs for our web app, we use an example people might be familiar with > >>>>> and get more from: YouTube. Here are some notes on a Google Tech Talk > >>>>> that was given, which also contains a link to the tech talk video: > >>>>> http://kylecordes.com/2007/07/12/youtube-scalability/ > >>>>> Here are some questions that come to mind when thinking about what > >>>>> they faced: > >>>>> 1. Their main reason for choosing Python as their language is because > >>>>> development speed and productivity was (and is) very important for > >>>>> various reasons (one important one was probably showing prototypes and > >>>>> their progresses to potential investors). Erlang seems to be a very > >>>>> productive environment for some applications, but how does it stand up > >>>>> for web app development? Using Python, PHP, or Ruby has the added > >>>>> benefit of having lots of developers to draw from their experiences -- > >>>>> how would Erlang compare? > >>>> Erlang definitely has fewer developers using it for webapps that PHP, > >>>> Python or Ruby. My subjective take is that, having done PHP > >>>> programming and played with RoR, ErlyWeb can be just as productive if > >>>> not more and its code is very concise and expressive. I rarely feel > >>>> that I'm writing repetitive code that doesn't directly solve the > >>>> problem domain. Plus, I prefer Erlang's functional style to imperative > >>>> one. I think FP makes your code more readable and debuggable. Erlang > >>>> has fewer libraries than more popular languages, which may or may not > >>>> affect you depending on your app's needs, but I think that the fewer > >>>> libraries isn't as big an issue as some people say. Erlang has a > >>>> variety of ways for talking to other lanugages and I'd rather write > >>>> most of my code in Erlang in occasionally outsource some functionality > >>>> to another language than lock my whole app to a language I don't like > >>>> as much just because of the existence of a couple of libraries for it. > >>>> Btw, Erlang also has some tools that other languages don't have -- > >>>> most notably, elegant concurrency and Mnesia. > >>>>> 2. Serving content was of course a big issue. They moved to lighttpd > >>>>> from Apache for their videos and got a lot of performance increases > >>>>> (they use CDNs for their most popular content). Yaws seems great for > >>>>> dynamic content, but is there anything better for the static content? > >>>> Use lighttpd for static content and Yaws (or MochiWeb) for dynamic > >>>> content. > >>>>> 3. Another major issue YouTube ran into was serving their > >>>>> thumbnails. Each video can have about 4 thumbnails and each page, > >>>>> while having only one video playing, could have 40-60 thumbnails per > >>>>> page. Thumbnails, unlike video, aren't distributed across hundreds of > >>>>> machines (due to their size). So, you have lots of requests per > >>>>> second and tons of disk seeks. Apache didn't really work out, so they > >>>>> then went to a modified version of lighttpd that put the disk reads > >>>>> into worker threads -- but they ran into issues with that as well. > >>>>> They finally went to Google's BTFE. How would Erlang look at this > >>>>> problem -- would Yaws help at all? > >>>> I'm not too familiar with this problem, but if it's static data, it's > >>>> generally better to serve if from lighttpd than Yaws. > >>> Agreed LightHttpd or apache will outperform Yaws with static content. > >>> If its lots of small files/images look at using a cache like server > >>> with lots and lots of RAM also consider using solid state disks (SSDs) > >>> in Raid 0 configs, these are much faster than traditional disks for > >>> random small files, they blow away performance of even 15K scsi arrays > >>> for this job, as seeks on SSD flash are near zero in comparison. The > >>> limit here is capacity. You can get 64GB SSDs with Sata interfaces > >>> (Sata II soon from Samsung) raid these up with Sata Raid Boards on > >>> fast PCIe interfaces + built in Sata ports in a mirror config You will > >>> be surprised at how kick ass fast these are for the job. You also need > >>> good server Gigabit ports that can be aggregated. > >>> > >>> > >>> > >>> > >>> > >>>>> 4. They use MySql to store metadata, but as the site got huge, they > >>>>> ran into some issues with it. They went from having one database to > >>>>> doing db replication. That caused issues eventually, one of them > >>>>> being that spreading read load (being asynchronous) caused the > >>>>> database to serve outdated data from time to time. They then went to > >>>>> partitioning the database into shards. Overall, though, they're > >>>>> running into issues due to the fact that they're trying to do more and > >>>>> more stuff with the data. For example, recommendation systems and > >>>>> data mining gets really hard on 100s of millions of views per day, so > >>>>> they are needing some solutions that can do parallel queries and > >>>>> handle distributed computation. Would Mnesia be able to help handle > >>>>> these issues, or anything else from the Erlang world that could deal > >>>>> with these issues? > >>>> Mnesia is more suited for storing live session data than large amounts > >>>> of persistent data. And Mnesia isn't really suited for data mining -- > >>>> just for simple queries. However, Erlang helps in that it allows you > >>>> to connect to multiple databases using its great connection pooling > >>>> capabilities (a process is spawned for each database connection, and a > >>>> dispatcher process is responsible for routing queries to connection > >>>> processes). If I were building YouTube in Erlang, I would consider > >>>> doing all replication and partitioning work in the application code > >>>> rather than relying on the database engine. > >>> Use flat files for the metadata and cache them where possible across > >>> machines, consider an appending meta file storage controller (write it > >>> yourself) and file formats that enable appending rather than editing. > >>> The trick heres is to only append to files where possible, editing is > >>> expense (Well percolating the changes is).Break the files up like you > >>> would database tables and crate summarized view files for common > >>> queries. > >>> > >>> If there is a lot of data + analysis consider something like a map > >>> reduce pattern to create views on the backend, erlang is excellent for > >>> this sort of stuff, blows the socks of most others. > >>> > >>> regards > >>> Al > >>> > >>> Regards > >>> Al > >> > > > > > > > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "erlyweb" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/erlyweb?hl=en -~----------~----~----~----~------~----~------~--~---
