John D. Mitchell wrote: >> TIOBE say they use search engine metrics using the search term >> "<language> programming" with some language specific post processing >> (see >> http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm) >> . >> Once the percentage score for a language falls below 5% I don't think >> the numbers are significant. Their longer term trends look to be more >> valuable and show just how jittery the metric is (I'm sure the actual >> usage of established language does not exhibit this degree of jitter. >> What we are seeing is an artefact of the metric. And what happened in >> 2004! http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html) >> . > > Indeed! Just using raw text search metrics is wildly bad. One nasty > effect is the low end numbers often get lost from the crawling and > indexing.
And here's a more insidious problem with search engine results: they don't filter out false positives. So for a reasonably common word like "groovy" you get inflated results. Even "groovy programming" gets hits from programmers that think programming is groovy. "jruby" on the other hand has pretty much only one meaning, as does "scala", "clojure", and "jython". "python" and "ruby" suffer from the same problems as "groovy" since they're common words, but "python programming" and "ruby programming" are less commonly false positives...unless you're a snake-handling jeweler who likes to program. >> "Tim O'Reilly"'s (actually, the latest one I can find is from Mike >> Hendrickson >> http://radar.oreilly.com/2008/03/state-of-the-computer-book-mar-23.html) >> analysis of the IT book market is based on hard data but, of course, >> is not a direct measure of the usage of languages. There are obvious >> problems with correlating book purchase with language use (Somebody >> who has been writing Fortran for 20 years is probably not going to be >> buying a book on Fortran this quarter - Very few of the people buying >> Haskel books this quarter will be using the language for serious >> work). > > Indeed. > > Book sales are much less driven by actual usage as they are by hype. > So, in that sense they can be useful as a leading indicator but they > aren't good at all for actual usage. I think what Tim uses book sales numbers for is to indicate uptick in adoption, rather than overall usage. If a language is great and basically finished for a decade, it's unlikely people will be buying a lot of new books for it. But it also skews toward corporate-mandated "bad" technologies; I bet SOA has sold lots of books. > Mailing lists stats are particularly interesting to me obviously > [ObDisclosure: Mad Scientist of MarkMail :-)] but be careful with them > as they have a variety of artifacts, too. For example some > communities show tremendous mailing list volume growth that then > flattens and declines even as the language becomes increasingly > popular -- one because they moved to a forum based Q&A without a > gateway with the mailing list. Also, different communities have > different cultures and so it's hard to normalize across them -- e.g., > some communities have lots of talkers on a regular basis and some are > more sedate and/or bursty. This backs up what Tom and I were saying about JRuby's lists. Generally low mailing list traffic for JRuby is a good thing, because it means people aren't having trouble using it. For Ruby-related stuff, there's other channels. And the IRC factor plays in heavily. - Charlie --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "JVM Languages" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/jvm-languages?hl=en -~----------~----~----~----~------~----~------~--~---
