Begin forwarded message:
From: Donna Ydreos <[email protected]> Subject: Re: [Jchat] We’re not prepared for the end of Moore’s Law Date: June 27, 2021 at 18:29:38 EDT To: 'Michael Day' via Chat <[email protected]> Richard Feynman—1985 > ...That leads us just with (a) the limitations in size to the size of atoms, > (b) the energy requirements depending on the time as worked out by Bennett, > (c) and the feature that I did not mention concerning the speed of light; we > can’t send the signals any faster than the speed of light. Those are the only > physical limitations that I know on computers. > Page 108 The Computing Machines in the Future Richard P. Feynman Abstract This address was presented by Richard P. Feynman as the Nishina Memo- rial Lecture at Gakushuin University (Tokyo), on August 9, 1985. http://cse.unl.edu/~seth/434/Web%20Links%20and%20Docs/Feynman-future%20computing%20machines.pdf <http://cse.unl.edu/~seth/434/Web%20Links%20and%20Docs/Feynman-future%20computing%20machines.pdf> > > >> On Jun 27, 2021, at 16:11, Skip Cave <[email protected] >> <mailto:[email protected]>> wrote: >> >> An argument for more efficient compiler/interpreter design? >> >> *We’re not prepared for the end of Moore’s Law* >> >> It has fueled prosperity of the last 50 years. But the end is now in sight. >> >> by >> >> - *David Rotman* <https://www.technologyreview.com/author/david-rotman/ >> <https://www.technologyreview.com/author/david-rotman/>> >> - >> >> February 24, 2020 >> >> >> Gordon Moore’s 1965 forecast that the number of components on an integrated >> circuit would double every year until it reached an astonishing 65,000 by >> 1975 is the greatest technological prediction of the last half-century. >> When it proved correct in 1975, he revised what has become known as Moore’s >> Law to a doubling of transistors on a chip every two years. >> >> Since then, his prediction has defined the trajectory of technology and, in >> many ways, of progress itself. >> >> Moore’s argument was an economic one. Integrated circuits, with multiple >> transistors and other electronic devices interconnected with aluminum metal >> lines on a tiny square of silicon wafer, had been invented a few years >> earlier by Robert Noyce at Fairchild Semiconductor. Moore, the company’s >> R&D director, realized, as he wrote in 1965, that with these new integrated >> circuits, “the cost per component is nearly inversely proportional to the >> number of components.” It was a beautiful bargain—in theory, the more >> transistors you added, the cheaper each one got. Moore also saw that there >> was plenty of room for engineering advances to increase the number of >> transistors you could affordably and reliably put on a chip. >> >> Soon these cheaper, more powerful chips would become what economists like >> to call a general purpose technology—one so fundamental that it spawns all >> sorts of other innovations and advances in multiple industries. A few years >> ago, leading economists credited the information technology made possible >> by integrated circuits with a third of US productivity growth since 1974. >> Almost every technology we care about, from smartphones to cheap laptops to >> GPS, is a direct reflection of Moore’s prediction. It has also fueled >> today’s breakthroughs in artificial intelligence and genetic medicine, by >> giving machine-learning techniques the ability to chew through massive >> amounts of data to find answers. >> >> But how did a simple prediction, based on extrapolating from a graph of the >> number of transistors by year—a graph that at the time had only a few data >> points—come to define a half-century of progress? In part, at least, >> because the semiconductor industry decided it would. >> >> [image: Cover of Electronics Magazine April, 1965]The April 1965 >> Electronics Magazine in which Moore's article appeared. >> >> Moore wrote that “cramming more components onto integrated circuits,” the >> title of his 1965 article, would “lead to such wonders as home computers—or >> at least terminals connected to a central computer—automatic controls for >> automobiles, and personal portable communications equipment.” In other >> words, stick to his road map of squeezing ever more transistors onto chips >> and it would lead you to the promised land. And for the following decades, >> a booming industry, the government, and armies of academic and industrial >> researchers poured money and time into upholding Moore’s Law, creating a >> self-fulfilling prophecy that kept progress on track with uncanny accuracy. >> Though the pace of progress has slipped in recent years, the most advanced >> chips today have nearly 50 billion transistors. >> >> Every year since 2001, MIT Technology Review has chosen the 10 most >> important breakthrough technologies of the year. It’s a list of >> technologies that, almost without exception, are possible only because of >> the computation advances described by Moore’s Law. >> >> For some of the items on this year’s list the connection is obvious: >> consumer devices, including watches and phones, infused with AI; >> climate-change attribution made possible by improved computer modeling and >> data gathered from worldwide atmospheric monitoring systems; and cheap, >> pint-size satellites. Others on the list, including quantum supremacy, >> molecules discovered using AI, and even anti-aging treatments and >> hyper-personalized drugs, are due largely to the computational power >> available to researchers. >> >> But what happens when Moore’s Law inevitably ends? Or what if, as some >> suspect, it has already died, and we are already running on the fumes of >> the greatest technology engine of our time? >> >> *RIP* >> >> “It’s over. This year that became really clear,” says Charles Leiserson, a >> computer scientist at MIT and a pioneer of parallel computing, in which >> multiple calculations are performed simultaneously. The newest Intel >> fabrication plant, meant to build chips with minimum feature sizes of 10 >> nanometers, was much delayed, delivering chips in 2019, five years after >> the previous generation of chips with 14-nanometer features. Moore’s Law, >> Leiserson says, was always about the rate of progress, and “we’re no longer >> on that rate.” Numerous other prominent computer scientists have also >> declared Moore’s Law dead in recent years. In early 2019, the CEO of the >> large chipmaker Nvidia agreed. >> >> In truth, it’s been more a gradual decline than a sudden death. Over the >> decades, some, including Moore himself at times, fretted that they could >> see the end in sight, as it got harder to make smaller and smaller >> transistors. In 1999, an Intel researcher worried that the industry’s goal >> of making transistors smaller than 100 nanometers by 2005 faced fundamental >> physical problems with “no known solutions,” like the quantum effects of >> electrons wandering where they shouldn’t be. >> >> For years the chip industry managed to evade these physical roadblocks. New >> transistor designs were introduced to better corral the electrons. New >> lithography methods using extreme ultraviolet radiation were invented when >> the wavelengths of visible light were too thick to precisely carve out >> silicon features of only a few tens of nanometers. But progress grew ever >> more expensive. Economists at Stanford and MIT have calculated that the >> research effort going into upholding Moore’s Law has risen by a factor of >> 18 since 1971. >> >> Likewise, the fabs that make the most advanced chips are becoming >> prohibitively pricey. The cost of a fab is rising at around 13% a year, and >> is expected to reach $16 billion or more by 2022. Not coincidentally, the >> number of companies with plans to make the next generation of chips has now >> shrunk to only three, down from eight in 2010 and 25 in 2002. >> >> *Finding successors to today’s silicon chips will take years of research.If >> you’re worried about what will replace moore’s Law, it’s time to panic.* >> >> Nonetheless, Intel—one of those three chipmakers—isn’t expecting a funeral >> for Moore’s Law anytime soon. Jim Keller, who took over as Intel’s head of >> silicon engineering in 2018, is the man with the job of keeping it alive. >> He leads a team of some 8,000 hardware engineers and chip designers at >> Intel. When he joined the company, he says, many were anticipating the end >> of Moore’s Law. If they were right, he recalls thinking, “that’s a drag” >> and maybe he had made “a really bad career move.” >> >> But Keller found ample technical opportunities for advances. He points out >> that there are probably more than a hundred variables involved in keeping >> Moore’s Law going, each of which provides different benefits and faces its >> own limits. It means there are many ways to keep doubling the number of >> devices on a chip—innovations such as 3D architectures and new transistor >> designs. >> >> These days Keller sounds optimistic. He says he has been hearing about the >> end of Moore’s Law for his entire career. After a while, he “decided not to >> worry about it.” He says Intel is on pace for the next 10 years, and he >> will happily do the math for you: 65 billion (number of transistors) times >> 32 (if chip density doubles every two years) is 2 trillion transistors. >> “That’s a 30 times improvement in performance,” he says, adding that if >> software developers are clever, we could get chips that are a hundred times >> faster in 10 years. >> >> Still, even if Intel and the other remaining chipmakers can squeeze out a >> few more generations of even more advanced microchips, the days when you >> could reliably count on faster, cheaper chips every couple of years are >> clearly over. That doesn’t, however, mean the end of computational progress. >> >> *Time to panic* >> >> Neil Thompson is an economist, but his office is at CSAIL, MIT’s sprawling >> AI and computer center, surrounded by roboticists and computer scientists, >> including his collaborator Leiserson. In a new paper, the two document >> ample room for improving computational performance through better software, >> algorithms, and specialized chip architecture. >> >> One opportunity is in slimming down so-called software bloat to wring the >> most out of existing chips. When chips could always be counted on to get >> faster and more powerful, programmers didn’t need to worry much about >> writing more efficient code. And they often failed to take full advantage >> of changes in hardware architecture, such as the multiple cores, or >> processors, seen in chips used today. >> >> Top of Form >> >> Bottom of Form >> >> Thompson and his colleagues showed that they could get a computationally >> intensive calculation to run some 47 times faster just by switching from >> Python, a popular general-purpose programming language, to the more >> efficient C. That’s because C, while it requires more work from the >> programmer, greatly reduces the required number of operations, making a >> program run much faster. Further tailoring the code to take full advantage >> of a chip with 18 processing cores sped things up even more. In just 0.41 >> seconds, the researchers got a result that took seven hours with Python >> code. >> >> That sounds like good news for continuing progress, but Thompson worries it >> also signals the decline of computers as a general purpose technology. >> Rather than “lifting all boats,” as Moore’s Law has, by offering ever >> faster and cheaper chips that were universally available, advances in >> software and specialized architecture will now start to selectively target >> specific problems and business opportunities, favoring those with >> sufficient money and resources. >> >> Indeed, the move to chips designed for specific applications, particularly >> in AI, is well under way. Deep learning and other AI applications >> increasingly rely on graphics processing units (GPUs) adapted from gaming, >> which can handle parallel operations, while companies like Google, >> Microsoft, and Baidu are designing AI chips for their own particular needs. >> AI, particularly deep learning, has a huge appetite for computer power, and >> specialized chips can greatly speed up its performance, says Thompson. >> >> But the trade-off is that specialized chips are less versatile than >> traditional CPUs. Thompson is concerned that chips for more general >> computing are becoming a backwater, slowing “the overall pace of computer >> improvement,” as he writes in an upcoming paper, “The Decline of Computers >> as a General Purpose Technology.” >> >> At some point, says Erica Fuchs, a professor of engineering and public >> policy at Carnegie Mellon, those developing AI and other applications will >> miss the decreases in cost and increases in performance delivered by >> Moore’s Law. “Maybe in 10 years or 30 years—no one really knows when—you’re >> going to need a device with that additional computation power,” she says. >> >> The problem, says Fuchs, is that the successors to today’s general purpose >> chips are unknown and will take years of basic research and development to >> create. If you’re worried about what will replace Moore’s Law, she >> suggests, “the moment to panic is now.” There are, she says, “really smart >> people in AI who aren’t aware of the hardware constraints facing long-term >> advances in computing.” What’s more, she says, because >> application--specific chips are proving hugely profitable, there are few >> incentives to invest in new logic devices and ways of doing computing. >> >> *Wanted: A Marshall Plan for chips* >> >> In 2018, Fuchs and her CMU colleagues Hassan Khan and David Hounshell wrote >> a paper tracing the history of Moore’s Law and identifying the changes >> behind today’s lack of the industry and government collaboration that >> fostered so much progress in earlier decades. They argued that “the >> splintering of the technology trajectories and the short-term private >> profitability of many of these new splinters” means we need to greatly >> boost public investment in finding the next great computer technologies. >> >> If economists are right, and much of the growth in the 1990s and early >> 2000s was a result of microchips—and if, as some suggest, the sluggish >> productivity growth that began in the mid-2000s reflects the slowdown in >> computational progress—then, says Thompson, “it follows you should invest >> enormous amounts of money to find the successor technology. We’re not doing >> it. And it’s a public policy failure.” >> >> There’s no guarantee that such investments will pay off. Quantum computing, >> carbon nanotube transistors, even spintronics, are enticing >> possibilities—but none are obvious replacements for the promise that Gordon >> Moore first saw in a simple integrated circuit. We need the research >> investments now to find out, though. Because one prediction is pretty much >> certain to come true: we’re always going to want more computing power. >> >> >> Skip Cave >> Cave Consulting LLC >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm >> <http://www.jsoftware.com/forums.htm> > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
