Re: [dev] JIT & VM discussion
http://c9x.me/compile/ Getting it to generate machine code is almost trivial. Porting it to ARM is only reasonably complicated. It is *very* fast. I will refactor it a bit and work more on it starting mid-september. Feel free to join the force. On Sat, Jun 18, 2016 at 10:33:23AM +0100, Connor Lane Smith wrote: > Hi all, > > I was wondering if others had an opinion on JIT. Suppose we don't need > anything fancy like adaptive optimisation, but just wanted to compile > a program at runtime. One possibility might be to generate C code and > store that in a file and then call the C compiler and then execute the > resulting binary... But that seems a bit unpleasant, prone to > compilation failure, and not particularly lightweight either. > > One solution could be simply to produce assembly code, but then that > is tied to a specific architecture, which is unfortunate. There's also > LLVM, but that is a very big and complicated dependency, which > shouldn't really be necessary if we're just jitting something quickly > and don't mind it being a little unoptimised for the sake of > simplicity and speed of compilation. We just want to portably generate > machine code and then run it. > > An ideal might be something like an abstract instruction set together > with a JIT for the abstract machine. To be honest a JIT might not even > be necessary so long as it is has very little interpretation overhead, > the instruction set is general purpose and fixed, and it plays well > with the C memory model. > > Does anyone have any ideas? > > Thanks, > cls
Re: [dev] JIT & VM discussion
How about QBE, which is one step up from assembly?
Re: [dev] JIT & VM discussion
If you don't want to use Lua, what about doing something more like CGI? Then you can just call the configuration program with what you want a dynamic answer for. You could then have a simple awk script parse your config file and answer queries to the host program. I suggest this because I have observed that the suckless community doesn't like dynamic loading and doesn't like parsing. Just use awk's built-in parsing, and don't dynamic load. Just call a standard parser directly. Or just use Lua or LuaJIT. I would just use Lua. But I have to say this about the crazy CGI thing: "Though I drew this conclusion, now it draws me." > On Jun 18, 2016, at 2:33 AM, Connor Lane Smith wrote: > > Hi all, > > I was wondering if others had an opinion on JIT. Suppose we don't need > anything fancy like adaptive optimisation, but just wanted to compile > a program at runtime. One possibility might be to generate C code and > store that in a file and then call the C compiler and then execute the > resulting binary... But that seems a bit unpleasant, prone to > compilation failure, and not particularly lightweight either. > > One solution could be simply to produce assembly code, but then that > is tied to a specific architecture, which is unfortunate. There's also > LLVM, but that is a very big and complicated dependency, which > shouldn't really be necessary if we're just jitting something quickly > and don't mind it being a little unoptimised for the sake of > simplicity and speed of compilation. We just want to portably generate > machine code and then run it. > > An ideal might be something like an abstract instruction set together > with a JIT for the abstract machine. To be honest a JIT might not even > be necessary so long as it is has very little interpretation overhead, > the instruction set is general purpose and fixed, and it plays well > with the C memory model. > > Does anyone have any ideas? > > Thanks, > cls >
Re: [dev] JIT & VM discussion
C JIT->have a look at tinycc from F. Bellard. llvm is c++ then, by definition is not suckless and a massive brain damaged kludged. cheers, -- Sylvain
Re: [dev] JIT & VM discussion
There's several examples of P-code/Pascal VMs around [0][1][2][3]. Some more detailed than others. [0] https://en.wikipedia.org/wiki/P-code_machine#Example_machine [1] http://www.icodeguru.com/vc/10book/books/book4/secg.htm [2] http://blackmesatech.com/2011/12/pl0/pl0.xhtml [3] https://github.com/lkesteloot/turbopascal On Sat, Jun 18, 2016 at 10:39 AM, Kamil Cholewiński wrote: > On Sat, 18 Jun 2016, Connor Lane Smith wrote: >> Hi all, >> >> I was wondering if others had an opinion on JIT. Suppose we don't need >> anything fancy like adaptive optimisation, but just wanted to compile >> a program at runtime. One possibility might be to generate C code and >> store that in a file and then call the C compiler and then execute the >> resulting binary... But that seems a bit unpleasant, prone to >> compilation failure, and not particularly lightweight either. >> >> One solution could be simply to produce assembly code, but then that >> is tied to a specific architecture, which is unfortunate. There's also >> LLVM, but that is a very big and complicated dependency, which >> shouldn't really be necessary if we're just jitting something quickly >> and don't mind it being a little unoptimised for the sake of >> simplicity and speed of compilation. We just want to portably generate >> machine code and then run it. >> >> An ideal might be something like an abstract instruction set together >> with a JIT for the abstract machine. To be honest a JIT might not even >> be necessary so long as it is has very little interpretation overhead, >> the instruction set is general purpose and fixed, and it plays well >> with the C memory model. >> >> Does anyone have any ideas? >> >> Thanks, >> cls > > Hi Connor, > > Creating a simple and general-purpose VM shouldn't be hard! It used to > be my favourite exercise for learning a new programming language. > > Probably much more difficult to get real-world performance; I wouldn't > be surprised if the initial efforts resulted in a 1000x-slower-than-C > execution speed for typical programs. > > With lots of test cases, tuning, benchmarks, and generally a lot of hard > work, I can imagine you could bring it to the 10-100x-slower[1] class. > > [1]: > https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=python3&lang2=gcc > > Of course this doesn't matter that much if your purpose is mostly > scripting behavior (games), or IO-bound stuff (as in waiting for > database - things like Snabb[2] actually do need some real power). > Having good C interop via FFI can save you in many cases. > > [2]: https://github.com/snabbco/snabb > > Yes, JITing is inherently architecture-specific, but the bytecode can be > designed with trade-offs between interpretation and compilation speed. > These days supporting x86-64, ARM and MIPS probably covers >99% of the > devices you'll ever encounter in the wild; the rest can run a bit slower > until someone is motivated enough to write a JIT backend. > > I've never had a close look at any of the Big Name VMs, as most of that > code must suck horribly. Some real-world VMs&JITs however remained > relatively simple - I think there might be a lot to learn from Dis[3] > and LuaJIT[4]. > > [3]: http://doc.cat-v.org/inferno/4th_edition/dis_VM_design > [4]: http://luajit.org/ > > If you have some concrete applications in mind, please do share. I'd > gladly give a shot at prototyping something in this area. > > <3,K. >
Re: [dev] JIT & VM discussion
On Sat, 18 Jun 2016, Connor Lane Smith wrote: > Hi all, > > I was wondering if others had an opinion on JIT. Suppose we don't need > anything fancy like adaptive optimisation, but just wanted to compile > a program at runtime. One possibility might be to generate C code and > store that in a file and then call the C compiler and then execute the > resulting binary... But that seems a bit unpleasant, prone to > compilation failure, and not particularly lightweight either. > > One solution could be simply to produce assembly code, but then that > is tied to a specific architecture, which is unfortunate. There's also > LLVM, but that is a very big and complicated dependency, which > shouldn't really be necessary if we're just jitting something quickly > and don't mind it being a little unoptimised for the sake of > simplicity and speed of compilation. We just want to portably generate > machine code and then run it. > > An ideal might be something like an abstract instruction set together > with a JIT for the abstract machine. To be honest a JIT might not even > be necessary so long as it is has very little interpretation overhead, > the instruction set is general purpose and fixed, and it plays well > with the C memory model. > > Does anyone have any ideas? > > Thanks, > cls Hi Connor, Creating a simple and general-purpose VM shouldn't be hard! It used to be my favourite exercise for learning a new programming language. Probably much more difficult to get real-world performance; I wouldn't be surprised if the initial efforts resulted in a 1000x-slower-than-C execution speed for typical programs. With lots of test cases, tuning, benchmarks, and generally a lot of hard work, I can imagine you could bring it to the 10-100x-slower[1] class. [1]: https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=python3&lang2=gcc Of course this doesn't matter that much if your purpose is mostly scripting behavior (games), or IO-bound stuff (as in waiting for database - things like Snabb[2] actually do need some real power). Having good C interop via FFI can save you in many cases. [2]: https://github.com/snabbco/snabb Yes, JITing is inherently architecture-specific, but the bytecode can be designed with trade-offs between interpretation and compilation speed. These days supporting x86-64, ARM and MIPS probably covers >99% of the devices you'll ever encounter in the wild; the rest can run a bit slower until someone is motivated enough to write a JIT backend. I've never had a close look at any of the Big Name VMs, as most of that code must suck horribly. Some real-world VMs&JITs however remained relatively simple - I think there might be a lot to learn from Dis[3] and LuaJIT[4]. [3]: http://doc.cat-v.org/inferno/4th_edition/dis_VM_design [4]: http://luajit.org/ If you have some concrete applications in mind, please do share. I'd gladly give a shot at prototyping something in this area. <3,K.
Re: [dev] JIT & VM discussion
On Sat, Jun 18, 2016 at 10:33:23AM +0100, Connor Lane Smith wrote: > Hi all, > > I was wondering if others had an opinion on JIT. Suppose we don't need > anything fancy like adaptive optimisation, but just wanted to compile > a program at runtime. Why?
[dev] JIT & VM discussion
Hi all, I was wondering if others had an opinion on JIT. Suppose we don't need anything fancy like adaptive optimisation, but just wanted to compile a program at runtime. One possibility might be to generate C code and store that in a file and then call the C compiler and then execute the resulting binary... But that seems a bit unpleasant, prone to compilation failure, and not particularly lightweight either. One solution could be simply to produce assembly code, but then that is tied to a specific architecture, which is unfortunate. There's also LLVM, but that is a very big and complicated dependency, which shouldn't really be necessary if we're just jitting something quickly and don't mind it being a little unoptimised for the sake of simplicity and speed of compilation. We just want to portably generate machine code and then run it. An ideal might be something like an abstract instruction set together with a JIT for the abstract machine. To be honest a JIT might not even be necessary so long as it is has very little interpretation overhead, the instruction set is general purpose and fixed, and it plays well with the C memory model. Does anyone have any ideas? Thanks, cls
