Re: [Mixxx-devel] Hate JavaScript and XML? ...then keep reading!
Lua and JavaScript are very very similar. I'm a big fan of Lua and would support a switch to that language. I don't know anything about Coffeescript. I don't have a problem with JavaScript, though. Neale (soon-to-be maintainer of four Hercules mappings) -- Try New Relic Now & We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, & servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ Get Mixxx, the #1 Free MP3 DJ Mixing software Today http://mixxx.org Mixxx-devel mailing list Mixxx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mixxx-devel
Re: [Mixxx-devel] Hate JavaScript and XML? ...then keep reading!
I disagree. First, there is nothing intrinsically "more performant" in Lua than in JavaScript -- they are both dynamically typed languages with objects and first-class functions. Also, there is a wide choice of VMs in in Javascript. While QtScript uses the webkit JS engine, which is fairly good and nothing prevents us to moving to V8 or something like that. Lua does not meet the realtime constraints neither anyway. In my experience, the scripting language should not be the fastest, but the most expressive, because its goal is to allow very fast development and hackability. If you look around, other music software are also using a wide variety of similar approaches, like Ableton Live, which uses Python (which is slower but more expressive than JS), or Max/MSP, which uses JavaScript. What its missing in Mixxx is, however, an API in the scripting engine to bind a MIDI signal to a control. This way, the script can handle mode selection, but for every mode, the important MIDI messages (cue button, knobs and faders) still go through the engine directly. Something like: engine.connectMidiToControl(status, midino, group, control, options); engine.connectControlToMidi(group, control, status, midino, options); Which would be equivalent to defining an entry in the or section of the XML file. If I were to critisize JavaScript, on the other hand, it would be because it is not always as concise and expressive as it could be. Fortunatelly, a wide range of languages nowadays compile to JavaScript, and that is one of the reasons my CoffeeScript project can exist at all. JP On 25/05/13 15:38, petah wrote: > Hi there, > > first post but I am NOT a student looking to rubberstamp GSoC all over my > resume :) > > RJ, you may remember me from www.hackerdjs.com/software/cdj2midi.html, > introduced by Mark Hills, through Adam D, I think. Since we last spoke I've > become a linux-head, been stalking Mixxx-devel for a month. Seeing you're > into 3D/games I'm surprised Mixxx uses Javascript, Python and qtScript but > not Lua. > > I'm guessing the initial choice of XML/Javascript was to attract more people > to write controller glue code but Mixxx has gotten a lot of > traction/attention since then. With a new focus on reliability, > Mixxx/hardware-interface drivers should really be written by somewhat > experienced programmers (at least not invite bad code). QA's a bitch now > cause you often don't have the hardware but if Mixxx's user base keeps > growing as it does you'll soon be able to threaten manufacturers *not* to > support their device:) > > Assuming Mixxx maintains its growth (vs withers & dies), driver and hardware > availability is a given, what matters is runtime quality/efficiency. Porting > the current ./res/controllers/ is a one-time effort. Also, the majority of > users will be non-techie DJs; at the top DJs with a coding assistant :) > > Wrt speed, Javascript engines can afford many luxuries when running in web > browsers (or servers) because they're not dealing with 10ms latencies. A > web-surfing user won't notice a garbage collector hiccup but in a "critical > realtime" app like Mixxx it's the end of the world for a very pissed DJ. > Hardcore videogames are somewhat similar and there's virtually no Javascript; > #1 engine is Lua because it's tiny and predictable; where the VM falls short > you bind native C code in a handful of lines. > > Either way I think it'd make sense to use the same VM for both drivers and > macros since they're really on the same path between the hardware interface > and Mixxx. > > (bowing out) > > -- p > > -- > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may > ___ > Get Mixxx, the #1 Free MP3 DJ Mixing software Today > http://mixxx.org > > > Mixxx-devel mailing list > Mixxx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/mixxx-devel > -- Try New Relic Now & We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, & servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ Get Mixxx, the #1 Free MP3 DJ Mixing software Today http://mixxx.org Mixxx-devel mailing list Mixxx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mixxx-devel
Re: [Mixxx-devel] Hate JavaScript and XML? ...then keep reading!
On Sun, 26 May 2013 16:52:59 +0200
Juan Pedro Bolívar Puente wrote:
> I disagree.
I didn't mean to start a language pissing contest... but it looks like I did :)
> First, there is nothing intrinsically "more performant" in Lua than in
> JavaScript -- they are both dynamically typed languages with objects and
> first-class functions. Also, there is a wide choice of VMs in in
> Javascript. While QtScript uses the webkit JS engine, which is fairly
> good and nothing prevents us to moving to V8 or something like that.
Javascript is performant in so far as it's JITed; there's LuaJIT too but I
don't recommend any type of JIT because it allocates a gigantic block of RAM
that's tagged as executable so it's harder to sandbox. JITs are largely
unnecessary if you identify functionality hotspots and can easily rewrite those
in C++, then expose those to the script layer.
> Lua does not meet the realtime constraints neither anyway.
What makes you say that? I see you have experience in just about every
interpreter expect Lua (btw I live next door in Valencia if you want to take it
outside:).
Lua's used by Reason and ReNoise and in game engines since the PS2 all the way
to Crytek's Far Cry/Crysis which have similar frame-lock/tick constraints.
There's a laundry list of realtime apps on www.lua.org/uses.html. Most
tellingly, you'll find it in many embedded systems like Cisco, Barracuda,
MicroTik RouterOS, OpenWrt, Wavecom GSM firmware. It hardly gets more realtime
than that.
> In my
> experience, the scripting language should not be the fastest, but the
> most expressive, because its goal is to allow very fast development and
> hackability. If you look around, other music software are also using a
> wide variety of similar approaches, like Ableton Live, which uses Python
> (which is slower but more expressive than JS)
You would know so correct me if I'm wrong, aren't large parts of Live itself
written in Python? Lua is much nimbler and doesn't offer Python's richer object
orientation. In Mixxx's case, I understand the idea is extensibility - not to
rewrite large parts of the application in script.
> What its missing in Mixxx is, however, an API in the scripting engine to
> bind a MIDI signal to a control.
That's where Lua shines; writing a native Lua 'module' takes very few lines of
code. Lua is meant to be an embeddable language, i.e. it plugs into an existing
application vs the other way around, it's unobtrusive because it is so tiny.
> This way, the script can handle mode
> selection, but for every mode, the important MIDI messages (cue button,
> knobs and faders) still go through the engine directly. Something like:
>
> engine.connectMidiToControl(status, midino, group, control, options);
> engine.connectControlToMidi(group, control, status, midino, options);
I wrote something like that and used it on all but my very first gigs. I
attached the glue code and script here; initialization is at the bottom. Please
don't judge Lua's merits (or mine for that matter:) based on this, it was never
meant to be used/seen by anyone but myself, is/was a work in progress and I
frequently tweaked scripts just before a gig, knowing full well it was pretty
risky/stupid.
> If I were to critisize JavaScript, on the other hand, it would be
> because it is not always as concise and expressive as it could be.
I'm fairly sure Lua is one of the most concise languages there is. Its
reference manual is 103 pages long, including the C API. Also Lua 5.2 has a
rich set of binary operators (for example bit range extract, ROL/ROR) that'd
help HID/MIDI processing.
cheers,
-- p
-- MIDIbots main lua scriptey
-- require "Traktor313"
-- MIDI NOTES START AT CONTROL 9 ?! (no fucking idea why)
-- device aliases (CACA -- MOVE TO SEPARATE INI -- different on each computer!)
dm2alias = "SB Audigy MIDI Port II [A800]"
traktoralias = "MIDI Yoke NT: 1"
-- some physical dj1 buttons are defined twice; once per selected deck
gDeckLUT = {[1] = "deck_a", [2] = "deck_b"}
Variables
cueMode = "auto"
Shift = false
gSnapFlag = false
RebootCnt = 4
LastJoyLeftClock = 0
LastJoyRightClock = 0
LastJoyYTime = 0
ScanFlag = false
Wheel2for1 = false
WheelHadMod = false
LastDeckLoaded = -1
LockedDeck = -1
gDeck = {[1] = {TempoEditFlag = false, TempoPerc = 0}, [2] = {TempoEditFlag = false, TempoPerc = 0}}
gMaxTempoDelta = 40 -- not is up AND down
min, perc = mb.GetBatteryLevel()
print ("battery = "..min.." minutes, "..perc.."%%")
print("RECOMPILED NAKED SCRIPT!!!")
printf ---
function printf(fmt, ...)
local argtab = {...}
-- replace %S with "%s"
local expanded_fmt = string.gsub(fmt, "%%S", "\"%%s\"")
local s = string.format(expanded_fmt, unpack(argtab))
print(s)
end
Get Editd Deck -
local
function GetEditedDeck()
local
Re: [Mixxx-devel] Hate JavaScript and XML? ...then keep reading!
Nice approach, Christian! while looking at the scripts, i thought it will be good to have something like this, it will reduce code and make it more readable. I see two sides here. The (hopefully complete) API that mixxx provides (connect etc), some built-in convenience functions and the same freedom as today to just use any valid ECMA script along with that. If more 'native' functions will be needed (for instance, send/receive OSC), they will be added and exposed to be used in scripts. For the language discussion, i wonder why many are keen to get rid of Javascript. Why? It's widely known, easy to understand and learn, non-exotic and there is no need for very high performance low-latency processing for switching on and off some LEDs on the controller. Furthermore, it seems to work well and is established in mixxx. I wonder if there's a benefit when swapping it with something else. There will be possibly no gain in functionality (that is, how good the controller will be mapped to the software). More importantly is to have a reasonable set of supported controllers IMHO. Greetings Thomas > Hi Juan Pedo, > > What you did looks pretty similar to what I published yesterday. But I > kept the xml stuff and reduced it as much as possible and assembled a > JavasSript library for the rest. :) > > http://blog.christian-hufgard.de/mixxx_midiscripting_made_easier-2013-05-23 > > > Christian > > -- Try New Relic Now & We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, & servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ Get Mixxx, the #1 Free MP3 DJ Mixing software Today http://mixxx.org Mixxx-devel mailing list Mixxx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mixxx-devel
Re: [Mixxx-devel] Hate JavaScript and XML? ...then keep reading!
> I didn't mean to start a language pissing contest... but it looks > like I did :) Sorry my message was a bit harsh... I did not mean to dismerit Lua. But lets make some comments... >> Lua does not meet the realtime constraints neither anyway. > > What makes you say that? I see you have experience in just about > every interpreter expect Lua (btw I live next door in Valencia if you > want to take it outside:). I am based in Berlin now, so lets argue here instead :) By "real-time constraints" I mean the real-time constraints that are usually required in the engine of most audio applications, which require algorithms to be O(bufferSize). Even non-GC heap allocations do not qualify for this, and because closures and dynamic typing can not be implemented without heap allocations (and GC) I don't think one can run Lua code in the engine threads. And if one can not run Lua code in the engine threads, there is going to be an added latency anyway to send the messages accross the engine boundaries, as long as the interpreter is not really bad, I do not think this is a big issue. If you mean, on the other hand, that if you do not use objects and closures then Lua can meet real-time conditions, the same holds for JavaScript. I would suggest you to take a look at "asm.js", a subset of JavaScript used to compile new-generation C++ games to JavaScript that run at 2X the native version (and it is the whole game running in JS, even the engine!). > Lua's used by Reason and ReNoise and in game engines since the PS2 > all the way to Crytek's Far Cry/Crysis which have similar > frame-lock/tick constraints. There's a laundry list of realtime apps > on www.lua.org/uses.html. Most tellingly, you'll find it in many > embedded systems like Cisco, Barracuda, MicroTik RouterOS, OpenWrt, > Wavecom GSM firmware. It hardly gets more realtime than that. Ok, that means its fast, but not real-time in the sense I mean, because they do not have a deadline. Or to put otherwise: in Far Cry if a single frame takes 50ms instead of an average of 16 (assuming 60fps) it will most probably go unnoticed. In an audio application engine if you get a 10ms cycle with a not-so-low bufferSize of 256, you already have an audio dropout with a noticeable click that I would not want to have at all in a big soundsystem. >> In my experience, the scripting language should not be the fastest, >> but the most expressive, because its goal is to allow very fast >> development and hackability. If you look around, other music >> software are also using a wide variety of similar approaches, like >> Ableton Live, which uses Python (which is slower but more >> expressive than JS) > > You would know so correct me if I'm wrong, aren't large parts of Live > itself written in Python? Lua is much nimbler and doesn't offer > Python's richer object orientation. In Mixxx's case, I understand the > idea is extensibility - not to rewrite large parts of the application > in script. Python is used mostly there for hardware mappings, same use-case as for Mixxx. It is true that those hardware mappings are more complex than in Mixxx (think for example about Ableton Push) but, wouldn't it be nice to have such kind of complex controllers also having crazy workflows in Mixxx? >> This way, the script can handle mode selection, but for every mode, >> the important MIDI messages (cue button, knobs and faders) still go >> through the engine directly. Something like: >> >> engine.connectMidiToControl(status, midino, group, control, >> options); engine.connectControlToMidi(group, control, status, >> midino, options); > > I wrote something like that and used it on all but my very first > gigs. I attached the glue code and script here; initialization is at > the bottom. Please don't judge Lua's merits (or mine for that > matter:) based on this, it was never meant to be used/seen by anyone > but myself, is/was a work in progress and I frequently tweaked > scripts just before a gig, knowing full well it was pretty > risky/stupid. Nice effort! Maybe you could put that in a branch in bzr so its easier for people interested in Lua can grab it and maintain it or reuse parts of it. > >> If I were to critisize JavaScript, on the other hand, it would be >> because it is not always as concise and expressive as it could be. > > I'm fairly sure Lua is one of the most concise languages there is. > Its reference manual is 103 pages long, including the C API. Also Lua > 5.2 has a rich set of binary operators (for example bit range > extract, ROL/ROR) that'd help HID/MIDI processing. By concise I meant with regards of the code written in the language. Anyway, I understand the merits of Lua w.r.t performance and so on. However, I do not see these strong enough to compensate for the benefits of JavaScript (many people know it, other languages can be compiled to it, wide choice fo VMs, etc.) JP --
Re: [Mixxx-devel] Hate JavaScript and XML? ...then keep reading!
Hi Thomas, > Nice approach, Christian! Thanks! > while looking at the scripts, i thought it will be good to have something > like this, it will reduce code and make it more readable. That was my plan. :) > I see two sides here. The (hopefully complete) API that mixxx provides > (connect etc), some built-in convenience functions and the same freedom as > today to just use any valid ECMA script along with that. If more 'native' > functions will be needed (for instance, send/receive OSC), they will be > added and exposed to be used in scripts. For my actual controler script I have some controller specific stuff added to control the vuMeter leds. I'll write another blog post about this script tomorrow. :) Christian -- Try New Relic Now & We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, & servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ Get Mixxx, the #1 Free MP3 DJ Mixing software Today http://mixxx.org Mixxx-devel mailing list Mixxx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mixxx-devel
Re: [Mixxx-devel] Hate JavaScript and XML? ...then keep reading!
This is a largely academic debate so apologies to other list readers for being off-topic. On Sun, 26 May 2013 21:47:45 +0200 Juan Pedro Bolívar Puente wrote: > > What makes you say that? I see you have experience in just about > > every interpreter expect Lua (btw I live next door in Valencia if you > > want to take it outside:). > > I am based in Berlin now, so lets argue here instead :) So... Ableton or NI? I was offered a job but couldn't con them into telecommuting :) > By "real-time constraints" I mean the real-time constraints that are > usually required in the engine of most audio applications, which require > algorithms to be O(bufferSize). Even non-GC heap allocations do not > qualify for this, and because closures and dynamic typing can not be > implemented without heap allocations (and GC) Lua has very fine-grained GC control otherwise it'd never run inside embedded devices or early-gen game consoles. To the extent modern C++ will have /some/ memory allocation (otherwise it's really ANSI C), good GC control may be even safer; it's easier to suspend/resume a VM if some time/CPU quota was reached than native code holding a mutex. Coders that design Lua-enabled engines take a simple perspective: what parts make sense in C/C++, what do in Lua. If some Lua logic become overwhelmed just move it to C++. In ~12 years of using Lua I came across a bottleneck only once (handling enormous flight-simulator samples); I moved that logic to a Lua C "userdata" in half an hour then it was solved. Now, a badly-written script driver can always do something stupid to abuse the VM but unlike web browsers, Mixxx isn't supposed to protect against hostile 3rd-party scripts. Although Wikipedia is switching to Lua and does provide such protections. > I don't think one can run Lua code in the engine threads. Do you mean secondary threads? Lua can definitely do that; I wrote a TUIO extension that ran Lua in its own thread and Lua context, while the main thread ran another Lua context. IIRC TUIO is OSC (or MIDI?) over UDP so you can't afford to lose packets either. Also, Lua has 'coroutines' or cooperative multitasking so you can run multi-thread logic in a single OS thread; Lua 5.2 can yield across C boundaries too. I know it's an old concept but everybody went all oh and ah about Go's 'goroutines' (seriously). > And if one can not run Lua code in the > engine threads, there is going to be an added latency anyway to send the > messages accross the engine boundaries, as long as the interpreter is > not really bad, I do not think this is a big issue. There's always /some/ latency to the extent you're running any CPU instructions. Typically, Lua-based engines use custom data types whose memory layout is defined in C, so there's no significant memory juggling when crossing C/Lua boundaries or moving data from one Lua context to another. > I would suggest you to take a look at "asm.js", a subset of > JavaScript used to compile new-generation C++ games to JavaScript that > run at 2X the native version (and it is the whole game running in JS, > even the engine!). I know asm.js but frankly using a LLVM backend is no different than JIT. Any scripting language that can allocate & fill executable byte arrays could do the same, f.ex. the book "Lua programming gems" has a chapter using Fabrice Bellard's Tiny C Compiler (from boot-linux-from source code and Qemu fame) to embed native C code in Lua code. IMHO it's a pointless pursuit though; VMs should be sandboxed and prevented from generating new machine language at runtime. > > Lua's used by Reason and ReNoise and in game engines since the PS2 > > all the way to Crytek's Far Cry/Crysis which have similar > > frame-lock/tick constraints. There's a laundry list of realtime apps > > on www.lua.org/uses.html. Most tellingly, you'll find it in many > > embedded systems like Cisco, Barracuda, MicroTik RouterOS, OpenWrt, > > Wavecom GSM firmware. It hardly gets more realtime than that. > > Ok, that means its fast, but not real-time in the sense I mean, because > they do not have a deadline. I know, you mean "critical real-time". > Or to put otherwise: in Far Cry if a > single frame takes 50ms instead of an average of 16 (assuming 60fps) it > will most probably go unnoticed. Nope it's exactly the same problem; both audio and video have the concept of frames. Missing a video monitor's vertical blank refresh (VBL) to draw a frame is like missing an audio frame; in one case you have tearing because it'll show an old frame, potentially very old since 3D cards are heavily pipelined with several pre-buffered frame, in the other you have a dropout or if the audio card's buffers are looping it'll click. In a professional application, video tearing is an unacceptable as audio dropouts. Video games have very large audiences and budgets, larger than Hollywood movies. Locking to a video frequency/bit-depth is as critical as it is for
