On Sunday 10, Benoit Germain wrote: > 2011/4/10 Robert G. Jakabosky <[email protected]>: > > Announcing lua-llthreads 1.0 release. > > Hello, > > Since I maintain Lanes, I can't help but reflect that this is some > sort of a Lanes subset where the inter-thread communication lindas are > gone. But I suppose that zeromq would be usable with Lanes threads?
Originally this module had been part of my lua-zmq project, but the first person I showed it to said that it would be interesting to see it as s separate module (since it wasn't dependent on zmq). > Also your documentation seems to show that the thread function must be > provided as unparsed Lua code that will be compiled in the target VM. This is a Low-Level threads module. People can use string.dump() or some other serialization library if they want to pass complex Lua values to the child thread. I wanted to keep this module as simple as possible. > Lanes also offers this possibility, but the only reason is to start > the path toward LuaJIT compatibility, since the other way (using > lua_dump to move the function proto) isn't available in LuaJIT. Is > this why you have restricted to this solution? lua-llthreads & lua-zmq work with LuaJIT2. Lua-zmq is also optimized to use LJ2's FFI support. > Also, since you have > felt the need for llthreads, there is probably some use case that is > not properly adressed by Lanes. What would that be? Lanes doesn't fit my needs like ZeroMQ does. I need a message passing library that handles thread-to-thread & host-to-host communication and that supports multiple languages [1]. Also most of the programs I write are network services that use event loops for non-blocking network IO and ZeroMQ sockets can be used in an event loop like normal sockets. I ported the latency & throughput benchmarks from lua-zmq to the Lanes interface (see the two attached lua scripts). I wasn't able to get Lanes to work on LuaJIT2 even when using unparsed Lua code for the thread function. Latency: Message size: 512 bytes roundtrip count: 1,000,000 Lanes 2.1 Lua 5.1.4: mean latency: 10.318 microseconds. ZeroMQ Lua 5.1.4: mean latency: 7.027 microseconds ZeroMQ LuaJIT2: mean latency: 6.280 microseconds Throughput for Lanes is much lower then ZeroMQ: Message size: 512 bytes message count: 1,000,000 Lanes 2.1 Lua 5.1.4: mean throughput: 97,141 msg/second mean throughput: 398 MBits/second ZeroMQ Lua 5.1.4: mean throughput: 1,275,092 msg/second mean throughput: 5,223 MBits/second ZeroMQ LuaJIT2: mean throughput: 1,777,966 msg/second mean throughput: 7,283 MBits/second For Lanes I had to tweak the queue limit, since it was taking way to long with the default queue length. For smaller messages (1-30 bytes) ZeroMQ+LuaJIT2 can get above 4Million message per second. The above tests where run on Linux 2.6.36 x86_64 CPU: AMD Athlon II X2 250 (Dual-core 3Ghz). Also sometime the lanes_*.lua scripts would crash with a double free error (See the attached lanes_lat_errors.log for the output from valgrind). 1. http://www.zeromq.org/bindings:_start -- Robert G. Jakabosky
-- Copyright (c) 2011 Robert G. Jakabosky <[email protected]> -- -- Permission is hereby granted, free of charge, to any person obtaining a copy -- of this software and associated documentation files (the "Software"), to deal -- in the Software without restriction, including without limitation the rights -- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -- copies of the Software, and to permit persons to whom the Software is -- furnished to do so, subject to the following conditions: -- -- The above copyright notice and this permission notice shall be included in -- all copies or substantial portions of the Software. -- -- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -- THE SOFTWARE. if #arg < 1 then print("usage: lua " .. arg[0] .. " [message-size] [roundtrip-count]") end local message_size = tonumber(arg[1] or 1) local roundtrip_count = tonumber(arg[2] or 100000) require "lanes" local linda= lanes.linda() local socket = require"socket" local time = socket.gettime local function child_code() local msg for i = 1, roundtrip_count do msg = assert(linda:receive("x")) assert(#msg == message_size, "Invalid message size") assert(linda:send("y", msg)) end end local child_thread = lanes.gen("", child_code)() print("child:", child_thread) local msg = ("0"):rep(message_size) print(string.format("message size: %i [B]", message_size)) print(string.format("roundtrip count: %i", roundtrip_count)) local start_time = time() for i = 1, roundtrip_count do assert(linda:send("x", msg)) msg = assert(linda:receive("y")) assert(#msg == message_size, "Invalid message size") end local end_time = time() print("child.join:",child_thread:join()) local elapsed = end_time - start_time local latency = elapsed * 1000000 / roundtrip_count / 2 print(string.format("mean latency: %.3f [us]", latency))
child: userdata: 0x5b637c8 message size: 30 [B] roundtrip count: 1000 child.join: mean latency: 421.994 [us] ==19730== Thread 2: ==19730== Invalid read of size 8 ==19730== at 0x5ED8F12: lane_main (lanes.c:856) ==19730== by 0x60E8D59: start_thread (pthread_create.c:301) ==19730== Address 0x5b36be8 is 136 bytes inside a block of size 144 free'd ==19730== at 0x4C278CD: free (vg_replace_malloc.c:366) ==19730== by 0x5ED9323: selfdestruct_atexit (lanes.c:1004) ==19730== by 0x552F8B4: __cxa_finalize (cxa_finalize.c:56) ==19730== by 0x5ED6925: ??? (in /usr/lib64/lua/luarocks/lib/lua/5.1/lua51-lanes.so) ==19730== by 0x5EDD420: ??? (in /usr/lib64/lua/luarocks/lib/lua/5.1/lua51-lanes.so) ==19730== by 0x401378D: _dl_close (dl-close.c:747) ==19730== by 0x400DA68: _dl_catch_error (dl-error.c:178) ==19730== by 0x50B336C: _dlerror_run (dlerror.c:164) ==19730== by 0x50B305E: dlclose (dlclose.c:48) ==19730== by 0x42C4BF: gctm (loadlib.c:64) ==19730== by 0x40B450: luaD_precall (ldo.c:319) ==19730== by 0x40BA72: luaD_call (ldo.c:376) ==19730== ==19730== Invalid free() / delete / delete[] ==19730== at 0x4C278CD: free (vg_replace_malloc.c:366) ==19730== by 0x5ED8F7D: lane_main (lanes.c:1329) ==19730== by 0x60E8D59: start_thread (pthread_create.c:301) ==19730== Address 0x5b36b60 is 0 bytes inside a block of size 144 free'd ==19730== at 0x4C278CD: free (vg_replace_malloc.c:366) ==19730== by 0x5ED9323: selfdestruct_atexit (lanes.c:1004) ==19730== by 0x552F8B4: __cxa_finalize (cxa_finalize.c:56) ==19730== by 0x5ED6925: ??? (in /usr/lib64/lua/luarocks/lib/lua/5.1/lua51-lanes.so) ==19730== by 0x5EDD420: ??? (in /usr/lib64/lua/luarocks/lib/lua/5.1/lua51-lanes.so) ==19730== by 0x401378D: _dl_close (dl-close.c:747) ==19730== by 0x400DA68: _dl_catch_error (dl-error.c:178) ==19730== by 0x50B336C: _dlerror_run (dlerror.c:164) ==19730== by 0x50B305E: dlclose (dlclose.c:48) ==19730== by 0x42C4BF: gctm (loadlib.c:64) ==19730== by 0x40B450: luaD_precall (ldo.c:319) ==19730== by 0x40BA72: luaD_call (ldo.c:376) ==19730== Killed 1 lane(s) at process end.
-- Copyright (c) 2011 Robert G. Jakabosky <[email protected]> -- -- Permission is hereby granted, free of charge, to any person obtaining a copy -- of this software and associated documentation files (the "Software"), to deal -- in the Software without restriction, including without limitation the rights -- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -- copies of the Software, and to permit persons to whom the Software is -- furnished to do so, subject to the following conditions: -- -- The above copyright notice and this permission notice shall be included in -- all copies or substantial portions of the Software. -- -- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -- THE SOFTWARE. if #arg < 1 then print("usage: lua " .. arg[0] .. " [message-size] [message-count]") end local message_size = tonumber(arg[1] or 1) local message_count = tonumber(arg[2] or 100000) require "lanes" local linda= lanes.linda() linda:limit("x", 100) local socket = require"socket" local time = socket.gettime local function child_code() local socket = require"socket" local time = socket.gettime local msg = ("0"):rep(message_size) local start_time = time() for i = 1, message_count do linda:send("x", msg) end local end_time = time() local elapsed = end_time - start_time if elapsed == 0 then elapsed = 1 end local throughput = message_count / elapsed local megabits = throughput * message_size * 8 / 1000000 print(string.format("Sender mean throughput: %i [msg/s]", throughput)) print(string.format("Sender mean throughput: %.3f [Mb/s]", megabits)) print("sending thread finished.") end local child_thread = lanes.gen("*", child_code)() print("child:", child_thread) print(string.format("message size: %i [B]", message_size)) print(string.format("message count: %i", message_count)) local msg msg = linda:receive("x") local start_time = time() for i = 1, message_count - 1 do msg = linda:receive(3.0, "x") assert(#msg == message_size, "Invalid message size") end local end_time = time() print("child.join:",child_thread:join()) local elapsed = end_time - start_time if elapsed == 0 then elapsed = 1 end local throughput = message_count / elapsed local megabits = throughput * message_size * 8 / 1000000 print(string.format("mean throughput: %i [msg/s]", throughput)) print(string.format("mean throughput: %.3f [Mb/s]", megabits))
_______________________________________________ Luarocks-developers mailing list [email protected] http://lists.luaforge.net/cgi-bin/mailman/listinfo/luarocks-developers
