Re: NEW: libmpack (needs discussion)
Edd Barrettwrites: > Hi, > > On Mon, Jan 23, 2017 at 09:56:04AM +, Stuart Henderson wrote: >> > The port looks fine, but I find all this FULLPKG-fu confusing. >> > Splitting the port would make things more readable IMHO. >> >> hmm, and that would make things a bit more sensible for building to support >> the multiple lua versions without having to re-build the whole thing. >> OK I have come round to that idea. > > Attached is the port revised to use SUBDIRs. > > Comments? OK? CPPFLAGS doesn't seem to be needed, please remove it. ok jca@ -- jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
Re: NEW: libmpack (needs discussion)
Hi, On Mon, Jan 23, 2017 at 09:56:04AM +, Stuart Henderson wrote: > > The port looks fine, but I find all this FULLPKG-fu confusing. > > Splitting the port would make things more readable IMHO. > > hmm, and that would make things a bit more sensible for building to support > the multiple lua versions without having to re-build the whole thing. > OK I have come round to that idea. Attached is the port revised to use SUBDIRs. Comments? OK? -- Best Regards Edd Barrett http://www.theunixzoo.co.uk libmpack-subdirs.tgz Description: application/tar-gz
Re: NEW: libmpack (needs discussion)
On 2017/01/22 22:46, Jeremie Courreges-Anglas wrote: > Stuart Hendersonwrites: > > $ FLAVOR=lua53 make show=FULLPKGPATH-lua > > devel/libmpack,-lua,lua53 > > $ FLAVOR=lua53 make show=PKGNAMES > > libmpack-1.0.3 lua53-mpack-1.0.3-lua53 > ^^ > This doesn't look right, I expect "lua53-mpack-1.0.3", which is what oops, why didn't I spot that... :/ > I get with Edd's tarball. I can't find a meaning for > "lua53-mpack-1.0.3-lua53": is it a lua53 FLAVOR of "lua53-mpack-1.0.3"? Edd's has the wrong pkgpath though. > > $ FLAVOR=lua53 make show=TEST_DEPENDS > > lua53-mpack-1.0.3-lua53:devel/libmpack,-lua,lua53 >^^ Same here. > > The port looks fine, but I find all this FULLPKG-fu confusing. > Splitting the port would make things more readable IMHO. hmm, and that would make things a bit more sensible for building to support the multiple lua versions without having to re-build the whole thing. OK I have come round to that idea.
Re: NEW: libmpack (needs discussion)
On Fri, Jan 20, 2017 at 05:59:19PM +0100, Florian Stinglmayr wrote: > > There is a test suite but it needs a load of lua deps we don't yet have > > Which ones? Lua deps are my speciality. It uses a test framework called "busted". Here are the deps from upstream's Makefile: https://github.com/tarruda/libmpack/blob/master/binding/lua/Makefile#L88 -- Best Regards Edd Barrett http://www.theunixzoo.co.uk
Re: NEW: libmpack (needs discussion)
Hi Stuart, Jeremie, On Sun, Jan 22, 2017 at 10:46:33PM +0100, Jeremie Courreges-Anglas wrote: > Stuart Hendersonwrites: > >> I have a feeling that the FULLPKGPATH-lua needs a version suffix, but > >> couldn't find a satisfying way (although what I currently have doesn't > >> seem to cause any superficial issues). Any ideas? > > If you're talking about having an additional -lua52/-lua53 suffix, then > I can't see a reason for it and I'd find this confusing at best. No, I meant shouldn't FULLPKGPATH be something like `devel/libmpack,-lua52`. Not sure. > > $ FLAVOR=lua53 make show=FULLPKGPATH-lua > > devel/libmpack,-lua,lua53 > > $ FLAVOR=lua53 make show=PKGNAMES > > libmpack-1.0.3 lua53-mpack-1.0.3-lua53 > ^^ > This doesn't look right, I expect "lua53-mpack-1.0.3", which is what > I get with Edd's tarball. I can't find a meaning for > "lua53-mpack-1.0.3-lua53": is it a lua53 FLAVOR of "lua53-mpack-1.0.3"? > AFAIK no other port is named lua53-foo-lua53. Right. This is what happens if you use PKGNAME-lua and not FULLPKGNAME-lua. Also note that once you use a FULLPKGPATH-lua, you also get warnings that you need a FULLPKGPATH-lua (although it is just a warning). > The port looks fine, but I find all this FULLPKG-fu confusing. > Splitting the port would make things more readable IMHO. It's not the most easy to understand, I must admit. Clearly lua version flavors and MULTIPACKAGES don't play well together. I'm not against splitting out the port into SUBDIRS, where each fetches the same DISTFILE, but I'd also like to hear what Stuart thinks about this. > Also: > - please use > FLAVORS = lua52 lua53 > FLAVOR ?= > just like in other lua ports > - y u no like libmpack.la? :) I've fixed those locally, but await to hear back on whether to split the port out. Thanks again. -- Best Regards Edd Barrett http://www.theunixzoo.co.uk
Re: NEW: libmpack (needs discussion)
Stuart Hendersonwrites: > On 2017/01/22 16:50, Edd Barrett wrote: >> Hi, >> >> On Thu, Jan 19, 2017 at 10:07:30PM +0100, Jeremie Courreges-Anglas wrote: >> > > It seems to build for all the lua versions we support, so good :) >> > >> > This is often not enough: pure lua builds typically don't run the code >> > at build time, and C lua extensions might use lua C API functions only >> > available is some lua versions, with a failure to resolve missing >> > symbols at runtime. >> >> I've added a (minimal) test case, which seems to work for lua51-53, >> which is good :) >> >> > >> Regarding the port: better use "lua-mpack" or "lua-libmpack" as >> > >> PKGNAME-lua, for consistency and because the lang/lua module uses this >> > >> to provide automatic support for multiple versions. >> >> I think I got this (almost) right, but I had to jump through some hoops >> to get the lua subpackages with the right name (I think complicated by >> the fact that the lua bindings are MULTI_PACKAGES). >> >> I have a feeling that the FULLPKGPATH-lua needs a version suffix, but >> couldn't find a satisfying way (although what I currently have doesn't >> seem to cause any superficial issues). Any ideas? If you're talking about having an additional -lua52/-lua53 suffix, then I can't see a reason for it and I'd find this confusing at best. >> Updated tarball attached. >> >> -- >> Best Regards >> Edd Barrett >> >> http://www.theunixzoo.co.uk > > > > I think you can get rid of the FULLPKGPATH-lua and use PKGNAME-lua > instead of FULLPKGNAME-lua. And adjust TEST_DEPENDS: > > $ grep PKG.*lua Makefile > PKGNAME-lua = ${_MODLUA_PKG_PREFIX}-mpack-${V} > TEST_DEPENDS += ${PKGNAME-lua}:${FULLPKGPATH-lua} > > This gives these, which seem right to me : > > $ make show=FULLPKGPATH-lua > devel/libmpack,-lua > $ make show=PKGNAMES > libmpack-1.0.3 lua-mpack-1.0.3 > $ make show=TEST_DEPENDS > lua-mpack-1.0.3:devel/libmpack,-lua > > $ FLAVOR=lua53 make show=FULLPKGPATH-lua > devel/libmpack,-lua,lua53 > $ FLAVOR=lua53 make show=PKGNAMES > libmpack-1.0.3 lua53-mpack-1.0.3-lua53 ^^ This doesn't look right, I expect "lua53-mpack-1.0.3", which is what I get with Edd's tarball. I can't find a meaning for "lua53-mpack-1.0.3-lua53": is it a lua53 FLAVOR of "lua53-mpack-1.0.3"? AFAIK no other port is named lua53-foo-lua53. > $ FLAVOR=lua53 make show=TEST_DEPENDS > lua53-mpack-1.0.3-lua53:devel/libmpack,-lua,lua53 ^^ Same here. The port looks fine, but I find all this FULLPKG-fu confusing. Splitting the port would make things more readable IMHO. Also: - please use FLAVORS = lua52 lua53 FLAVOR ?= just like in other lua ports - y u no like libmpack.la? :) -- jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
Re: NEW: libmpack (needs discussion)
On 2017/01/22 16:50, Edd Barrett wrote: > Hi, > > On Thu, Jan 19, 2017 at 10:07:30PM +0100, Jeremie Courreges-Anglas wrote: > > > It seems to build for all the lua versions we support, so good :) > > > > This is often not enough: pure lua builds typically don't run the code > > at build time, and C lua extensions might use lua C API functions only > > available is some lua versions, with a failure to resolve missing > > symbols at runtime. > > I've added a (minimal) test case, which seems to work for lua51-53, > which is good :) > > > >> Regarding the port: better use "lua-mpack" or "lua-libmpack" as > > >> PKGNAME-lua, for consistency and because the lang/lua module uses this > > >> to provide automatic support for multiple versions. > > I think I got this (almost) right, but I had to jump through some hoops > to get the lua subpackages with the right name (I think complicated by > the fact that the lua bindings are MULTI_PACKAGES). > > I have a feeling that the FULLPKGPATH-lua needs a version suffix, but > couldn't find a satisfying way (although what I currently have doesn't > seem to cause any superficial issues). Any ideas? > > Updated tarball attached. > > -- > Best Regards > Edd Barrett > > http://www.theunixzoo.co.uk I think you can get rid of the FULLPKGPATH-lua and use PKGNAME-lua instead of FULLPKGNAME-lua. And adjust TEST_DEPENDS: $ grep PKG.*lua Makefile PKGNAME-lua = ${_MODLUA_PKG_PREFIX}-mpack-${V} TEST_DEPENDS += ${PKGNAME-lua}:${FULLPKGPATH-lua} This gives these, which seem right to me : $ make show=FULLPKGPATH-lua devel/libmpack,-lua $ make show=PKGNAMES libmpack-1.0.3 lua-mpack-1.0.3 $ make show=TEST_DEPENDS lua-mpack-1.0.3:devel/libmpack,-lua $ FLAVOR=lua53 make show=FULLPKGPATH-lua devel/libmpack,-lua,lua53 $ FLAVOR=lua53 make show=PKGNAMES libmpack-1.0.3 lua53-mpack-1.0.3-lua53 $ FLAVOR=lua53 make show=TEST_DEPENDS lua53-mpack-1.0.3-lua53:devel/libmpack,-lua,lua53
Re: NEW: libmpack (needs discussion)
Hi, On Thu, Jan 19, 2017 at 10:07:30PM +0100, Jeremie Courreges-Anglas wrote: > > It seems to build for all the lua versions we support, so good :) > > This is often not enough: pure lua builds typically don't run the code > at build time, and C lua extensions might use lua C API functions only > available is some lua versions, with a failure to resolve missing > symbols at runtime. I've added a (minimal) test case, which seems to work for lua51-53, which is good :) > >> Regarding the port: better use "lua-mpack" or "lua-libmpack" as > >> PKGNAME-lua, for consistency and because the lang/lua module uses this > >> to provide automatic support for multiple versions. I think I got this (almost) right, but I had to jump through some hoops to get the lua subpackages with the right name (I think complicated by the fact that the lua bindings are MULTI_PACKAGES). I have a feeling that the FULLPKGPATH-lua needs a version suffix, but couldn't find a satisfying way (although what I currently have doesn't seem to cause any superficial issues). Any ideas? Updated tarball attached. -- Best Regards Edd Barrett http://www.theunixzoo.co.uk libmpack2.tgz Description: application/tar-gz
Re: NEW: libmpack (needs discussion)
On Thu, Jan 19, 2017 at 09:21:42PM +, Edd Barrett wrote: > On Thu, Jan 19, 2017 at 10:07:30PM +0100, Jeremie Courreges-Anglas wrote: > > The best way to know if things are right is probably to have a test > > target for the lua module. ;) > > There is a test suite but it needs a load of lua deps we don't yet have Which ones? Lua deps are my speciality. > in tree. I think I'll do as you suggest and make a minimal test target > of my own. > > > Do you have any reason to remove libmpack.la? > > Not really. I'll bring it back.
Re: NEW: libmpack (needs discussion)
On Thu, Jan 19, 2017 at 10:07:30PM +0100, Jeremie Courreges-Anglas wrote: > The best way to know if things are right is probably to have a test > target for the lua module. ;) There is a test suite but it needs a load of lua deps we don't yet have in tree. I think I'll do as you suggest and make a minimal test target of my own. > Do you have any reason to remove libmpack.la? Not really. I'll bring it back. -- Best Regards Edd Barrett http://www.theunixzoo.co.uk
Re: NEW: libmpack (needs discussion)
On Thu, Jan 19, 2017 at 08:49:28PM +, Stuart Henderson wrote: > FULLPKGNAME-main= libmpack-$V > FULLPKGPATH-main= devel/libmpack,-main Nicley done! That's the ticket. -- Best Regards Edd Barrett http://www.theunixzoo.co.uk
Re: NEW: libmpack (needs discussion)
Edd Barrettwrites: > Hi Jeremie, Stuart, > > Thanks for the info and suggestions. > > On Thu, Jan 19, 2017 at 06:03:10PM +0100, Jeremie Courreges-Anglas wrote: >> LuaJIT is not portable. >> >> ONLY_FOR_ARCHS = powerpc i386 amd64 >> >> IMHO this is the kind of fragile software you'd better avoid if you can. > > Maybe we can try using regular lua until users (or ourselves) start > complaining about performance. That seems like a decent incremental step > forward. > >> Please try to support as many versions as supported by upstream. If you >> find out that only 5.1 is supported, then setting MODLUA_VERSION to 5.1 >> makes sense to save other people's time (at least, that's the current >> pattern I see). > > It seems to build for all the lua versions we support, so good :) This is often not enough: pure lua builds typically don't run the code at build time, and C lua extensions might use lua C API functions only available is some lua versions, with a failure to resolve missing symbols at runtime. The best way to know if things are right is probably to have a test target for the lua module. ;) >> Regarding the port: better use "lua-mpack" or "lua-libmpack" as >> PKGNAME-lua, for consistency and because the lang/lua module uses this >> to provide automatic support for multiple versions. > > I've renamed the lua MULTI_PACKAGE to lua-mpack, and added the lua > flavors. Although this mostly works, the problem is, you get one flavor > of the -main MULTI_PACKAGE (the C library, nothing to do with lua) for > each lua version. > > Is there a way to only FLAVOR the -lua MULTI_PACKAGE? I suspect not. No way that I am aware of, but I didn't really look the details. > In > which case I suppose I will use sub-dirs that fetch the same DISTFILE > with a Makefile.inc. > > What do you think? Something like libmpack/ Makefile.inc libmpack/ lua-libmpack/ would be a possible workaround indeed. And it will probably help lang/lua set the FULLPKGNAME according to the lua FLAVOR. Of course, make sure you use MODLUA_WANTLIB. Do you have any reason to remove libmpack.la? -- jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
Re: NEW: libmpack (needs discussion)
On 2017/01/19 20:25, Edd Barrett wrote: > Hi Jeremie, Stuart, > > Thanks for the info and suggestions. > > On Thu, Jan 19, 2017 at 06:03:10PM +0100, Jeremie Courreges-Anglas wrote: > > LuaJIT is not portable. > > > > ONLY_FOR_ARCHS = powerpc i386 amd64 > > > > IMHO this is the kind of fragile software you'd better avoid if you can. > > Maybe we can try using regular lua until users (or ourselves) start > complaining about performance. That seems like a decent incremental step > forward. Or use luajit on supported arches (luajit itself supports arm and mips as well as x86/powerpc), and interpreted lua on others.. btw I am using luajit in production now, rspamd wants it - I haven't noticed any fragility, it's just not particularly portable.. > > Please try to support as many versions as supported by upstream. If you > > find out that only 5.1 is supported, then setting MODLUA_VERSION to 5.1 > > makes sense to save other people's time (at least, that's the current > > pattern I see). > > It seems to build for all the lua versions we support, so good :) > > > Regarding the port: better use "lua-mpack" or "lua-libmpack" as > > PKGNAME-lua, for consistency and because the lang/lua module uses this > > to provide automatic support for multiple versions. > > I've renamed the lua MULTI_PACKAGE to lua-mpack, and added the lua > flavors. Although this mostly works, the problem is, you get one flavor > of the -main MULTI_PACKAGE (the C library, nothing to do with lua) for > each lua version. > > Is there a way to only FLAVOR the -lua MULTI_PACKAGE? I suspect not. In > which case I suppose I will use sub-dirs that fetch the same DISTFILE > with a Makefile.inc. Yes, override FULLPKGPATH-main and FULLPKGNAME-main. FULLPKGNAME-main= libmpack-$V FULLPKGPATH-main= devel/libmpack,-main I'm not sure but you may need ".include " first. Do some "make show=PKGNAMES", "FLAVOR=xx make show=PKGNAMES" etc to test.
Re: NEW: libmpack (needs discussion)
Hi Jeremie, Stuart, Thanks for the info and suggestions. On Thu, Jan 19, 2017 at 06:03:10PM +0100, Jeremie Courreges-Anglas wrote: > LuaJIT is not portable. > > ONLY_FOR_ARCHS = powerpc i386 amd64 > > IMHO this is the kind of fragile software you'd better avoid if you can. Maybe we can try using regular lua until users (or ourselves) start complaining about performance. That seems like a decent incremental step forward. > Please try to support as many versions as supported by upstream. If you > find out that only 5.1 is supported, then setting MODLUA_VERSION to 5.1 > makes sense to save other people's time (at least, that's the current > pattern I see). It seems to build for all the lua versions we support, so good :) > Regarding the port: better use "lua-mpack" or "lua-libmpack" as > PKGNAME-lua, for consistency and because the lang/lua module uses this > to provide automatic support for multiple versions. I've renamed the lua MULTI_PACKAGE to lua-mpack, and added the lua flavors. Although this mostly works, the problem is, you get one flavor of the -main MULTI_PACKAGE (the C library, nothing to do with lua) for each lua version. Is there a way to only FLAVOR the -lua MULTI_PACKAGE? I suspect not. In which case I suppose I will use sub-dirs that fetch the same DISTFILE with a Makefile.inc. What do you think? -- Best Regards Edd Barrett http://www.theunixzoo.co.uk
Re: NEW: libmpack (needs discussion)
Edd Barrettwrites: > Hi, > > (CCing some people who have worked on lang/lua, hoping one of them knows > the answer to my question below). I don't consider I have much experience with lua, but I recently found myself untangling the versioning mess we had in the ports tree. Some additional details after Stuart's mail. > This is (yet another) neovim dependency. libmpack (not to be confused > with libmsgpack): > > ---8<--- > libmpack is a small binary serialization/RPC library that implements both the > msgpack and msgpack-rpc specifications. > --->8--- > > This isn't quite ready as it requires some thought. Neovim (on some > arches) will use this via LuaJIT, not plain Lua. We don't have to do > anything special to build against LuaJIT as plain Lua extensions are > binary compatible with LuaJIT [1]. LuaJIT is not portable. ONLY_FOR_ARCHS = powerpc i386 amd64 IMHO this is the kind of fragile software you'd better avoid if you can. > What I don't think will work is mixing Lua versions. So how do we guard > against the case where (e.g.) the default Lua is bumped to target a > newer Lua than LuaJIT? This means the library will not be found by > LuaJIT. > > Although I could explicitly set the Lua version used in this port, > setting it to anything other than the default version seems wrong too, > since then regular Lua users won't have access. lang/lua doesn't add FLAVORS automatically any more. If you want to use a port with lua52 or lua53, you have to test and set FLAVORS explicitely anyway. > Do we need (versioned) lua flavors, e.g. -lua51, -lua52? Please try to support as many versions as supported by upstream. If you find out that only 5.1 is supported, then setting MODLUA_VERSION to 5.1 makes sense to save other people's time (at least, that's the current pattern I see). > We could just use plain Lua across the board, but that would be a shame. If you can selectively use LuaJIT on *some* platforms, well, go for it. But I doubt that using LuaJIT will make an actual difference regarding speed... > One of the attractions of neovim is to use a higher performance (and > a less "argh, my eyes, they burn") scripting language than vimscript. > > Any ideas, or other comments? Regarding the port: better use "lua-mpack" or "lua-libmpack" as PKGNAME-lua, for consistency and because the lang/lua module uses this to provide automatic support for multiple versions. > [1] http://luajit.org/extensions.html -- jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
Re: NEW: libmpack (needs discussion)
We already have infrastructure for multiple flavours of lua ports, so I'd just use that. If the default lua version is changed we'll need a wider diff anyway because we'll need to switch from "unflavoured = 5.1" / "lua-XX prefix = 5.1" to something else, and that will need handling across the whole tree, so I wouldn't worry about that situation here. -- Sent from a phone, apologies for poor formatting. On 17 January 2017 23:56:14 Edd Barrettwrote: Hi, (CCing some people who have worked on lang/lua, hoping one of them knows the answer to my question below). This is (yet another) neovim dependency. libmpack (not to be confused with libmsgpack): ---8<--- libmpack is a small binary serialization/RPC library that implements both the msgpack and msgpack-rpc specifications. --->8--- This isn't quite ready as it requires some thought. Neovim (on some arches) will use this via LuaJIT, not plain Lua. We don't have to do anything special to build against LuaJIT as plain Lua extensions are binary compatible with LuaJIT [1]. What I don't think will work is mixing Lua versions. So how do we guard against the case where (e.g.) the default Lua is bumped to target a newer Lua than LuaJIT? This means the library will not be found by LuaJIT. Although I could explicitly set the Lua version used in this port, setting it to anything other than the default version seems wrong too, since then regular Lua users won't have access. Do we need (versioned) lua flavors, e.g. -lua51, -lua52? We could just use plain Lua across the board, but that would be a shame. One of the attractions of neovim is to use a higher performance (and a less "argh, my eyes, they burn") scripting language than vimscript. Any ideas, or other comments? [1] http://luajit.org/extensions.html -- Best Regards Edd Barrett http://www.theunixzoo.co.uk
NEW: libmpack (needs discussion)
Hi, (CCing some people who have worked on lang/lua, hoping one of them knows the answer to my question below). This is (yet another) neovim dependency. libmpack (not to be confused with libmsgpack): ---8<--- libmpack is a small binary serialization/RPC library that implements both the msgpack and msgpack-rpc specifications. --->8--- This isn't quite ready as it requires some thought. Neovim (on some arches) will use this via LuaJIT, not plain Lua. We don't have to do anything special to build against LuaJIT as plain Lua extensions are binary compatible with LuaJIT [1]. What I don't think will work is mixing Lua versions. So how do we guard against the case where (e.g.) the default Lua is bumped to target a newer Lua than LuaJIT? This means the library will not be found by LuaJIT. Although I could explicitly set the Lua version used in this port, setting it to anything other than the default version seems wrong too, since then regular Lua users won't have access. Do we need (versioned) lua flavors, e.g. -lua51, -lua52? We could just use plain Lua across the board, but that would be a shame. One of the attractions of neovim is to use a higher performance (and a less "argh, my eyes, they burn") scripting language than vimscript. Any ideas, or other comments? [1] http://luajit.org/extensions.html -- Best Regards Edd Barrett http://www.theunixzoo.co.uk libmpack.tgz Description: application/tar-gz