Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
Should be fixed now. ATM, I don't have any code that's using the prepared statements feature, and I need to head out, so can you verify it? Yes, with those changes everything I'm working with now works as expected.
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Mon, 20 May 2013 07:28:29 +0200 dennis luehring dl.so...@gmx.net wrote: Am 18.05.2013 17:42, schrieb Gary Willoughby: Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code. wouldn't it be a good idea to port the official mysql_client_test.c or parts of it into the unit test? its huge but seem to test every (corner) case around http://bazaar.launchpad.net/~mysql/mysql-server/5.7/view/head:/tests/mysql_client_test.c I'm assuming that's [L]GPL, right? If so, then I don't think it could be used in whole, in part, or in any modified form, in mysql-native (Boost license). However, maybe it could safely be done as a separate tester for mysql-native project? (Of course, IANAL, so I don't know what the hell I'm talking about. YMMV. OMG WTF BBQ. Etc...)
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
Fixed. Removing the assert turned out to be the correct solution after all. The assert was based on a faulty assumption, and none of the code actually relied on the assertion passing (see the ticket above for more detailed info). Great, thanks Nick. I have however found a few other issues. Using this code: auto connection = new Connection(MySQLSocketType.phobos, 127.0.0.1, root, r00t, Project); auto command= new Command(connection, INSERT INTO tblUser (name, age) VALUES (?, ?)); command.prepare(); ulong rowsAffected; string name; foreach (int age; 0..10) { name = Test User; command.bindParameter(name, 0); command.bindParameter(age, 1); command.execPrepared(rowsAffected); } Raises two simple to fix errors in the connection code. 1). mysql/connection.d(3845): Error: more initializers than fields (5) of ParameterSpecialization Basically the initialisation of the structure contains too many parameters. 2). mysql/connection.d(3858): Error: no property 'dummy' for type 'ParameterSpecialization' The structure doesn't contain a 'dummy' property. Debug info i guess?
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Sun, 19 May 2013 12:43:37 +0200 Gary Willoughby d...@kalekold.net wrote: Fixed. Removing the assert turned out to be the correct solution after all. The assert was based on a faulty assumption, and none of the code actually relied on the assertion passing (see the ticket above for more detailed info). Great, thanks Nick. I have however found a few other issues. Using this code: auto connection = new Connection(MySQLSocketType.phobos, 127.0.0.1, root, r00t, Project); auto command= new Command(connection, INSERT INTO tblUser (name, age) VALUES (?, ?)); command.prepare(); ulong rowsAffected; string name; foreach (int age; 0..10) { name = Test User; command.bindParameter(name, 0); command.bindParameter(age, 1); command.execPrepared(rowsAffected); } Raises two simple to fix errors in the connection code. 1). mysql/connection.d(3845): Error: more initializers than fields (5) of ParameterSpecialization Basically the initialisation of the structure contains too many parameters. 2). mysql/connection.d(3858): Error: no property 'dummy' for type 'ParameterSpecialization' The structure doesn't contain a 'dummy' property. Debug info i guess? Thanks, I'll look into these...
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Sun, 19 May 2013 17:17:47 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: 1). mysql/connection.d(3845): Error: more initializers than fields (5) of ParameterSpecialization Basically the initialisation of the structure contains too many parameters. 2). mysql/connection.d(3858): Error: no property 'dummy' for type 'ParameterSpecialization' The structure doesn't contain a 'dummy' property. Debug info i guess? Thanks, I'll look into these... That actually explains some changes I saw in one fork of mysqln that I was puzzled about. I have a feeling this dummy thing is a debugging-related relic from an earlier version that was removed incompletely. I'm going to see if I can find any history behind it in the earlier revisions. Likely I'll just end up removing the remaining traces of it: It doesn't appear to be serving any real purpose now, and it is the cause of both of those errors above.
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Sun, 19 May 2013 17:17:47 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Sun, 19 May 2013 12:43:37 +0200 Gary Willoughby d...@kalekold.net wrote: 1). mysql/connection.d(3845): Error: more initializers than fields (5) of ParameterSpecialization Basically the initialisation of the structure contains too many parameters. 2). mysql/connection.d(3858): Error: no property 'dummy' for type 'ParameterSpecialization' The structure doesn't contain a 'dummy' property. Debug info i guess? Thanks, I'll look into these... Should be fixed now. ATM, I don't have any code that's using the prepared statements feature, and I need to head out, so can you verify it?
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
Am 18.05.2013 17:42, schrieb Gary Willoughby: Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code. wouldn't it be a good idea to port the official mysql_client_test.c or parts of it into the unit test? its huge but seem to test every (corner) case around http://bazaar.launchpad.net/~mysql/mysql-server/5.7/view/head:/tests/mysql_client_test.c
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code. MySql schema: CREATE TABLE tblUser ( id int(11) NOT NULL AUTO_INCREMENT, name varchar(255) NOT NULL, age varchar(45) NOT NULL, date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (id), UNIQUE KEY (name) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 D code: import mysql.connection; import std.stdio; void main(string[] arguments) { auto connection = new Connection(MySQLSocketType.phobos, 127.0.0.1, root, r00t, Project); auto command= new Command(connection, SELECT * FROM tblUser); auto results= command.execSQLResult(); foreach (row; results) { writeln(row); } connection.close(); } Error: core.exception.AssertError@mysql.connection(1046): Assertion failure /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(_d_assertm+0x26) [0x4a6a06] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main() [0x4a38ce] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.SQLValue mysql.connection.consumeNonBinaryValueIfComplete!(std.datetime.DateTime).consumeNo nBinaryValueIfComplete(ref ubyte[], bool)+0x126) [0x49e8ee] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.SQLValue mysql.connection.consumeIfComplete!(std.datetime.DateTime, 8).consumeIfComplete(re f ubyte[], bool, bool)+0x36) [0x49ded2] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.SQLValue mysql.connection.consumeIfComplete!().consumeIfComplete(ref ubyte[], mysql.connect ion.SQLType, bool, bool)+0x308) [0x494efc] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.Row mysql.connection.Row.__ctor(mysql.connection.Connection, ref ubyte[], mysql.connection. ResultSetHeaders, bool)+0x221) [0x48be95] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.ResultSet mysql.connection.Command.execSQLResult(mysql.connection.ColumnSpecialization[])+0 x1a3) [0x490f13] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(_Dmain+0xc9) [0x469a19] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void runMain()+0x18) [ 0x4a71c8] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void tryExec(scope voi d delegate())+0x2a) [0x4a6d06] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void runAll()+0x3b) [0 x4a720f] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void tryExec(scope voi d delegate())+0x2a) [0x4a6d06] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(_d_run_main+0x1a8) [0x4a6cc0] /tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(main+0x17) [0x4a6b13] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f391473476d] core.exception.InvalidMemoryOperationError If i comment out the assert on line 1046 in connection.d everything seems to work fine. Any thoughts?
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Sat, 18 May 2013 17:42:24 +0200 Gary Willoughby d...@kalekold.net wrote: Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code. Hmm, I'll look into it...
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Sat, 18 May 2013 14:30:13 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Sat, 18 May 2013 17:42:24 +0200 Gary Willoughby d...@kalekold.net wrote: Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code. Hmm, I'll look into it... The problem is triggered by retrieving a TIMESTAMP value (although DATETIME works fine). I've filed the issue here: https://github.com/rejectedsoftware/mysql-native/issues/9 I'm still looking into it.
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Sat, 18 May 2013 15:11:24 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Sat, 18 May 2013 14:30:13 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Sat, 18 May 2013 17:42:24 +0200 Gary Willoughby d...@kalekold.net wrote: Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code. Hmm, I'll look into it... The problem is triggered by retrieving a TIMESTAMP value (although DATETIME works fine). I've filed the issue here: https://github.com/rejectedsoftware/mysql-native/issues/9 I'm still looking into it. Fixed. Removing the assert turned out to be the correct solution after all. The assert was based on a faulty assumption, and none of the code actually relied on the assertion passing (see the ticket above for more detailed info).
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Tue, 14 May 2013 15:37:50 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Tue, 14 May 2013 04:35:20 +0200 Nathan M. Swan nathanms...@gmail.com wrote: preclude a design like this? - interface MysqlSocket { // ... } class PhobosSocket : MysqlSocket { this(std.socket.Socket _backend) { // ... } // ... } class VibeSocket : MysqlSocket { this(vibe.core.net.TCPConnection _backend) { // ... } } - That's actually somewhat similar to what I ended up doing internally: I created a minimal wrapper for std.socket.Socket to expose a vibe-like socket API (but only for what's actually necessary for mysqln, nothing more, so it isn't a general, full-fledged phobos-vibed socket adapter). Then, based on how the templates are instantiated, the socket will be typed as either vibe.core.net.TCPConnection or the internally-defined phobos socket wrapper. I had some reasons for doing it that way, however... However, now that you mention it, I think you're right, I should be able to just add a trivial pass-through wrapper for Vibe.d's sockets, define a minimal common interface for that and the Phobos socket wrapper, and just pass that interface into Connection.this(). There were some reasons I didn't go down that route before, but now that I think about more, I don't think those reasons were valid. And this definitely leads to a much cleaner design and API, plus it's more powerful (can use subclasses of either socket type). And while virtual method invocation isn't free, it should be completely negligible compared to actually performing network I/O, even if it's just localhost. I'm going to try that, see how it plays out. Hmm, there is a problem: For Vibe.d, mysql-native's Connection class is expected to derive from EventedObject. But EventedObject is part of Vibe.d. I'm not sure offhand how to handle that without making mysql-native's Connection templated.
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Thu, 16 May 2013 18:50:57 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: Hmm, there is a problem: For Vibe.d, mysql-native's Connection class is expected to derive from EventedObject. But EventedObject is part of Vibe.d. I'm not sure offhand how to handle that without making mysql-native's Connection templated. Actually, I think I figured out how to handle it: There's no reason Connection's base type can't *always* be EventedObject *WHEN* Vibe.d is actually available, regardless of whether the user chooses to use Vibe's sockets or Phobos sockets. The *only* time Connection actually *needs* to not use EventedObject is when the user specifies -version=MySQLN_NoVibeD to disable mysqln's static dependency on Vibe.d (a feature I was planning to add anyway). So I just statically choose to inherit from EventedObject or plain-old Object depending on whether MySQLN_NoVibeD exists, and that should work fine. Sorry for the noise!
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On 2013-05-14 01:53, Nick Sabalausky wrote: As this is intended as a core-ish low-level lib for any MySQL-client needs, I didn't want to unduly preclude any potential use-cases, even if it's not a typical use-case. However, since it isn't a typical use-case, maybe what I'll do is this: - Use new names for all of the newly templated types: Ex: 'Connection' becomes 'ConnectionX!mySQLSocketVibeD' and 'ConnectionX!mySQLSocketPhobos'. That sounds reasonable. -- /Jacob Carlborg
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Tue, 14 May 2013 04:35:20 +0200 Nathan M. Swan nathanms...@gmail.com wrote: On Sunday, 12 May 2013 at 21:16:33 UTC, Nick Sabalausky wrote: There is need for mysql-native https://github.com/rejectedsoftware/mysql-native to support both Vibe.d's sockets and Phobos sockets. Since Phobos's and Vibe.d's sockets have incompatible APIs, my design converts most of the types in mysql-native into templates. For example, 'Connection' becomes either 'Connection!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the same goes for most other types, such as Row, ResultSet, etc. Does incompatible APIs By incompatible APIs I just meant that I can't use structural/duck typing on them (at least not directly), and they don't have a common base type aside from Object. preclude a design like this? - interface MysqlSocket { // ... } class PhobosSocket : MysqlSocket { this(std.socket.Socket _backend) { // ... } // ... } class VibeSocket : MysqlSocket { this(vibe.core.net.TCPConnection _backend) { // ... } } - That's actually somewhat similar to what I ended up doing internally: I created a minimal wrapper for std.socket.Socket to expose a vibe-like socket API (but only for what's actually necessary for mysqln, nothing more, so it isn't a general, full-fledged phobos-vibed socket adapter). Then, based on how the templates are instantiated, the socket will be typed as either vibe.core.net.TCPConnection or the internally-defined phobos socket wrapper. I had some reasons for doing it that way, however... However, now that you mention it, I think you're right, I should be able to just add a trivial pass-through wrapper for Vibe.d's sockets, define a minimal common interface for that and the Phobos socket wrapper, and just pass that interface into Connection.this(). There were some reasons I didn't go down that route before, but now that I think about more, I don't think those reasons were valid. And this definitely leads to a much cleaner design and API, plus it's more powerful (can use subclasses of either socket type). And while virtual method invocation isn't free, it should be completely negligible compared to actually performing network I/O, even if it's just localhost. I'm going to try that, see how it plays out. Sorry if I've overlooked anything obvious, NMS I think I'm the one who was overlooking things ;) Thanks!
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On 2013-05-12 23:16, Nick Sabalausky wrote: I *could* push that to the main mysql-native master repo right now https://github.com/rejectedsoftware/mysql-native. But I'm hesitant for one reason: Since Phobos's and Vibe.d's sockets have incompatible APIs, my design converts most of the types in mysql-native into templates. For example, 'Connection' becomes either 'Connection!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the same goes for most other types, such as Row, ResultSet, etc. That is the only breaking change, but depending how your code uses mysql-native, it can be a very significant breaking change. Why don't you just relay on a version flag? Is it really necessary to change this at runtime? -- /Jacob Carlborg
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Mon, 13 May 2013 09:03:08 +0200 Jacob Carlborg d...@me.com wrote: On 2013-05-12 23:16, Nick Sabalausky wrote: I *could* push that to the main mysql-native master repo right now https://github.com/rejectedsoftware/mysql-native. But I'm hesitant for one reason: Since Phobos's and Vibe.d's sockets have incompatible APIs, my design converts most of the types in mysql-native into templates. For example, 'Connection' becomes either 'Connection!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the same goes for most other types, such as Row, ResultSet, etc. That is the only breaking change, but depending how your code uses mysql-native, it can be a very significant breaking change. Why don't you just relay on a version flag? Is it really necessary to change this at runtime? As this is intended as a core-ish low-level lib for any MySQL-client needs, I didn't want to unduly preclude any potential use-cases, even if it's not a typical use-case. However, since it isn't a typical use-case, maybe what I'll do is this: - Use new names for all of the newly templated types: Ex: 'Connection' becomes 'ConnectionX!mySQLSocketVibeD' and 'ConnectionX!mySQLSocketPhobos'. - Require a -version to specify which socket type is the default, and then set up backwards-compatible aliases. So for example, if '-version=MySQLN_DefaultSocketVibeD' is used, then 'Connection' (for example) is defined as an alias to 'ConnectionX!mySQLSocketVibeD'. - If a unified socket interface ever emerges, and obviates the needs for the templates, then these -version idents can just simply be ignored and output that -version is no longer needed pragma messages.
Re: RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
On Sunday, 12 May 2013 at 21:16:33 UTC, Nick Sabalausky wrote: There is need for mysql-native https://github.com/rejectedsoftware/mysql-native to support both Vibe.d's sockets and Phobos sockets. Since Phobos's and Vibe.d's sockets have incompatible APIs, my design converts most of the types in mysql-native into templates. For example, 'Connection' becomes either 'Connection!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the same goes for most other types, such as Row, ResultSet, etc. Does incompatible APIs preclude a design like this? - interface MysqlSocket { // ... } class PhobosSocket : MysqlSocket { this(std.socket.Socket _backend) { // ... } // ... } class VibeSocket : MysqlSocket { this(vibe.core.net.TCPConnection _backend) { // ... } } - Sorry if I've overlooked anything obvious, NMS
RFC: mysql-native - Supporting both Vibe.d and Phobos sockets
There is need for mysql-native https://github.com/rejectedsoftware/mysql-native to support both Vibe.d's sockets and Phobos sockets. First, a little background (feel free to skip ahead if you wish): Steve Teale's original version of mysql-native only supported Phobos's sockets (there was no Vibe.d at the time). But all activity and maintenance since then has been on a fork that *replaces* use of Phobos's sockets with Vibe.d's sockets. Vibe.d uses its own socket implementation instead of Phobos's because (AIUI) Phobos sockets don't support the particular Async I/O model that's a cornerstone of Vibe.d. There are people, including myself, who have need to use mysql-native together with Vibe.d. But naturally, there's also very valid reasons why mysql-native should also be usable with nothing more than Phobos, in projects that choose not to rely on Vibe.d. (End of background) So, there's a need for mysql-native to support *both* Vibe.d and Phobos-only, and I've already created a branch which does exactly that: https://github.com/Abscissa/mysql-native/blob/optional-phobos-sockets/source/mysql/connection.d You can use that with either type of socket, even switch between them at runtime (although not within the same connection, obviously). And you can also use an optional -verson= to completely eliminate any compile-time dependency on Vibe.d (which naturally disables the ability to use Vibe.d sockets, of course). I've tested it on both types of sockets with mysql-native's included test app ('source/app.d'), and it all works for me. I *could* push that to the main mysql-native master repo right now https://github.com/rejectedsoftware/mysql-native. But I'm hesitant for one reason: Since Phobos's and Vibe.d's sockets have incompatible APIs, my design converts most of the types in mysql-native into templates. For example, 'Connection' becomes either 'Connection!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the same goes for most other types, such as Row, ResultSet, etc. That is the only breaking change, but depending how your code uses mysql-native, it can be a very significant breaking change. So before merging that, I wanted to get input from anyone else who has used or contributed to either mysql-native or HibernateD http://sourceforge.net/p/hibernated/ which currently uses a Phobos-only fork of mysql-native for its MySQL capability (which is actually my main original motivating factor for making these changes). Couple alternatives I've considered: - I considered templating the types on an actual socket type instead of on enum MySQLSocketType { phobos, vibed }, but I can't imagine a need for any other socket type being likely to arise. And even if it did, mysql-native would likely still need special-handling for it anyway. - The *ideal* solution would be for Phobos and Vibe.d to agree on a common interface, defined in Phobos, and inherit their socket classes from that interface. The reasons I didn't go this route are because 1) I don't know enough about the background/internals/motiovations of either of the socket types to be the right person to spearhead this. and 2) Such a change in Phobos would likely require a lot of time and effort whereas my primary interest is to take mysql-native and just *make it work* on both Phobos and Vibe.d. If the appropriate people on both the Phobos and Vibe.d sides wanted to spearhead an effort to make the respective sockets compatible, and fast-track it all, then I'd be happy to make mysql-native just use that. But I realize priorities might be elsewhere right now, and that's perfectly fine. I figure we can just make it work with both types of sockets right now (as I've already done, albeit in an unofficial branch), and if/when a merged socket design emerges, then mysql-native can just keep one option and deprecate the other. Thoughts? Opinions? Etc?