Re: Matrix-type-friendly syntax and more
On 10.10.2011 03:18, bearophile wrote: > kenji hara: > >> I got an idea for multidimentional indexing and slicing. >> http://d.puremagic.com/issues/show_bug.cgi?id=6798 > > Thank you, you are good. > > So is this: > y[$-6, 0..$:2] > > Translated like this? > y.opIndex(y.opDollar!0 - 6, y.opSlice!1(0, y.opDollar!1, 2)) Nice! This seems like the way to go.
Static arrays, typeof and IFTI?
Hi there,
after a very busy and eventful year in my personal life, I have now
finally found some time to play with D2. I am very impressed by the
progress!
One thing I noticed was that static arrays are somehow strangely limited:
It is possible to overload based on the length:
void f(int[3] x) {
writeln("int[3]: ",x);
}
void f(int[4] x) {
writeln("int[4]: ",x);
}
int main(string argv[]) {
f([1,2,3]);
f([1,2,3,4]);
return 0;
}
However, used as function template argument, a static array is casted to
a dynamic array:
---
void g(T)(T x) {
static assert (__traits(isStaticArray,T));
enum N = T.init.length;
writeln(N,": ",x);
}
int main(string argv[]) {
g([1,2,3]);
return 0;
}
gives the error message:
| Error: static assert (__traits(isStaticArray,int[])) is false
| instantiated from here: g!(int[])
Without the assertion, N is defined to 0.
Further investigation shows:
---
g!(int[3])([1,2,3]); // passes a static array
---
int[3] x3 = [1,2,3];
g(x3);// passes a static array
---
auto z3 = [1,2,3];// defines z3 as dynamic array
g(y3);// passes a dynamic array
---
So it seems, the problem is that array literals on their own turned into
dynamic arrays unless you explicitly state a static array type in some way.
Wouldn't it make more sense the other way around? After all, turning a
static array into a dynamic array is easy, the other way around is
prohibited by the compiler. If array literals simply had a static array
type, they could be implicitly casted to dynamic arrays when necessary
but stay static if possible.
Greetings,
Norbert
Re: [std.database]
On 2011-10-10 06:54, Steve Teale wrote: I'd like to suggest that, as a first step, simple translations of the C header files for the various databases be added to etc.c. That'll at least enable people who need to use a database now to get started. Walter, I'm sure they're already out there waiting. I have MySQL. Any offers for Postgres and SQLite? Does MS-SQL have a C interface these days I wonder? Steve I don't know but FreeTDS has implemented the TDS protocol (which SQL server uses). Either we use FreeTDS (LGPL license) or role our own implementation. -- /Jacob Carlborg
Re: [std.database]
On 2011-10-10 00:31, Walter Bright wrote: On 10/9/2011 5:28 AM, Piotr Szturmaj wrote: 1. I think that we should not design this API using the least common denominator approach. This is to not limit some databases. For example PostgreSQL has many great features not available in MySQL. That's why I started with postgres in my ddb project. I think DB API should be designed to support the most featureful databases and those that have less features may be easily adapted to that API. Haven't common denominator designs been more or less failures in at least one category - gui libraries? Some driver models have succeeded only because a powerful entity forced the issue - like for device drivers for an OS. I suspect that trying to design a common api to popular databases is an expensive and quixotic quest. If it weren't, wouldn't it have happened already? At some level we need a common API for the databases, but still be able to get to database specific API's when there is need for that. -- /Jacob Carlborg
Hooray for DustMite
Just wanted to give kudos to Vladimir Panteleev for his excellent
DustMite utility, which reduced a DMD 2.055 backend ICE in Dinecraft
(a ~10,000 line codebase) to this:
struct ChunkLoc {
}
class Chunk {
private ChunkLoc _loc;
@property
public ChunkLoc loc() {
return _loc;
}
}
private struct FiberWaitingForChunk {
}
class WorldCache {
FiberWaitingForChunk*[ChunkLoc] _fibersWaitingForChunks;
void chunkLoaded(Chunk chunk) {
_fibersWaitingForChunks.remove(chunk.loc);
}
}
Which has now been filed at http://d.puremagic.com/issues/show_bug.cgi?id=6799
Thanks!
Re: Color your terminal's output
Am 09.10.2011, 12:32 Uhr, schrieb Jens Mueller : Johannes Pfau wrote: Jens Mueller wrote: BTW: you could also use isatty (http://linux.die.net/man/3/isatty) to detect if stdout has been redirected and disable color output in that case. For windows: https://forums.embarcadero.com/thread.jspa?threadID=21194 or http://msdn.microsoft.com/de-de/library/f4s0ddew(v=vs.80).aspx Nice idea. Thanks. Jens But allow the programmer to force color output anyway. In some cases you redirect to a program that itself supports or passes through escape sequences. "grep --color=always | less" would be an example of that.
Re: Matrix-type-friendly syntax and more
On 10.10.2011 04:41, kenji hara wrote: 2011/10/10 bearophile: So is this: y[$-6, 0..$:2] Translated like this? y.opIndex(y.opDollar!0 - 6, y.opSlice!1(0, y.opDollar!1, 2)) I have no thought about it. I'm not sure that the additional stepping is really useful, but I think adding it into syntax is not impossible -- it might not conflict with associative array literal. Kenji Hara Personally, I think that since strided operations are so inefficient, they should remain ugly.
Re: [std.database]
> I'd like to suggest that, as a first step, simple translations of the C > header files for the various databases be added to etc.c. That'll at > least enable people who need to use a database now to get started. Walter, I'm sure they're already out there waiting. I have MySQL. Any offers for Postgres and SQLite? Does MS-SQL have a C interface these days I wonder? Steve
Re: Matrix-type-friendly syntax and more
2011/10/10 bearophile : > So is this: > y[$-6, 0..$:2] > > Translated like this? > y.opIndex(y.opDollar!0 - 6, y.opSlice!1(0, y.opDollar!1, 2)) I have no thought about it. I'm not sure that the additional stepping is really useful, but I think adding it into syntax is not impossible -- it might not conflict with associative array literal. Kenji Hara
Re: Matrix-type-friendly syntax and more
kenji hara: > I got an idea for multidimentional indexing and slicing. > http://d.puremagic.com/issues/show_bug.cgi?id=6798 Thank you, you are good. So is this: y[$-6, 0..$:2] Translated like this? y.opIndex(y.opDollar!0 - 6, y.opSlice!1(0, y.opDollar!1, 2)) Bye, bearophile
Re: [std.database]
On 10/9/11 6:31 PM, Sean Kelly wrote: On Oct 9, 2011, at 3:56 PM, Andrei Alexandrescu wrote: On 10/9/11 5:31 PM, Walter Bright wrote: On 10/9/2011 5:28 AM, Piotr Szturmaj wrote: 1. I think that we should not design this API using the least common denominator approach. This is to not limit some databases. For example PostgreSQL has many great features not available in MySQL. That's why I started with postgres in my ddb project. I think DB API should be designed to support the most featureful databases and those that have less features may be easily adapted to that API. Haven't common denominator designs been more or less failures in at least one category - gui libraries? A common database interface is not a common denominator API; more like the opposite. This is not difficult because most differences across database systems lie in their SQL, which is strings from D's perspective. Assuming that by "database" you mean SQL. Pretty fair assumption, though NoSQL databases (which cover a broad range of designs since there's no standard language yet for key-value DBs, etc) are rapidly gaining popularity. I almost wonder if the base type should be named SqlDatabase instead of Database. Good point and good idea. Andrei
Re: Matrix-type-friendly syntax and more
I got an idea for multidimentional indexing and slicing. http://d.puremagic.com/issues/show_bug.cgi?id=6798 I think opSlice!n (n is dimension integer typed size_t) is rarely used, therefore using that name for the enhancement is almost safe for backward compatibility. Kenji Hara 2011/10/10 Denis Shelomovskij : > 09.10.2011 21:29, bearophile пишет: >> >> Kenji Hara has just copied in GIT a large patch by Don: >> https://github.com/9rnsr/dmd/branches/opDollar >> >> It allows user defined collections usable with this syntax: >> >> x[$-2, y[$-6, $-9], $-2] >> >> See: >> http://d.puremagic.com/issues/show_bug.cgi?id=3474 >> >> >> But a syntax like this too is quite important for a user-defined matrix >> type to be used in numerical code: >> >> m[i..j, k..w] >> >> Similar slicing and dicing of arrays is used all the time in NumPy (array >> library for Python) code. >> >> This is a comment by Don about it: >> >>> (4) I have NOT implemented $ inside opSlice(), opSliceAssign(). >>> It could be done, but I believe those language features need work. They >>> don't >>> permit multi-dimensional slicing. I think they should be removed, and the >>> functionality folded into opIndex. >> >> Isn't it better to improve opSlice() instead of deprecating it? >> >> >> I don't remember if people appreciate the idea of a stride (like the third >> step argument of std.range.iota), this too is used in scientific >> array-oriented code: >> >> m[i..j:2, k..w:3] >> >> --- >> >> In Python code there is often the need for a good multi-dimensional array >> type, even in not-scientific code. >> >> In the Python standard library there is a array module, but it's almost a >> joke, it's 1D, and it's not used much: >> http://docs.python.org/library/array.html >> >> Python programmers use the multi-dimensional array of NumPy, it's widely >> used around the world, it's used by SciPy too (a scientific programming >> library). >> >> The experience of Python, with its sorely felt lack of a good >> multi-dimensional array type, the failure of its array module, and the >> success of NumPy, the problems caused by two precedent incompatible array >> libraries Numeric (http://people.csail.mit.edu/jrennie/python/numeric/ ) and >> numarray >> (http://www.stsci.edu/resources/software_hardware/numarray/numarray.html), >> tells me that it will be good to have a bare-bones, but efficient >> multi-dimensional array type. Plus external libraries (not present in >> Phobos) that use those arrays to implement all the things they want. >> >> I think that's a good tradeoff between the opposed needs of: >> - Keeping Phobos of reasonable size (to not increase too much the burden >> of its management, to not slow down too much its development); >> - Avoiding the risk of incompatible multi-dimensional array types. Most >> code out there is able to build on a common foundation. This avoids >> duplication (like the creation of Python numarray and Numeric), allows a >> better focusing of efforts and speeds up the development of a language-wide >> standard for such arrays; >> - Offer a nD array type to the casual D programmer, even one that doesn't >> want or can't install other libraries. Even some 30-lines long D programs >> need multi-dimensional arrays, but they often don't need a complex >> scientific library too (example of a problem: in the preconditions of my >> functions that take an array of arrays I always have to test the input is >> not jagged and it is a rectangular matrix. Such test is not necessary for a >> multi-dimensional array that is never jagged). Putting the bare bones >> multi-dimensional array type in Phobos allows people to use them with zero >> other installs. >> >> This multi-dimensional Phobos array type doesn't even need to contain code >> to invert a matrix or compute determinant, etc. It just needs basic >> operations like allocation, indexing, multi-dimensional slicing, change of >> shape, iteration... Everything else is in modules/packages external to >> Phobos. >> >> I am not suggesting to put a sparse multi-dimensional array type in >> Phobos. This need is much less common in casual short programs. This is >> better left to external modules. >> >> Bye, >> bearophile > > Suddenly, I had a talk with an Fortran guy at previous weekend and have > wrote something like Fortran arrays in D last week. I had an idea of adding > it into Phobos, but just had no time for review of my code. Draft version is > in attachment. > > rarray.d - something like Fortran array > main.d - example of using rarray module > output.txt - main.d output >
Re: RFC - mysqlD
On Sunday, October 09, 2011 16:35:20 Sean Kelly wrote: > On Oct 9, 2011, at 2:42 PM, Jonathan M Davis wrote: > > On Sunday, October 09, 2011 14:35:35 Jonathan M Davis wrote: > >> On Sunday, October 09, 2011 19:37:31 GrahamC wrote: > >> > >> The C types which may differ between x86 and x86_64 are defined in > >> core.stdc.config. > > > > Though it currently only does so for the integral types. > > Which other standard C types vary between x86 and x86_64? On Windows, I don't believe that there's a difference on anything. On Linux, long int, unsigned long int, and long double all differ. I thought that long long int differed too (being 128 bit on 64-bit systems), but trying it now on my 64-bit Linux box, it's 64-bit for both -m32 and -m64, so I guess that it doesn't differ. But regardless, the primary one missing from core.stdc.config is long double. - Jonathan M Davis
Re: [std.database]
On Sunday, October 09, 2011 16:31:35 Sean Kelly wrote: > On Oct 9, 2011, at 3:56 PM, Andrei Alexandrescu wrote: > > On 10/9/11 5:31 PM, Walter Bright wrote: > >> On 10/9/2011 5:28 AM, Piotr Szturmaj wrote: > >>> 1. I think that we should not design this API using the least common > >>> denominator > >>> approach. This is to not limit some databases. For example > >>> PostgreSQL > >>> has many > >>> great features not available in MySQL. That's why I started with > >>> postgres in my > >>> ddb project. I think DB API should be designed to support the most > >>> featureful > >>> databases and those that have less features may be easily adapted to > >>> that API. > >> > >> Haven't common denominator designs been more or less failures in at > >> least one category - gui libraries? > > > > A common database interface is not a common denominator API; more like > > the opposite. This is not difficult because most differences across > > database systems lie in their SQL, which is strings from D's > > perspective. > Assuming that by "database" you mean SQL. Pretty fair assumption, though > NoSQL databases (which cover a broad range of designs since there's no > standard language yet for key-value DBs, etc) are rapidly gaining > popularity. I almost wonder if the base type should be named SqlDatabase > instead of Database. If we were to do that, then maybe it should just be sql, since it's shorter and just as clear: std.sql.*. - Jonathan M Davis
Re: RFC - mysqlD
On Oct 9, 2011, at 2:42 PM, Jonathan M Davis wrote: > On Sunday, October 09, 2011 14:35:35 Jonathan M Davis wrote: >> On Sunday, October 09, 2011 19:37:31 GrahamC wrote: >> >> The C types which may differ between x86 and x86_64 are defined in >> core.stdc.config. > > Though it currently only does so for the integral types. Which other standard C types vary between x86 and x86_64?
Re: RFC - mysqlD
On Oct 9, 2011, at 12:37 PM, GrahamC wrote:
> Don't forget that quite a few MySQL data structure members are C long types
> which are 64 bit on x86_64.
>
> So if you want code to work on x86_64 as well as x86 it needs a type alias.
>
> For example in st_net these members:
>
> struct st_net
> {
>Vio *vio;
>ubyte *buff;
>ubyte *buff_end;
>ubyte *write_pos;
>ubyte *read_pos;
>SOCKET fd;
>uint remain_in_buf; <<<
>uint length; <<<
>uint buf_length; <<<
>uint where_b; <<<
>uint max_packet; <<<
>uint max_packet_size; <<<
>
>
> are 32 bit on x86 and 64 bit on x86_64
So use core.stdc.config.c_long? Or is this type not actually a C long but
rather a typedef that varies in some other way?
Re: [std.database]
On Oct 9, 2011, at 3:56 PM, Andrei Alexandrescu wrote: > On 10/9/11 5:31 PM, Walter Bright wrote: >> On 10/9/2011 5:28 AM, Piotr Szturmaj wrote: >>> 1. I think that we should not design this API using the least common >>> denominator >>> approach. This is to not limit some databases. For example PostgreSQL >>> has many >>> great features not available in MySQL. That's why I started with >>> postgres in my >>> ddb project. I think DB API should be designed to support the most >>> featureful >>> databases and those that have less features may be easily adapted to >>> that API. >> >> >> Haven't common denominator designs been more or less failures in at >> least one category - gui libraries? > > A common database interface is not a common denominator API; more like the > opposite. This is not difficult because most differences across database > systems lie in their SQL, which is strings from D's perspective. Assuming that by "database" you mean SQL. Pretty fair assumption, though NoSQL databases (which cover a broad range of designs since there's no standard language yet for key-value DBs, etc) are rapidly gaining popularity. I almost wonder if the base type should be named SqlDatabase instead of Database.
Re: [std.database]
On 10/9/11 5:31 PM, Walter Bright wrote: On 10/9/2011 5:28 AM, Piotr Szturmaj wrote: 1. I think that we should not design this API using the least common denominator approach. This is to not limit some databases. For example PostgreSQL has many great features not available in MySQL. That's why I started with postgres in my ddb project. I think DB API should be designed to support the most featureful databases and those that have less features may be easily adapted to that API. Haven't common denominator designs been more or less failures in at least one category - gui libraries? A common database interface is not a common denominator API; more like the opposite. This is not difficult because most differences across database systems lie in their SQL, which is strings from D's perspective. Some driver models have succeeded only because a powerful entity forced the issue - like for device drivers for an OS. I suspect that trying to design a common api to popular databases is an expensive and quixotic quest. If it weren't, wouldn't it have happened already? It has. All database APIs for programming languages do exactly that. Andrei
Re: [std.database]
On 10/7/2011 11:43 PM, Steve Teale wrote: I use this title at Andrei's suggestion, and repeat his idea that it be used as a prefix for discussions as we navigate toward a design. Unless there is resistance to the idea, I will on the job of implementing whatever we decide is appropriate. I am retired, and have the time to do it. Thanks, Steve, for being the champion for this project. I'd like to suggest that, as a first step, simple translations of the C header files for the various databases be added to etc.c. That'll at least enable people who need to use a database now to get started.
Re: RFC - mysqlD
On Sunday, October 09, 2011 14:35:35 Jonathan M Davis wrote:
> On Sunday, October 09, 2011 19:37:31 GrahamC wrote:
> > Don't forget that quite a few MySQL data structure members are C long
> > types which are 64 bit on x86_64.
> >
> > So if you want code to work on x86_64 as well as x86 it needs a type
> > alias.
> >
> > For example in st_net these members:
> >
> > struct st_net
> > {
> >
> > Vio *vio;
> > ubyte *buff;
> > ubyte *buff_end;
> > ubyte *write_pos;
> > ubyte *read_pos;
> > SOCKET fd;
> > uint remain_in_buf; <<<
> > uint length; <<<
> > uint buf_length; <<<
> > uint where_b; <<<
> > uint max_packet; <<<
> > uint max_packet_size; <<<
> >
> > are 32 bit on x86 and 64 bit on x86_64
> >
> > Many other MySQL structures and quite a few function arguments have the
> > same issue.
> >
> > Of course, not many people seem to be using the compiler for 64 bit code
> > at the moment.
>
> The C types which may differ between x86 and x86_64 are defined in
> core.stdc.config.
Though it currently only does so for the integral types.
- Jonathan M Davis
Re: [std.database]
On 10/9/2011 5:28 AM, Piotr Szturmaj wrote: 1. I think that we should not design this API using the least common denominator approach. This is to not limit some databases. For example PostgreSQL has many great features not available in MySQL. That's why I started with postgres in my ddb project. I think DB API should be designed to support the most featureful databases and those that have less features may be easily adapted to that API. Haven't common denominator designs been more or less failures in at least one category - gui libraries? Some driver models have succeeded only because a powerful entity forced the issue - like for device drivers for an OS. I suspect that trying to design a common api to popular databases is an expensive and quixotic quest. If it weren't, wouldn't it have happened already?
Re: [std.database]
On 10/09/11 13:22, Andrei Alexandrescu wrote: On 10/9/11 11:40 AM, Steve Teale wrote: Further generic question. (Yes, I am listening to the answers too) If some underlying databases don't support the features that our chosen interface requires, do we attempt to synthesize them - presumably at cost to performance, or do we just throw a compile-time exception that basically tells the user to use a lower interface and code it themself? No. Andrei Sorry, that was awfully unclear. I meant to say the driver shouldn't do little miracles in adapting support from one engine to the next. It's a losing race. It should be fine if certain queries or API calls fail either statically or dynamically. Andrei
Re: RFC - mysqlD
On Sunday, October 09, 2011 19:37:31 GrahamC wrote:
> Don't forget that quite a few MySQL data structure members are C long types
> which are 64 bit on x86_64.
>
> So if you want code to work on x86_64 as well as x86 it needs a type alias.
>
> For example in st_net these members:
>
> struct st_net
> {
> Vio *vio;
> ubyte *buff;
> ubyte *buff_end;
> ubyte *write_pos;
> ubyte *read_pos;
> SOCKET fd;
> uint remain_in_buf; <<<
> uint length; <<<
> uint buf_length; <<<
> uint where_b; <<<
> uint max_packet; <<<
> uint max_packet_size; <<<
>
>
> are 32 bit on x86 and 64 bit on x86_64
>
> Many other MySQL structures and quite a few function arguments have the same
> issue.
>
> Of course, not many people seem to be using the compiler for 64 bit code at
> the moment.
The C types which may differ between x86 and x86_64 are defined in
core.stdc.config.
- Jonathan M Davis
Re: Formal Review of std.regex (FReD)
I've not had a proper look at the code yet, but I recall from when I read the docs during the pre-review period that the introduction was a little on the informal side. It doesn't seem to have changed since then, and IMHO the introduction/description needs a bit of a polish to bring it up to the standard that is required of official documentation. I'll be busy over the next few weeks, but I will try to make time to assemble some more specific comments. I just wanted to let you know that I thought the docs needed some work, just in case. A...
Re: Matrix-type-friendly syntax and more
09.10.2011 21:29, bearophile пишет: Kenji Hara has just copied in GIT a large patch by Don: https://github.com/9rnsr/dmd/branches/opDollar It allows user defined collections usable with this syntax: x[$-2, y[$-6, $-9], $-2] See: http://d.puremagic.com/issues/show_bug.cgi?id=3474 But a syntax like this too is quite important for a user-defined matrix type to be used in numerical code: m[i..j, k..w] Similar slicing and dicing of arrays is used all the time in NumPy (array library for Python) code. This is a comment by Don about it: (4) I have NOT implemented $ inside opSlice(), opSliceAssign(). It could be done, but I believe those language features need work. They don't permit multi-dimensional slicing. I think they should be removed, and the functionality folded into opIndex. Isn't it better to improve opSlice() instead of deprecating it? I don't remember if people appreciate the idea of a stride (like the third step argument of std.range.iota), this too is used in scientific array-oriented code: m[i..j:2, k..w:3] --- In Python code there is often the need for a good multi-dimensional array type, even in not-scientific code. In the Python standard library there is a array module, but it's almost a joke, it's 1D, and it's not used much: http://docs.python.org/library/array.html Python programmers use the multi-dimensional array of NumPy, it's widely used around the world, it's used by SciPy too (a scientific programming library). The experience of Python, with its sorely felt lack of a good multi-dimensional array type, the failure of its array module, and the success of NumPy, the problems caused by two precedent incompatible array libraries Numeric (http://people.csail.mit.edu/jrennie/python/numeric/ ) and numarray (http://www.stsci.edu/resources/software_hardware/numarray/numarray.html), tells me that it will be good to have a bare-bones, but efficient multi-dimensional array type. Plus external libraries (not present in Phobos) that use those arrays to implement all the things they want. I think that's a good tradeoff between the opposed needs of: - Keeping Phobos of reasonable size (to not increase too much the burden of its management, to not slow down too much its development); - Avoiding the risk of incompatible multi-dimensional array types. Most code out there is able to build on a common foundation. This avoids duplication (like the creation of Python numarray and Numeric), allows a better focusing of efforts and speeds up the development of a language-wide standard for such arrays; - Offer a nD array type to the casual D programmer, even one that doesn't want or can't install other libraries. Even some 30-lines long D programs need multi-dimensional arrays, but they often don't need a complex scientific library too (example of a problem: in the preconditions of my functions that take an array of arrays I always have to test the input is not jagged and it is a rectangular matrix. Such test is not necessary for a multi-dimensional array that is never jagged). Putting the bare bones multi-dimensional array type in Phobos allows people to use them with zero other installs. This multi-dimensional Phobos array type doesn't even need to contain code to invert a matrix or compute determinant, etc. It just needs basic operations like allocation, indexing, multi-dimensional slicing, change of shape, iteration... Everything else is in modules/packages external to Phobos. I am not suggesting to put a sparse multi-dimensional array type in Phobos. This need is much less common in casual short programs. This is better left to external modules. Bye, bearophile Suddenly, I had a talk with an Fortran guy at previous weekend and have wrote something like Fortran arrays in D last week. I had an idea of adding it into Phobos, but just had no time for review of my code. Draft version is in attachment. rarray.d - something like Fortran array main.d - example of using rarray module output.txt - main.d output FortranArrays.7z Description: Binary data
Re: RFC - mysqlD
Don't forget that quite a few MySQL data structure members are C long types
which are 64 bit on x86_64.
So if you want code to work on x86_64 as well as x86 it needs a type alias.
For example in st_net these members:
struct st_net
{
Vio *vio;
ubyte *buff;
ubyte *buff_end;
ubyte *write_pos;
ubyte *read_pos;
SOCKET fd;
uint remain_in_buf; <<<
uint length; <<<
uint buf_length; <<<
uint where_b; <<<
uint max_packet; <<<
uint max_packet_size; <<<
are 32 bit on x86 and 64 bit on x86_64
Many other MySQL structures and quite a few function arguments have the same
issue.
Of course, not many people seem to be using the compiler for 64 bit code at the
moment.
Re: [std.database]
On 2011-10-09 18:54, Adam Ruppe wrote:
The way I'd do it is:
interface Database {
// support shared functions here, and other stuff useful enough to
// warrant emulation
}
class Postgres : Database {
// implement the interface, of course, but also all other postgres
// specific stuff
}
When you go to use it, if you're happy with the basics, declare
Databases.
If you need something special, use Postgres objects.
+1
--
/Jacob Carlborg
Re: Formal Review of std.regex (FReD)
On 2011-10-09 17:29, Dmitry Olshansky wrote: On 09.10.2011 19:09, Jacob Carlborg wrote: On 2011-10-09 17:01, Dmitry Olshansky wrote: On 09.10.2011 18:49, Jacob Carlborg wrote: On 2011-10-09 16:09, Dmitry Olshansky wrote: On 09.10.2011 14:33, Jacob Carlborg wrote: On 2011-10-08 21:56, Jesse Phillips wrote: Hello everyone, I have taken the role of review manager of the std.regex replacement by Dmitry Olshansky. The review period begins now 2011-10-8 and will end on 2011-10-23 at midnight UTC. A voting thread to include into Phobos will be held after review assuming such is appropriate. The Voting period is one week. Please note that you can try FRed as part of Phobos (Code) or by itself (Package of FReD) which includes docs. Doc: http://nascent.freeshell.org/fred/doc/ What's the difference between Regex and RegEx? I can see RegEx in the documentation but I cannot find its definition in the docs. RegEx is a template parameter (it's that usual abstract 'T'), that in the end deduced as StaticRegex!Char or Regex!Char where Char is char/wchar/dchar. I don't think the documentation should refer to RegEx if it's not defined in the docs. Yes, I think I see the typo now, thanks. The second parameter type of the match function (and a couple of other functions) is RegEx, is that possible to fix as well? No, that's what I tried to point out but failed obviously. The thing is that it is a templated parameter and due to constraint it could be either StaticRegex!Char or Regex!Char. They represent pattern compiled as machine code or bytecode respectively for character width of Char. All of the 6 versions of compiled patterns in the end do not have a common type nor one is technically possible (w/o some quite bad performance trade offs). Aha, ok, I see. Could RegEx be explained in the docs so it won't cause further confusion? -- /Jacob Carlborg
Matrix-type-friendly syntax and more
Kenji Hara has just copied in GIT a large patch by Don: https://github.com/9rnsr/dmd/branches/opDollar It allows user defined collections usable with this syntax: x[$-2, y[$-6, $-9], $-2] See: http://d.puremagic.com/issues/show_bug.cgi?id=3474 But a syntax like this too is quite important for a user-defined matrix type to be used in numerical code: m[i..j, k..w] Similar slicing and dicing of arrays is used all the time in NumPy (array library for Python) code. This is a comment by Don about it: > (4) I have NOT implemented $ inside opSlice(), opSliceAssign(). > It could be done, but I believe those language features need work. They don't > permit multi-dimensional slicing. I think they should be removed, and the > functionality folded into opIndex. Isn't it better to improve opSlice() instead of deprecating it? I don't remember if people appreciate the idea of a stride (like the third step argument of std.range.iota), this too is used in scientific array-oriented code: m[i..j:2, k..w:3] --- In Python code there is often the need for a good multi-dimensional array type, even in not-scientific code. In the Python standard library there is a array module, but it's almost a joke, it's 1D, and it's not used much: http://docs.python.org/library/array.html Python programmers use the multi-dimensional array of NumPy, it's widely used around the world, it's used by SciPy too (a scientific programming library). The experience of Python, with its sorely felt lack of a good multi-dimensional array type, the failure of its array module, and the success of NumPy, the problems caused by two precedent incompatible array libraries Numeric (http://people.csail.mit.edu/jrennie/python/numeric/ ) and numarray (http://www.stsci.edu/resources/software_hardware/numarray/numarray.html), tells me that it will be good to have a bare-bones, but efficient multi-dimensional array type. Plus external libraries (not present in Phobos) that use those arrays to implement all the things they want. I think that's a good tradeoff between the opposed needs of: - Keeping Phobos of reasonable size (to not increase too much the burden of its management, to not slow down too much its development); - Avoiding the risk of incompatible multi-dimensional array types. Most code out there is able to build on a common foundation. This avoids duplication (like the creation of Python numarray and Numeric), allows a better focusing of efforts and speeds up the development of a language-wide standard for such arrays; - Offer a nD array type to the casual D programmer, even one that doesn't want or can't install other libraries. Even some 30-lines long D programs need multi-dimensional arrays, but they often don't need a complex scientific library too (example of a problem: in the preconditions of my functions that take an array of arrays I always have to test the input is not jagged and it is a rectangular matrix. Such test is not necessary for a multi-dimensional array that is never jagged). Putting the bare bones multi-dimensional array type in Phobos allows people to use them with zero other installs. This multi-dimensional Phobos array type doesn't even need to contain code to invert a matrix or compute determinant, etc. It just needs basic operations like allocation, indexing, multi-dimensional slicing, change of shape, iteration... Everything else is in modules/packages external to Phobos. I am not suggesting to put a sparse multi-dimensional array type in Phobos. This need is much less common in casual short programs. This is better left to external modules. Bye, bearophile
Re: [std.database]
On 10/9/11 11:40 AM, Steve Teale wrote: Further generic question. (Yes, I am listening to the answers too) If some underlying databases don't support the features that our chosen interface requires, do we attempt to synthesize them - presumably at cost to performance, or do we just throw a compile-time exception that basically tells the user to use a lower interface and code it themself? No. Andrei
Re: [std.database]
On 10/9/11 11:15 AM, Steve Teale wrote: If using variants for a set of out parameters you must have something equivalent to: Variant[3] va; va[0] = cast(byte) 0; va[1] = 0.0F; va[2] = cast(char[]) []; So you have to have exactly the same information at compile time for the Variant array as you do for the struct - you still have to specify a set of types. Not at all, fortunately it's much simpler than that. Far as I can tell, a resultset emitted by a DBMS comes in the form of data (either untyped buffers or strings) plus column type information, usually in the form of an enumeration (e.g. 0 for NULL, 1 for int, 2 for double, 3 for string etc). You of course also have the total column count. So all you need to do is use that enum to pour the data into the Variants. See this for example: http://msdn.microsoft.com/en-us/library/ms131297.aspx You can glean all needed information from the resultset after having issued the query. It's probably true to say that the syntax/semantics of the interface will suck slightly more in the Variant case than in the struct case. That's a given. The suckiness won't come, however, in the form of additional trips to the database. Andrei
Re: [std.database]
On 10/9/11 11:54 AM, Adam Ruppe wrote:
The way I'd do it is:
interface Database {
// support shared functions here, and other stuff useful enough to
// warrant emulation
}
class Postgres : Database {
// implement the interface, of course, but also all other postgres
// specific stuff
}
When you go to use it, if you're happy with the basics, declare
Databases.
If you need something special, use Postgres objects.
Makes sense. JDBC does that, too.
Andrei
Re: [std.database]
Steve Teale wrote: We probably should write a page on a wiki describing the API, without actually implementing anything. Then anyone involved may contribute to its design, so it may evolve into somewhat more thought out API. I like that idea! Must find out how to put up a wiki. Or use existing one: http://www.prowiki.org/wiki4d/wiki.cgi :-)
Re: [std.database]
> We probably should write a page on a wiki describing the API, without > actually implementing anything. Then anyone involved may contribute to > its design, so it may evolve into somewhat more thought out API. I like that idea! Must find out how to put up a wiki. Steve
Re: Xinok Sort (was: Sorting algorithm)
On 10/8/2011 11:11 AM, Xinok wrote: On 10/7/2011 12:42 PM, Xinok wrote: Hi, it's been years since I've posted here. I just wanted to share something I worked on, a new sorting algorithm. It's a variant of merge sort, but much more memory efficient with only a small loss in performance. The most similar algorithm I know of is Timsort. I had need for a stable sorting algorithm, but the performance of stable sort in Phobos is terrible. This pretty much left merge sort, which has good performance but requires O(n) space. That persuaded me to design my own sorting algorithm. Here, you can find the code, details of the algorithm, and benchmarks (introSort is the unstable sort in Phobos). http://www.neowin.net/forum/blog/422/entry-3737-sort-algorithm-complete/ To follow up on my original post, I wrote a text file which explains the algorithm in detail. The most important thing to understand is the "range swap", which I tried to explain as simply as possible. http://cl.ly/0H193k2s0G2T1A002v3I/xinokSort.txt And to follow up on this post, I created a permanent page for xinok sort on Sourceforge. There's nothing new to see on this page yet, but I'll work on that in due time. https://sourceforge.net/projects/xinoksort/ My first goal is to write an implementation which I can contribute as the new stable sorting algorithm for Phobos. This includes string mixins, ranges, asserts, unittests, and anything else that is required. If anybody wants to assist me with this, I welcome the help.
Re: [std.database]
Steve Teale wrote: To express a personal opinion, then as a first pass we should do something that is at about the same level as JDBC but without the concessions to DBs like Postgres that have fancy SQL types. I disagree. Doing it this way may introduce difficulties or incompabilities in the future. I think we should design it from the ground up, keeping in mind the all databases. We probably should write a page on a wiki describing the API, without actually implementing anything. Then anyone involved may contribute to its design, so it may evolve into somewhat more thought out API.
Re: [std.database]
Adam Ruppe wrote:
The way I'd do it is:
interface Database {
// support shared functions here, and other stuff useful enough to
// warrant emulation
}
class Postgres : Database {
// implement the interface, of course, but also all other postgres
// specific stuff
}
When you go to use it, if you're happy with the basics, declare
Databases.
If you need something special, use Postgres objects.
+1
Re: [std.database]
The way I'd do it is:
interface Database {
// support shared functions here, and other stuff useful enough to
// warrant emulation
}
class Postgres : Database {
// implement the interface, of course, but also all other postgres
// specific stuff
}
When you go to use it, if you're happy with the basics, declare
Databases.
If you need something special, use Postgres objects.
Re: about const and immutable (again)
Also, this would be a whole lot easier if D got a full built-in tuple
support, because then all functions would always take a single
argument. :-)
On Thu, Oct 6, 2011 at 10:56 PM, Steven Schveighoffer
wrote:
> On Thu, 06 Oct 2011 12:27:16 -0400, Gor Gyolchanyan
> wrote:
>
>> I see. Thanks for the detailed answer.
>
> I should clarify one point, I realized I am somewhat inaccurate on the
> reason the type is set to immutable(dchar). In fact, nobody actually wrote
> the immutable(dchar) function, it's just that the element type is
> immutable(dchar). However, the reasons why someone would want to create a
> function that takes an immutable(dchar) function are as I stated -- so you
> don't accidentally change the value.
>
> Still no excuse that delegates cannot be implicitly cast to compatible
> versions.
>
>> I just looked up Wikipedia for lambdas in different languages and
>> found out, that C# has an awesome syntax for that:
>>
>> x => x * x
>> (x, y) => x == y
>> (int x, string s) => s.Length > x
>> () => SomeMethod()
>> n => { string s = n + " " + "World"; Console.WriteLine(s); }
>
> In the not so distant past, Walter explored different ways to improve the
> lambda syntax on this news group (which included similarities to the above).
> This usually means he is considering adding that change, let's hope so :)
>
> -Steve
>
Re: Note about variant
On Sun, 09 Oct 2011 04:56:45 -0400, Mariusz Gliwiński wrote: Dnia niedziela, 9 października 2011 00:13:29 Jonathan M Davis pisze: Part of your problem may be that D doesn't use copy constructors. Rather, it has postblit constructors: Yes, i know about naming difference and behavioral difference. It's not a problem for me anymore, this post is for people who will encounter this problem in the future (so they can google it, and quickly find the reason why are they getting this error). Lastly, if someone would like to fix this, now he knows the reason. The code works in the improved std.variant I've been working on.
Re: [std.database]
Further generic question. (Yes, I am listening to the answers too) If some underlying databases don't support the features that our chosen interface requires, do we attempt to synthesize them - presumably at cost to performance, or do we just throw a compile-time exception that basically tells the user to use a lower interface and code it themself? It's important that we establish such boundaries. Otherwise we'll never ever have an alpha version of 0.1. To express a personal opinion, then as a first pass we should do something that is at about the same level as JDBC but without the concessions to DBs like Postgres that have fancy SQL types. When we have decided on an interface, we can always go forward, but going back is embarrassing. Steve
Re: [std.database]
Further question. Should we assume in the first instance that we should only attempt to accommodate those DBs that are free or have some free version that may be limited in some way - e.g. the developer version of MS SQL Server. Presumably when D reaches the point of being all-conquering, then Oracle, IBM and so on will chip in. Steve
Re: [std.database]
There was some discussion prior to this thread about the relative virtues
of binding to structs or binding to arrays of Variants. I was thinking
about this, and have experimented with Variants in my trial MySQL
implementation. My conclusions below - do they make sense?
Using Variant to capture the output of prepared queries or provide input,
as opposed to a struct, gets you out of the realm of what must be
determined at compile time, but only at the expense of extra DB server
round trip(s).
If you want to use them in a deterministic way with a database table of
known and stable structure you have to bind them. To bind them you must
be able to determine their type. But with Variants you can't do that
until they are initialized. So, with a struct you must have something
like:
struct MyTableQuery42
{
byte col1;
float col2;
char[] col3;
}
This can be bound for output without doing any explicit initialization of
an instance, since you can write a template function to bind it that
discovers everything you need to know about the struct at compile time.
If using variants for a set of out parameters you must have something
equivalent to:
Variant[3] va;
va[0] = cast(byte) 0;
va[1] = 0.0F;
va[2] = cast(char[]) [];
So you have to have exactly the same information at compile time for the
Variant array as you do for the struct - you still have to specify a set
of types.
The difference is that if you have prepared a statement, you can go to
the server and ask for the relevant metadata. With a struct you can use
this to check if the struct is a match for the query. With an array of
Variants you can make if conform to the query.
However, in a large number of cases, using the struct you won't need to
bother with the metadata, because you 'know' the types of the query
result. You don't have to bother with them for the Variant array either,
but in that case you have to provide a function like
Variant[] MyTableQuery42Init() { ... }
which 'knows' the same stuff.
It's probably true to say that the syntax/semantics of the interface will
suck slightly more in the Variant case than in the struct case.
Steve
Re: Is it a Variant?
On Sun, 09 Oct 2011 01:39:10 -0400, Robert Jacques wrote: > Yes, there is. Variant is now a struct so: > > is(typeof(target)==Variant) > > works just like you'd expect. So it does. Thanks.
Re: Formal Review of std.regex (FReD)
On 09.10.2011 19:09, Jacob Carlborg wrote: On 2011-10-09 17:01, Dmitry Olshansky wrote: On 09.10.2011 18:49, Jacob Carlborg wrote: On 2011-10-09 16:09, Dmitry Olshansky wrote: On 09.10.2011 14:33, Jacob Carlborg wrote: On 2011-10-08 21:56, Jesse Phillips wrote: Hello everyone, I have taken the role of review manager of the std.regex replacement by Dmitry Olshansky. The review period begins now 2011-10-8 and will end on 2011-10-23 at midnight UTC. A voting thread to include into Phobos will be held after review assuming such is appropriate. The Voting period is one week. Please note that you can try FRed as part of Phobos (Code) or by itself (Package of FReD) which includes docs. Doc: http://nascent.freeshell.org/fred/doc/ What's the difference between Regex and RegEx? I can see RegEx in the documentation but I cannot find its definition in the docs. RegEx is a template parameter (it's that usual abstract 'T'), that in the end deduced as StaticRegex!Char or Regex!Char where Char is char/wchar/dchar. I don't think the documentation should refer to RegEx if it's not defined in the docs. Yes, I think I see the typo now, thanks. The second parameter type of the match function (and a couple of other functions) is RegEx, is that possible to fix as well? No, that's what I tried to point out but failed obviously. The thing is that it is a templated parameter and due to constraint it could be either StaticRegex!Char or Regex!Char. They represent pattern compiled as machine code or bytecode respectively for character width of Char. All of the 6 versions of compiled patterns in the end do not have a common type nor one is technically possible (w/o some quite bad performance trade offs). -- Dmitry Olshansky
Re: Lazy Literal Range: Stooid late-nite idea?
On 2011-10-09 13:43, Lutger Blijdestijn wrote:
Jacob Carlborg wrote:
On 2011-10-09 08:33, Vladimir Panteleev wrote:
On Sun, 09 Oct 2011 06:06:21 +0300, Nick Sabalausky wrote:
Great, right? But what about this?:
auto x = [runtimeExpressionA, runtimeExprB, runtimeExprC,
etc].find(blah);
With the anonymous delegate literal syntax suggested by Andrei a while
ago, you should be able to write this as:
auto x = [() => runtimeExpressionA, () => runtimeExprB, () =>
runtimeExprC, () => etc].find(blah);
I guess it looks quirky with an empty parameter list, but it's shorter
than writing {return runtimeExpressionA;}, {return runtimeExpressionB;}
etc.
Or wrap it in a function taking a list of lazy parameters:
http://www.d-programming-language.org/function.html#parameters
auto x = lazyRange(runtimeExpressionA, runtimeExprB, runtimeExprC);
typesafe variadics with delegates also allow for this syntax, though not
with type inference. I hacked this together:
struct LazyRange(T)
{
import std.array;
this(T delegate()[] input...)
{
_input = input;
}
@property
bool empty() const
{
return std.array.empty(_input);
}
void popFront()
{
std.array.popFront(_input);
_isCached = false;
}
@property
T front()
{
if (_isCached)
return _front;
_front = std.array.front(_input)();
_isCached = true;
return _front;
}
private:
T _front;
bool _isCached = false;
T delegate()[] _input;
}
unittest
{
import std.algorithm;
auto stuff = LazyRange!string("foo",
"foo" ~ "bar",
{ assert(false); return "bat"; }(),
"meow");
assert(find(stuff,"foobar").front == "foobar");
}
Interesting, I didn't know you could do that with a delegate without
declaring a lazy parameter. This works with D1:
void lazyRange (T) (lazy T[] e ...)
{
foreach (a ; e)
println(a);
}
void main ()
{
lazyRange(3, 4);
}
But compiling the same code with D2 results in this error:
Error: escaping reference to local __arrayArg1344
--
/Jacob Carlborg
Re: Formal Review of std.regex (FReD)
On 2011-10-09 17:01, Dmitry Olshansky wrote: On 09.10.2011 18:49, Jacob Carlborg wrote: On 2011-10-09 16:09, Dmitry Olshansky wrote: On 09.10.2011 14:33, Jacob Carlborg wrote: On 2011-10-08 21:56, Jesse Phillips wrote: Hello everyone, I have taken the role of review manager of the std.regex replacement by Dmitry Olshansky. The review period begins now 2011-10-8 and will end on 2011-10-23 at midnight UTC. A voting thread to include into Phobos will be held after review assuming such is appropriate. The Voting period is one week. Please note that you can try FRed as part of Phobos (Code) or by itself (Package of FReD) which includes docs. Doc: http://nascent.freeshell.org/fred/doc/ What's the difference between Regex and RegEx? I can see RegEx in the documentation but I cannot find its definition in the docs. RegEx is a template parameter (it's that usual abstract 'T'), that in the end deduced as StaticRegex!Char or Regex!Char where Char is char/wchar/dchar. I don't think the documentation should refer to RegEx if it's not defined in the docs. Yes, I think I see the typo now, thanks. The second parameter type of the match function (and a couple of other functions) is RegEx, is that possible to fix as well? -- /Jacob Carlborg
Re: Formal Review of std.regex (FReD)
On 09.10.2011 18:49, Jacob Carlborg wrote: On 2011-10-09 16:09, Dmitry Olshansky wrote: On 09.10.2011 14:33, Jacob Carlborg wrote: On 2011-10-08 21:56, Jesse Phillips wrote: Hello everyone, I have taken the role of review manager of the std.regex replacement by Dmitry Olshansky. The review period begins now 2011-10-8 and will end on 2011-10-23 at midnight UTC. A voting thread to include into Phobos will be held after review assuming such is appropriate. The Voting period is one week. Please note that you can try FRed as part of Phobos (Code) or by itself (Package of FReD) which includes docs. Doc: http://nascent.freeshell.org/fred/doc/ What's the difference between Regex and RegEx? I can see RegEx in the documentation but I cannot find its definition in the docs. RegEx is a template parameter (it's that usual abstract 'T'), that in the end deduced as StaticRegex!Char or Regex!Char where Char is char/wchar/dchar. I don't think the documentation should refer to RegEx if it's not defined in the docs. Yes, I think I see the typo now, thanks. -- Dmitry Olshansky
Re: Formal Review of std.regex (FReD)
On 2011-10-09 16:09, Dmitry Olshansky wrote: On 09.10.2011 14:33, Jacob Carlborg wrote: On 2011-10-08 21:56, Jesse Phillips wrote: Hello everyone, I have taken the role of review manager of the std.regex replacement by Dmitry Olshansky. The review period begins now 2011-10-8 and will end on 2011-10-23 at midnight UTC. A voting thread to include into Phobos will be held after review assuming such is appropriate. The Voting period is one week. Please note that you can try FRed as part of Phobos (Code) or by itself (Package of FReD) which includes docs. Doc: http://nascent.freeshell.org/fred/doc/ What's the difference between Regex and RegEx? I can see RegEx in the documentation but I cannot find its definition in the docs. RegEx is a template parameter (it's that usual abstract 'T'), that in the end deduced as StaticRegex!Char or Regex!Char where Char is char/wchar/dchar. I don't think the documentation should refer to RegEx if it's not defined in the docs. -- /Jacob Carlborg
Re: Formal Review of std.regex (FReD)
On 09.10.2011 14:33, Jacob Carlborg wrote: On 2011-10-08 21:56, Jesse Phillips wrote: Hello everyone, I have taken the role of review manager of the std.regex replacement by Dmitry Olshansky. The review period begins now 2011-10-8 and will end on 2011-10-23 at midnight UTC. A voting thread to include into Phobos will be held after review assuming such is appropriate. The Voting period is one week. Please note that you can try FRed as part of Phobos (Code) or by itself (Package of FReD) which includes docs. Doc: http://nascent.freeshell.org/fred/doc/ What's the difference between Regex and RegEx? I can see RegEx in the documentation but I cannot find its definition in the docs. RegEx is a template parameter (it's that usual abstract 'T'), that in the end deduced as StaticRegex!Char or Regex!Char where Char is char/wchar/dchar. -- Dmitry Olshansky
Re: [std.database]
On 10/9/11 7:28 AM, Piotr Szturmaj wrote:
1. I think that we should not design this API using the least common
denominator approach. This is to not limit some databases. For example
PostgreSQL has many great features not available in MySQL. That's why I
started with postgres in my ddb project. I think DB API should be
designed to support the most featureful databases and those that have
less features may be easily adapted to that API.
Some PostgreSQL vs MySQL differences to note:
- the former does support arrays and composite types that may be stored
in single column, the latter doesn't
Yah, Hive also has some really interesting data types, such as
parameterized arrays and maps and JSON data. Our API should be generic
enough to accommodate such types seamlessly.
- the former support asynchronous queries, but I guess its not that
important
- the former support async notifications (for example, fired by a
trigger, or when server is shutting down)
Not sure how we could support such. Ideas would be welcome.
2. Compile type mapping of fields should support postgres's composites
and arrays. For example, this is obvious:
execQuery!(int, string)("select 5, 'abc'");
but this is not:
execQuery!(int, string)("select ROW(5, 'abc')"); // valid postgres query
I think that should be
execQuery!(Tuple!(int, string))("select ROW(5, 'abc')");
Does it return two columns or one composite column? I already addressed
this ambiguities in ddb, please see documentation on github
(http://pszturmaj.github.com/ddb/db.html). See also 'advanced example'
on that page.
3. I think that compile type mapping of fields should be generalized to
not only support DB API. It could also be used with CSV files or other
tabular data. It may also be extended to support tree structures with
XML (mapping xml to structs/tuples/arrays).
Interesting.
Andrei
Re: Color your terminal's output
Jens Mueller wrote: >Johannes Pfau wrote: >> Jens Mueller wrote: >> >Johannes Pfau wrote: >> >> You could use ANSI codes on posix to avoid a dependency on curses: >> >> http://en.wikipedia.org/wiki/ANSI_escape_code#Colors >> >> But I think using curses is ok. ncurses is MIT licensed and can be >> >> used as a dynamic library, so I don't think there are license >> >> problems. >> >> >> >> However, I'd recommend to load ncurses dynamically with >> >> dlopen/dlsym and fallback to simple text output if the ncurses >> >> library cannot be loaded. >> > >> >Using the ANSI codes is fine with me. I assumed they aren't that >> >portable but it seems fine. >> >> I'd still use curses though, as it abstracts some things away. >> Finding terminal capabilities and similar stuff. >> >> >> >Any help is very appreciated. >> >> > >> >> >Though this module is functionality-wise inferior to something >> >> >like ncurses it conveniently allows coloring output for most use >> >> >cases. >> >> >> >> as you already use these functions: >> >> http://linux.die.net/man/3/setupterm >> >> it'd be nice to have wget-like progressbars and 'updateable' text >> >> labels. Shouldn't be as fancy as full ncurses, for most use cases >> >> it's good enough to modify the current line. +Points if it >> >> properly handles terminal width and resizing. >> > >> >I believe progress bars are easy to add. Boost's progress_bar >> >should be fairly easy to port. It'll be nice if you could provide a >> >pull request for this. Is this feasible for you? >> >> Sure, I'll have a look at it soon. You're talking about this >> progress_display class, right? >> http://www.boost.org/doc/libs/1_47_0/boost/progress.hpp > >Yes. Maybe it's too simple compared to wget's. > >> >Regarding update able text labels I'm not sure how they are >> >typically used. So I would also prefer some pull request from >> >somebody with a common use case. >> >> Well, think of wget's output: >> 0% [ ] 1.154.567123K/s >> ETA 57m 6s || >> | | | updateable label >> progressbar u-label u-label u-label > >That looks very useful. Is this difficult to add? >Can't one just delete the entire line and replace it with an >appropriately updated one? Is this too naive? Yes it's implemented exactly that way. But I think having it in a library is useful nevertheless. Here's what I have so far: https://gist.github.com/1273678 Some improvements to be done: -Autodetect terminal width -test on windows -use isatty and produce useful output for non-ttys -refactoring, docs >This would be easy to implement on Windows as well, I suppose. It should be quite portable. I think "\r" should work on windows terminals. If not, the only needed function is to reset the cursor to the beginning of the current line. > >Jens -- Johannes Pfau
Re: [std.database]
Steve Teale wrote:
I use this title at Andrei's suggestion, and repeat his idea that it be used
as a prefix for discussions as we navigate toward a design. Unless there is
resistance to the idea, I will on the job of implementing whatever we decide
is appropriate. I am retired, and have the time to do it.
It seems that every man, and possibly his dog, has a private implementation
for at least a favorite DB, so we should have plenty of material to build on.
At this point I would like to get responses from those who feel they are
likely to contribute to the design through to completion.
I'd also like to get a feel for the magnitude of the task, so I'd like to ask
what database systems you think should be supported.
I have started a github account, and will put my mysqld stuff there shortly,
then you can kick me out if you don't like what you see.
1. I think that we should not design this API using the least common
denominator approach. This is to not limit some databases. For example
PostgreSQL has many great features not available in MySQL. That's why I
started with postgres in my ddb project. I think DB API should be
designed to support the most featureful databases and those that have
less features may be easily adapted to that API.
Some PostgreSQL vs MySQL differences to note:
- the former does support arrays and composite types that may be stored
in single column, the latter doesn't
- the former support asynchronous queries, but I guess its not that
important
- the former support async notifications (for example, fired by a
trigger, or when server is shutting down)
2. Compile type mapping of fields should support postgres's composites
and arrays. For example, this is obvious:
execQuery!(int, string)("select 5, 'abc'");
but this is not:
execQuery!(int, string)("select ROW(5, 'abc')"); // valid postgres query
Does it return two columns or one composite column? I already addressed
this ambiguities in ddb, please see documentation on github
(http://pszturmaj.github.com/ddb/db.html). See also 'advanced example'
on that page.
3. I think that compile type mapping of fields should be generalized to
not only support DB API. It could also be used with CSV files or other
tabular data. It may also be extended to support tree structures with
XML (mapping xml to structs/tuples/arrays).
Re: Formal Review of std.regex (FReD)
Christian Kamm wrote: > Andrei Alexandrescu wrote: > >> On 10/8/11 3:34 PM, Dmitry Olshansky wrote: >>> I've found out what caused my builds to break. The thing is that both >>> std.file & std.stdio use fully qualified std.c.stdio.func calls but >>> never actually import std.c.stdio in any way. I wasn't even aware that's >>> possible. >> >> That may be a bug in the compiler. A symbol shouldn't be visible unless >> e.g. publicly imported from an imported module (could that be the case)? > > It's definitely a bug. Once an import is processed, the package is visible > globally as long as the parent package is accessible. Heh, I actually reported it a while ago and then forgot about it. :) http://d.puremagic.com/issues/show_bug.cgi?id=6307
Re: Lazy Literal Range: Stooid late-nite idea?
Jacob Carlborg wrote:
> On 2011-10-09 08:33, Vladimir Panteleev wrote:
>> On Sun, 09 Oct 2011 06:06:21 +0300, Nick Sabalausky wrote:
>>
>>> Great, right? But what about this?:
>>>
>>> auto x = [runtimeExpressionA, runtimeExprB, runtimeExprC,
>>> etc].find(blah);
>>
>> With the anonymous delegate literal syntax suggested by Andrei a while
>> ago, you should be able to write this as:
>>
>> auto x = [() => runtimeExpressionA, () => runtimeExprB, () =>
>> runtimeExprC, () => etc].find(blah);
>>
>> I guess it looks quirky with an empty parameter list, but it's shorter
>> than writing {return runtimeExpressionA;}, {return runtimeExpressionB;}
>> etc.
>>
>
> Or wrap it in a function taking a list of lazy parameters:
> http://www.d-programming-language.org/function.html#parameters
>
> auto x = lazyRange(runtimeExpressionA, runtimeExprB, runtimeExprC);
>
typesafe variadics with delegates also allow for this syntax, though not
with type inference. I hacked this together:
struct LazyRange(T)
{
import std.array;
this(T delegate()[] input...)
{
_input = input;
}
@property
bool empty() const
{
return std.array.empty(_input);
}
void popFront()
{
std.array.popFront(_input);
_isCached = false;
}
@property
T front()
{
if (_isCached)
return _front;
_front = std.array.front(_input)();
_isCached = true;
return _front;
}
private:
T _front;
bool _isCached = false;
T delegate()[] _input;
}
unittest
{
import std.algorithm;
auto stuff = LazyRange!string("foo",
"foo" ~ "bar",
{ assert(false); return "bat"; }(),
"meow");
assert(find(stuff,"foobar").front == "foobar");
}
Re: [std.database]
Am 08.10.2011 23:11, schrieb Adam D. Ruppe: Microsoft SQL Server is important to cover too. Yeah, it's probably pretty widely used in the Windows world so it should be supported. And Oracle should probably be supported as well. But if we have a generic DB API support for these can be added later (based on ODBC, some product specific API or whatever). I think support for SQLite, MySQL and PostgreSQL would be a good start. Cheers, - Daniel
Re: Formal Review of std.regex (FReD)
On 2011-10-08 21:56, Jesse Phillips wrote: Hello everyone, I have taken the role of review manager of the std.regex replacement by Dmitry Olshansky. The review period begins now 2011-10-8 and will end on 2011-10-23 at midnight UTC. A voting thread to include into Phobos will be held after review assuming such is appropriate. The Voting period is one week. Please note that you can try FRed as part of Phobos (Code) or by itself (Package of FReD) which includes docs. Doc: http://nascent.freeshell.org/fred/doc/ What's the difference between Regex and RegEx? I can see RegEx in the documentation but I cannot find its definition in the docs. -- /Jacob Carlborg
Re: Color your terminal's output
Johannes Pfau wrote: > Jens Mueller wrote: > >Trass3r wrote: > >> >You could use ANSI codes on posix to avoid a dependency on curses: > >> >http://en.wikipedia.org/wiki/ANSI_escape_code#Colors > >> >But I think using curses is ok. ncurses is MIT licensed and can be > >> >used as a dynamic library, so I don't think there are license > >> >problems. > >> > > >> >However, I'd recommend to load ncurses dynamically with dlopen/dlsym > >> >and fallback to simple text output if the ncurses library cannot be > >> >loaded. > >> > >> +1 > >> There shouldn't be a hard dependency on curses. > > > >I had the impression that even though there is this standard how do I > >know that I have a standard-compliant terminal. Can I just assume this? > >I started using curses because I had the impression there may be > >non-standard terminals. But this seems to be minor issue. I will change > >this if people are happy with Windows and ISO/IEC 6429 compliant > >terminals only. > >Thanks. > > > >Jens > > AFAIK the capabilities of a terminal are found using the terminfo > database: > http://en.wikipedia.org/wiki/Terminal_capabilities > But it seems there's no standard stand-alone library to access this > database. Curses seems the only simple way to access this database, so That's how it seems to be. > I'd say: > Try to load curses at runtime, if that fails, fall back to simple > non-colored text output. I'd better throw an exception. > BTW: you could also use isatty (http://linux.die.net/man/3/isatty) to > detect if stdout has been redirected and disable color output in that > case. > For windows: > https://forums.embarcadero.com/thread.jspa?threadID=21194 > or > http://msdn.microsoft.com/de-de/library/f4s0ddew(v=vs.80).aspx Nice idea. Thanks. Jens
Re: Color your terminal's output
Johannes Pfau wrote: > Jens Mueller wrote: > >Johannes Pfau wrote: > >> You could use ANSI codes on posix to avoid a dependency on curses: > >> http://en.wikipedia.org/wiki/ANSI_escape_code#Colors > >> But I think using curses is ok. ncurses is MIT licensed and can be > >> used as a dynamic library, so I don't think there are license > >> problems. > >> > >> However, I'd recommend to load ncurses dynamically with dlopen/dlsym > >> and fallback to simple text output if the ncurses library cannot be > >> loaded. > > > >Using the ANSI codes is fine with me. I assumed they aren't that > >portable but it seems fine. > > I'd still use curses though, as it abstracts some things away. Finding > terminal capabilities and similar stuff. > > >> >Any help is very appreciated. > >> > > >> >Though this module is functionality-wise inferior to something like > >> >ncurses it conveniently allows coloring output for most use cases. > >> > >> as you already use these functions: > >> http://linux.die.net/man/3/setupterm > >> it'd be nice to have wget-like progressbars and 'updateable' text > >> labels. Shouldn't be as fancy as full ncurses, for most use cases > >> it's good enough to modify the current line. +Points if it properly > >> handles terminal width and resizing. > > > >I believe progress bars are easy to add. Boost's progress_bar should be > >fairly easy to port. It'll be nice if you could provide a pull request > >for this. Is this feasible for you? > > Sure, I'll have a look at it soon. You're talking about this > progress_display class, right? > http://www.boost.org/doc/libs/1_47_0/boost/progress.hpp Yes. Maybe it's too simple compared to wget's. > >Regarding update able text labels I'm not sure how they are typically > >used. So I would also prefer some pull request from somebody with a > >common use case. > > Well, think of wget's output: > 0% [ ] 1.154.567123K/s ETA 57m 6s > || | | | > updateable label progressbar u-label u-label u-label That looks very useful. Is this difficult to add? Can't one just delete the entire line and replace it with an appropriately updated one? Is this too naive? This would be easy to implement on Windows as well, I suppose. Jens
Re: Lazy Literal Range: Stooid late-nite idea?
On 2011-10-09 08:33, Vladimir Panteleev wrote:
On Sun, 09 Oct 2011 06:06:21 +0300, Nick Sabalausky wrote:
Great, right? But what about this?:
auto x = [runtimeExpressionA, runtimeExprB, runtimeExprC,
etc].find(blah);
With the anonymous delegate literal syntax suggested by Andrei a while
ago, you should be able to write this as:
auto x = [() => runtimeExpressionA, () => runtimeExprB, () =>
runtimeExprC, () => etc].find(blah);
I guess it looks quirky with an empty parameter list, but it's shorter
than writing {return runtimeExpressionA;}, {return runtimeExpressionB;}
etc.
Or wrap it in a function taking a list of lazy parameters:
http://www.d-programming-language.org/function.html#parameters
auto x = lazyRange(runtimeExpressionA, runtimeExprB, runtimeExprC);
--
/Jacob Carlborg
Re: Formal Review of std.regex (FReD)
On 2011-10-08 23:37, Andrei Alexandrescu wrote: On 10/8/11 3:34 PM, Dmitry Olshansky wrote: I've found out what caused my builds to break. The thing is that both std.file & std.stdio use fully qualified std.c.stdio.func calls but never actually import std.c.stdio in any way. I wasn't even aware that's possible. That may be a bug in the compiler. A symbol shouldn't be visible unless e.g. publicly imported from an imported module (could that be the case)? Andrei I think it's a bug, but sometimes it can be useful. -- /Jacob Carlborg
Re: [std.database]
On 2011-10-08 23:12, Jonathan M Davis wrote: On Saturday, October 08, 2011 12:00:37 Andrei Alexandrescu wrote: 1. If we build a D wrapper for ODBC, then we allow people to write code for any database that has an ODBC driver. This, assuming we commit to ODBC as D's standard database interface, would complete the project. 2. If we want to go the route of "one std.database API with drivers for each DBMS" and consider ODBC one of several DBMSs, then we need to define our own driver architecture, write a few drivers ourselves (including probably ODBC), and hope that people will add more drivers. That's a larger project but it unties us from ODBC. 3. If we want to go the route of "similar but not identical specialized APIs per database system" and consider ODBC again only one of the database systems, then we need to define one specialized API per DBMS and force users to essentially choose upfront what DBMS they'll use, and code for it. It's an even larger project and I don't see obvious advantages for it than less of a need for upfront design. I definitely vote for #2 or #3. One of our projects at work uses it (though not one that I personally work on), and I've never heard good things about it. Supporting it makes good sense, but I wouldn't want us to design Phobos' database solution around it. We have some trouble with ODBC at work too. We should probably explore #2 first, and then if it doesn't work well enough, then at least we have a solid base for the API's being similar with #3. However, given what I've heard about ODBC and its attempts to unify databases, I'm skeptical of how well we'll be able to have a unified DBMS API without harming performance. And from what I understand, it's pretty rare to change DBMSes for a project. You just pick one and use it. And then in the rare case where you have to change, you do the work to do it (and as long as the DB is appropriately modularized with regards to the rest of the program, it doesn't have a hugely negative affect on the rest of the program). So, I question that #2 gains us a whole lot over #3 ultimately (_especially_ if it ends up costing much of anything in terms of performance or usability), but I'm not a DB expert, and I do think that it's at least worth exploring #2 first - if nothing else because it could lead to the APIs for #3 being better unified without harming their performance or usability. - Jonathan M Davis I think it can be quite useful to change DBMSes for a project. For example, start with SQLite because it easy to set up and then move to MySQL or similar. It's especially easy to do with Ruby on Rails on Mac OS X. Rails uses SQLite as the default database and Mac OS X comes bundle with SQLite. There is no extra steps to set up this environment. -- /Jacob Carlborg
Re: [std.database]
On 2011-10-08 23:11, Adam D. Ruppe wrote: Microsoft SQL Server is important to cover too. I'm pretty sure ODBC works fine for that (there's ODBC bindings for D already, it's part of the Windows headers) and I wrote a little something for my database.d, but I haven't actually tested it yet! I have some bad experience with ODBC, connecting to SQL Server, from work. It can't handle multiple result sets and it's not very good at handle invalid Unicode characters. This might not be true for ODBC in general but it's a problem we have with the implementation we currently use. If we don't want to completely role our own implementation I suggest we use freetds directly instead (ODBC uses freetds). I think it would be good to have an ODBC implementation but it should not be the only way to connect to SQL Server. -- /Jacob Carlborg
Re: Formal Review of std.regex (FReD)
Brad Roberts wrote: > Isn't this bug #314? Very well known, super old, highly voted for, etc, > etc. No, bug 314 is about privately imported symbols being accessible even though they shouldn't be. This problem is about modules that aren't imported at all in a file or any of its imports still being accessible. Btw: I've updated my pull request to fix #314 to apply cleanly against the current dmd/master: https://github.com/D-Programming-Language/dmd/pull/190
Re: [std.database]
On 2011-10-08 19:00, Andrei Alexandrescu wrote: 1. If we build a D wrapper for ODBC, then we allow people to write code for any database that has an ODBC driver. This, assuming we commit to ODBC as D's standard database interface, would complete the project. 2. If we want to go the route of "one std.database API with drivers for each DBMS" and consider ODBC one of several DBMSs, then we need to define our own driver architecture, write a few drivers ourselves (including probably ODBC), and hope that people will add more drivers. That's a larger project but it unties us from ODBC. 3. If we want to go the route of "similar but not identical specialized APIs per database system" and consider ODBC again only one of the database systems, then we need to define one specialized API per DBMS and force users to essentially choose upfront what DBMS they'll use, and code for it. It's an even larger project and I don't see obvious advantages for it than less of a need for upfront design. Andrei I would say that we declare a high level interface for database drivers. std.database can use this interface to connect to all databases there are drivers for. We then provide driver implementations for this interface for the databases we choose to support. It should also be possible for a user to create his/her own driver implementation for a database we haven't yet implemented or choose not to implement. Driver implementations could be: * MySQL/MariaDB * PostgreSQL * SQLite * ODBC * Oracle * SQL Server The smart thing would probably be to implement ODBC as the first database driver since it would allow most of the common databases to be used. But I don't want ODBC to be the only driver. I have some bad experience with ODBC from work. It can't handle multiple result sets and it's not very good at handle invalid Unicode characters. This might not be true for ODBC in general but it's a problem we have with the implementation we currently use. Also, ODBC adds an extra layer between the application and the database. -- /Jacob Carlborg
Re: Formal Review of std.regex (FReD)
On 10/9/2011 12:30 AM, Andrei Alexandrescu wrote:
> On 10/9/11 2:26 AM, Christian Kamm wrote:
>> Andrei Alexandrescu wrote:
>>
>>> On 10/8/11 3:34 PM, Dmitry Olshansky wrote:
I've found out what caused my builds to break. The thing is that both
std.file& std.stdio use fully qualified std.c.stdio.func calls but
never actually import std.c.stdio in any way. I wasn't even aware that's
possible.
>>>
>>> That may be a bug in the compiler. A symbol shouldn't be visible unless
>>> e.g. publicly imported from an imported module (could that be the case)?
>>
>> It's definitely a bug. Once an import is processed, the package is visible
>> globally as long as the parent package is accessible. This compiles:
>>
>> touch dmd2/src/phobos/std/empty.d
>>
>> a.d:
>> import std.stdio;
>>
>> b.d:
>> import std.empty;
>> void main() { std.stdio.writeln("hi!"); }
>
> Hm, this is important. But what is the contribution of a.d to the example? Do
> you compile it together with b.d?
>
> Andrei
Isn't this bug #314? Very well known, super old, highly voted for, etc, etc.
Re: [std.database]
Steve Teale Wrote: > I use this title at Andrei's suggestion, and repeat his idea that it be used > as a prefix for discussions as we navigate toward a design. Unless there is > resistance to the idea, I will on the job of implementing whatever we decide > is appropriate. I am retired, and have the time to do it. > > It seems that every man, and possibly his dog, has a private implementation > for at least a favorite DB, so we should have plenty of material to build on. > > At this point I would like to get responses from those who feel they are > likely to contribute to the design through to completion. > > I'd also like to get a feel for the magnitude of the task, so I'd like to ask > what database systems you think should be supported. > > I have started a github account, and will put my mysqld stuff there shortly, > then you can kick me out if you don't like what you see. > > Steve Please refer to the successful experience of the industry, such as ejb, jdbc, jdo, jpa, ado.net, delphi db api, ruby on rails active recorder, orm's hibernate, absorb they the advantages of, to create a powerful and simple for both for enterprise applications database api framework, to achieve this goal will Requires more than cooperation, or you personal proficient in all of the above api, Hoping to start directly from the orm(such as hibernate does not depend on jdbc,or Low dependence ), Improve orm performance. Hope more database experts involved in the D database api design ! thanks all ! dolive
Re: Formal Review of std.regex (FReD)
Andrei Alexandrescu wrote:
> On 10/9/11 2:26 AM, Christian Kamm wrote:
>> Andrei Alexandrescu wrote:
>>
>>> On 10/8/11 3:34 PM, Dmitry Olshansky wrote:
I've found out what caused my builds to break. The thing is that both
std.file& std.stdio use fully qualified std.c.stdio.func calls but
never actually import std.c.stdio in any way. I wasn't even aware
that's possible.
>>>
>>> That may be a bug in the compiler. A symbol shouldn't be visible unless
>>> e.g. publicly imported from an imported module (could that be the case)?
>>
>> It's definitely a bug. Once an import is processed, the package is
>> visible globally as long as the parent package is accessible. This
>> compiles:
>>
>> touch dmd2/src/phobos/std/empty.d
>>
>> a.d:
>> import std.stdio;
>>
>> b.d:
>> import std.empty;
>> void main() { std.stdio.writeln("hi!"); }
>
> Hm, this is important. But what is the contribution of a.d to the
> example? Do you compile it together with b.d?
Yes, 'dmd b.d' fails, 'dmd a.d b.d' succeeds.
Re: Formal Review of std.regex (FReD)
On 10/9/11 2:26 AM, Christian Kamm wrote:
Andrei Alexandrescu wrote:
On 10/8/11 3:34 PM, Dmitry Olshansky wrote:
I've found out what caused my builds to break. The thing is that both
std.file& std.stdio use fully qualified std.c.stdio.func calls but
never actually import std.c.stdio in any way. I wasn't even aware that's
possible.
That may be a bug in the compiler. A symbol shouldn't be visible unless
e.g. publicly imported from an imported module (could that be the case)?
It's definitely a bug. Once an import is processed, the package is visible
globally as long as the parent package is accessible. This compiles:
touch dmd2/src/phobos/std/empty.d
a.d:
import std.stdio;
b.d:
import std.empty;
void main() { std.stdio.writeln("hi!"); }
Hm, this is important. But what is the contribution of a.d to the
example? Do you compile it together with b.d?
Andrei
Re: Formal Review of std.regex (FReD)
Andrei Alexandrescu wrote:
> On 10/8/11 3:34 PM, Dmitry Olshansky wrote:
>> I've found out what caused my builds to break. The thing is that both
>> std.file & std.stdio use fully qualified std.c.stdio.func calls but
>> never actually import std.c.stdio in any way. I wasn't even aware that's
>> possible.
>
> That may be a bug in the compiler. A symbol shouldn't be visible unless
> e.g. publicly imported from an imported module (could that be the case)?
It's definitely a bug. Once an import is processed, the package is visible
globally as long as the parent package is accessible. This compiles:
touch dmd2/src/phobos/std/empty.d
a.d:
import std.stdio;
b.d:
import std.empty;
void main() { std.stdio.writeln("hi!"); }
Re: Note about variant
Dnia niedziela, 9 października 2011 00:13:29 Jonathan M Davis pisze: > Part of your problem may be that D doesn't use copy constructors. Rather, it > has postblit constructors: Yes, i know about naming difference and behavioral difference. It's not a problem for me anymore, this post is for people who will encounter this problem in the future (so they can google it, and quickly find the reason why are they getting this error). Lastly, if someone would like to fix this, now he knows the reason.
Re: GitHub or dsource?
On Thu, 07 Jul 2011 19:33:01 +0200, Mirko Pilger wrote: > google code is a good choice if you prefer mercurial. > > http://code.google.com/projecthosting/ Not really, it's pretty blah. Of the hg providers, BitBucket is the best. Not a patch on GitHub though.
Re: Note about variant
Part of your problem may be that D doesn't use copy constructors. Rather, it has postblit constructors: http://d-programming-language.org/struct.html#StructPostblit - Jonathan M Davis
