Re: New programming paradigm
On Sunday, 3 June 2018 at 16:36:52 UTC, Simen Kjærås wrote: On Sunday, 3 June 2018 at 14:57:37 UTC, DigitalDesigns wrote: On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote: You might want to have a look at https://wiki.dlang.org/Dynamic_typing This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea. No, this is not what I'm talking about, although maybe it could be related in some way. Actually, it sort of is. Your mapEnum is essentially the same as std.variant.visit (https://dlang.org/phobos/std_variant#.visit), and std.variant.Algebraic is the type that encapsulates both the runtime tag and the void[] containing the data of unknown type. Now, there may be many important differences - Algebraic encapsulates the data and tag, which may or may not be what you want, visit only takes one algebraic argument, mapEnum may be faster, more or less generic, etc. The idea of converting a run-time value to a compile-time value is the same, though. -- Simen I didn't know that variants had those functions! pretty nice. Yes, it is similar to what I'm doing. Same principles but just a little different perspective. I use enums, UDA's, and templates rather than a Algebraic and delegates. The difference is that the enum stores only the type information rather than the variable and the type info that Algebraic stores. If I were to have know about this before I might have used it instead and everything would have probably been fine. The only thing is that the enum version lets me store the type info separately than with the data. When several variables depend on the type id I think it will make it a little easier than having to manage several Algebraic type info's across several variables to sync them. For example dataType type; void[] in, out; rather than Algebraic!(type1,..., typen) in, out; and then having to make sure the types are synced between in and out. At least in my case it might be a little easier. Also my way uses a templated function directly rather than an array of lambads, although they are equivalent: Algebraic!(string, int) variant; variant.visit!((string s) => cast(int) s.length, (int i)=> i)(); which could be written as variant.visit!((string s) => foo(s), (int i)=> foo(i))(); auto foo(T)(T t) { } would become enum variant { @("int") _int, @("string") _string, } mixin(variant.MapEnum!("foo")()); auto foo(T)(T t) { } So, they definitely are very similar and actually might be identical. I haven't used Algebraic and visit any to know. What I do know is that for several Algebraics you would have to do something like variant.visit!((string s) => variant2.visit!((double d) => { foo(s,d); })), (int i)=> foo(i))(); etc. Which is creating the nested switch structure and can become complicated while my method still remains one line but foo just takes more than one template parameter. My feeling is mine is a little less robust since it's more for specific types of code while visit is a little more general. Mainly because of the hard coding of the mixin structure.
Re: Confusion/trying to understand CTFE keywords
On Sunday, June 03, 2018 21:32:06 gdelazzari via Digitalmars-d-learn wrote: > Hello everyone, I'm new here on the forum but I've been exploring > D for quite a while. I'm not an expert programmer by any means, > so this one may be a really silly question and, in that case, > please forgive me. > > With the premise that I've still not looked a lot into "complex" > compile time "stuff" (whether it's templates, CTFE, compile-time > constants, etc...), so that may be the reason I may be missing > the point... I'm trying to understand why keywords such as > "static" or "enum" are used to denote compile time "things". What > I mean is that those keywords are also used for other purposes, > so I find it a bit confusing. Couldn't a keyword like "ctfe" > (just making it up right now) exist? So that, when seeing > something like > > ctfe myNumber = 5; > > ctfe if (myNumber + 2 == 7) > { >// ... > } > > one could immediately understand that the code is > executed/evaluated at compile time. True, after someone knows > that "static" and "enum" mean (in the above example) that some > compile-time things are happening, it's fine. I just find it a > bit confusing not having a dedicated keyword but re-using > existing ones that also serve other purposes... > > Note that this is not an attack to the language or anything (I > actually really love it), I'm just trying to understand the > reasoning behind this choice. > > Thank you very much in advance. I think that part of your problem here comes from the fact that you think of enum or static are "CTFE keywords." That's not what they are at all. Yes, they can trigger CTFE, but they're not the only way. Given how CTFE works in D, it really wouldn't make sense to have a keyword for it. CTFE is simply what kicks in when you have an expression that _must_ be evaluated at compile time. e.g. enum a = 42; enum Foo { a = 42, b = 92 c = 12 } struct S { int i = 9; } all are cases where CTFE is used, because all of them require that the value be known at compile time. In the example, they're just integers, so no functions are called, but any one of them could be initialized with an expression that involved calling a function. e.g. struct S { int i = foo() + 7; } There is no special keyword here, and there is no need for one. By the language's design, if i is directly initialized, its value must be known at compile time, and so any expression that's used to directly initialize it must be evaluated at compile time. The same goes for enums or static variables. How would you expect something like this to even work with a special keyword for CTFE? Now, the fact that enum was used for manifest constants such as enum foo = "hello"; in addition to actual enums such as enum Color { red, green, blue, orange, yellow } is arguably unnecessary and confusing (and as some of the other posts in this thread mention, this was done to avoid adding a new keyword). So, maybe, we should have had something like manifest foo = "hello"; and made enum foo = "hello"; illegal, but even if we had done something like that, it would not have had any effect on how CTFE works. It would have just clarified the difference between manifest constants and proper enums. As for why enum was reused for manifest constants, it was not only to save a keyword but because they bascially act like anonymous enums (and in fact, that's what the spec calls them) in that how they act is exactly like enums except for the fact that they don't declare a new type. So, whether it would have been better to use a new keyword is a matter of debate. As for static, most of what it does is inherited from C/C++ and Java, and while does get used in several contexts, it's actually used quite consistantly, much as it might not seem that way at first. When talking about a static variable, the key difference between a static variable and an enum (be it a manifest constant or an actual enum) is that a static variable is an actual variable with a location in memory, whereas an enum is just a value that you can refer to by name. The value of the enum is basically copy-pasted wherever it it is used. It's not legal to take the address of an enum, and in fact, that means that if you do something like enum arr = [1, 2, 3]; then every time you use arr, you're potentially allocating a new dynamic array, because foo(arr) is the same as foo([1, 2, 3]) except for two things: 1. If you change the value of arr, it changes its value everywhere, wheres if you use [1, 2, 3] directly, you'd have to change every place that you used it if you wanted to change it. 2. The value of arr is copied, not the expression used to initialize it. So, if you had enum arr = [1, 2, bar()]; and bar() resulted in 42, then foo(arr) would be the same as foo([1, 2, 42]) and not foo([1, 2, bar()]). However, static variables are actually variables. The key difference between them and other variables at the same scope is that
Re: Confusion/trying to understand CTFE keywords
On Sunday, 3 June 2018 at 21:32:06 UTC, gdelazzari wrote: Note that this is not an attack to the language or anything (I actually really love it), I'm just trying to understand the reasoning behind this choice. Because they have a thing about not adding new keywords, apparently it's more important that people dont lose a word they can use for a variable name than it is for expressive non context dependent keywords.
Re: Confusion/trying to understand CTFE keywords
On Sunday, 3 June 2018 at 21:32:06 UTC, gdelazzari wrote: I'm trying to understand why keywords such as "static" or "enum" are used to denote compile time "things". What I mean is that those keywords are also used for other purposes, so I find it a bit confusing. Couldn't a keyword like "ctfe" (just making it up right now) exist? I believe the enum was chosen over a dedicated keyword for compile-time constants to prevent adding another keyword, as D already has quite a few.
Confusion/trying to understand CTFE keywords
Hello everyone, I'm new here on the forum but I've been exploring D for quite a while. I'm not an expert programmer by any means, so this one may be a really silly question and, in that case, please forgive me. With the premise that I've still not looked a lot into "complex" compile time "stuff" (whether it's templates, CTFE, compile-time constants, etc...), so that may be the reason I may be missing the point... I'm trying to understand why keywords such as "static" or "enum" are used to denote compile time "things". What I mean is that those keywords are also used for other purposes, so I find it a bit confusing. Couldn't a keyword like "ctfe" (just making it up right now) exist? So that, when seeing something like ctfe myNumber = 5; ctfe if (myNumber + 2 == 7) { // ... } one could immediately understand that the code is executed/evaluated at compile time. True, after someone knows that "static" and "enum" mean (in the above example) that some compile-time things are happening, it's fine. I just find it a bit confusing not having a dedicated keyword but re-using existing ones that also serve other purposes... Note that this is not an attack to the language or anything (I actually really love it), I'm just trying to understand the reasoning behind this choice. Thank you very much in advance.
Re: how to define infix function
On Saturday, 2 June 2018 at 23:17:48 UTC, Simen Kjærås wrote: On Saturday, 2 June 2018 at 22:09:49 UTC, Neia Neutuladh wrote: On Saturday, 2 June 2018 at 21:44:39 UTC, greatsam4sure wrote: Sorry for the typo is it possible to define infix function in D 3.min(5)// 3: where min is a function, works in D 3 min 5 // does not work. thanks in advance This is a horrible abuse of D's operator overloading discovered by FeepingCreature in the distant past. You have to delimit your custom infix operator with slashes; you can't make `3 min 5` work, but you can make `3 /min/ 5` work. Observe: struct Min { MinIntermediate!T opBinaryRight(string op, T)(T value) if (op == "/") { return MinIntermediate!T(value); } } struct MinIntermediate(T) { T value; T opBinary(string op, T)(T value2) if (op == "/") { if (value < value2) return value; return value2; } } Min min; void main() { writeln(1 /min/ 2); } And of course, this can be generalized: struct Operator(alias fn, string operator = "/") { static auto opBinaryRight(string op : operator, T...)(T value1) { struct Result { auto opBinary(string op : operator, U...)(U value2) if (__traits(compiles, fn(value1, value2))) { return fn(value1, value2); } } Result result; return result; } } unittest { import std.algorithm.comparison; alias min = Operator!(std.algorithm.comparison.min); assert(1 /min/ 3 == 1); } Note also the use of static opBinaryRight, allowing one to eschew the 'min' variable. All of this said, I would suggest not using this in prod - it's a neat trick that shows off some of D's power, but I don't see a case where this would be easier to understand than a straightforward function call. -- Simen That is interesting. I don't know yet whether it's good or bad, but certainly, it's interesting.
Re: New programming paradigm
On Sunday, 3 June 2018 at 14:57:37 UTC, DigitalDesigns wrote: On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote: You might want to have a look at https://wiki.dlang.org/Dynamic_typing This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea. No, this is not what I'm talking about, although maybe it could be related in some way. Actually, it sort of is. Your mapEnum is essentially the same as std.variant.visit (https://dlang.org/phobos/std_variant#.visit), and std.variant.Algebraic is the type that encapsulates both the runtime tag and the void[] containing the data of unknown type. Now, there may be many important differences - Algebraic encapsulates the data and tag, which may or may not be what you want, visit only takes one algebraic argument, mapEnum may be faster, more or less generic, etc. The idea of converting a run-time value to a compile-time value is the same, though. -- Simen
Re: Line endings when redirecting output to file on windows.
On Sunday, 3 June 2018 at 15:42:48 UTC, rikki cattermole wrote: On 04/06/2018 3:24 AM, Bastiaan Veelo wrote: I need some help understanding where extra '\r' come from when output is redirected to file on Windows. First, this works correctly: rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");" As expected, out.txt contains "hello\r\n". I would expect the following to do the same, but it doesn't: rdmd --eval="write(\"hello\" ~ newline);" > out.txt Now out.txt contains "hello\r\r\n". Who is doing the extra conversion here, and how do I stop it? Thanks! Bastiaan. That would be cmd. Not sure you can stop it without piping it after rdmd to remove the \r. Thanks. It is starting to dawn on me that I shouldn't use `newline` and `toFile` to write text files, but rather always use "\n" as line ending and use `write` for both writing to stdout and file. rdmd --eval="File(\"out.txt\", \"w\").write(\"hello\n\");" and rdmd --eval="write(\"hello\n\");" > out.txt both produce "hello\r\n" on Windows. Am I correct, or is there a more idiomatic way of writing strings to text files?
Re: pipeProcess failing
On Sunday, 3 June 2018 at 15:07:07 UTC, DigitalDesigns wrote: I'm calling pipe process using pipeProcess([AliasSeq!args], Redirect.stdout | Redirect.stdin); where args is a tuple. Everything works when I pass each argument individually. If I combine any args using a space it fails or if I pass an argument with "". So I guess something like this pipeProcess(["dmd", "", "-m32 -JC:\"], Redirect.stdout | Redirect.stdin); will fail while pipeProcess(["dmd", "-m32", "-JC:\"], Redirect.stdout | Redirect.stdin); works. Is this a bug or something else going on I'm not aware of? I'm just wrapping pipe process in a function foo(Args...)(Args args) and calling it like foo("dmd", "", "-m32 -JC:\"). The reason why it is a problem is that it will simplify some code to be able to combine some arguments. The argument list is just passed along to the child process as its argv. So if the child process is able to handle empty strings or space-combined arguments in argv, then it will work, and if the child process can't do that, it will fail. Normally, the command-line shell takes care of splitting arguments into separate strings, and never passes empty arguments, so most programs are not prepared to handle those cases. If you want the shell to do this work for you, you can try using pipeShell instead of pipeProcess. Otherwise, you will have to do it yourself.
Re: New programming paradigm
On Monday, 4 September 2017 at 03:26:23 UTC, EntangledQuanta wrote: Take a variant type. It contains the "type" and the data. To simplify, we will treat look at it like (pseudo-code, use your brain) enum Type { int, float } foo(void* Data, Type type); The normal way to deal with this is a switch: switch(type) { case int: auto val = *(cast(int*)Data); case float: auto val = *(cast(float*)Data); } But what if the switch could be generated for us? [...] But, in fact, since we can specialize on the type we don't have to use switch and in some cases do not even need to specialize: for example: foo(T)(T* Data) { writeln(*Data); } is a compile time template that is called with the correct type value at run-time due to the "magic" which I have yet to introduce. Note that if we just use a standard runtime variant, writeln would see a variant, not the correct type that Data really is. This is the key difference and what makes this "technique" valuable. We can treat our dynamic variables as compile time types(use the compile time system) without much hassle. They fit naturally in it and we do not clutter our code switches. We can have a true auto/var like C# without the overhead of the IR. The cost, of course, is that switches are still used, they are generated behind the scenes though and the runtime cost is a few instructions that all switches have and that we cannot avoid. To get a feel for what this new way of dealing with dynamic types might look like: void foo(var y) { writeln(y); } var x = "3"; // or possibly var!(string, int) for the explicit types used foo(x); x = 3; foo(x); It sounds like what you are describing is a sum type. There is an implementation of one in the standard library, std.variant.Algebraic, as well as several alternative implementations on code.dlang.org, including my own, "sumtype" [1]. Using sumtype, your example would look like this: alias Var = SumType!(string, int); void foo(Var y) { var.match!( (value) { writeln(value); } // template lambda ); } Var x = "3"; foo(x); x = 3; foo(x); The match method takes a list of functions as template arguments, and generates a switch statement that maps each possible type of Var to one of those functions. All type checking is done at compile time. [1] https://code.dlang.org/packages/sumtype
Re: Line endings when redirecting output to file on windows.
On 04/06/2018 3:24 AM, Bastiaan Veelo wrote: I need some help understanding where extra '\r' come from when output is redirected to file on Windows. First, this works correctly: rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");" As expected, out.txt contains "hello\r\n". I would expect the following to do the same, but it doesn't: rdmd --eval="write(\"hello\" ~ newline);" > out.txt Now out.txt contains "hello\r\r\n". Who is doing the extra conversion here, and how do I stop it? Thanks! Bastiaan. That would be cmd. Not sure you can stop it without piping it after rdmd to remove the \r.
Line endings when redirecting output to file on windows.
I need some help understanding where extra '\r' come from when output is redirected to file on Windows. First, this works correctly: rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");" As expected, out.txt contains "hello\r\n". I would expect the following to do the same, but it doesn't: rdmd --eval="write(\"hello\" ~ newline);" > out.txt Now out.txt contains "hello\r\r\n". Who is doing the extra conversion here, and how do I stop it? Thanks! Bastiaan.
pipeProcess failing
I'm calling pipe process using pipeProcess([AliasSeq!args], Redirect.stdout | Redirect.stdin); where args is a tuple. Everything works when I pass each argument individually. If I combine any args using a space it fails or if I pass an argument with "". So I guess something like this pipeProcess(["dmd", "", "-m32 -JC:\"], Redirect.stdout | Redirect.stdin); will fail while pipeProcess(["dmd", "-m32", "-JC:\"], Redirect.stdout | Redirect.stdin); works. Is this a bug or something else going on I'm not aware of? I'm just wrapping pipe process in a function foo(Args...)(Args args) and calling it like foo("dmd", "", "-m32 -JC:\"). The reason why it is a problem is that it will simplify some code to be able to combine some arguments.
Re: New programming paradigm
On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote: On Saturday, 2 June 2018 at 23:12:46 UTC, DigitalDesigns wrote: On Thursday, 7 September 2017 at 22:53:31 UTC, Biotronic wrote: [...] I use something similar where I use structs behaving like enums. Each field in the struct is an "enum value" which an attribute, this is because I have not had luck with using attributes on enum values directly and that structs allow enums with a bit more power. [...] You might want to have a look at https://wiki.dlang.org/Dynamic_typing This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea. No, this is not what I'm talking about, although maybe it could be related in some way. What I am talking about is hooking up a runtime variable that can take a few values, such as from an enum and have those be mapped to a compile time template value. This way you get full static time checking of runtime code. Seems impossible? It's not! What it does is leverage D's meta programming engine to deal with all the routine possiblities. A large switch statement is what makes the runtime to compile time magic happen int x; switch(x) { default: foo!void(); case 0: foo!int(); case 1: foo!double(); etc... } See how the switch maps a runtime value x to a templated function foo? we then can handle the x values with foo void foo(T)() { // if x = 0 then T = int // if x = 1 then T = double } But inside foo we have T, the template variable that is the compile time representation of the dynamic variable x. Remember, x's value is unknown at compile time... the switch is what maps the runtime to the compile time. But in foo, because we have T, the type system all works fine. What makes this very useful that we can call templated functions using T and the meta engine will pick the right template function specialization. e.g., void bar(S)() { } can be used inside foo by calling bar!T(). It doesn't seem like much here if you had to use x it would be a pain. Either you would have to manually create switches or create a rats nest of if statements. But we never have to worry about that stuff when using the above method because it is exactly like programming at compile time as if x a compile time value(like say, just int). It works great when you have several template variables and just want everything to work together without having to go in to much trouble: foo(A,B)() { B b = 4; bar!(A)(b) } suppose A can come from int, double, and B from float and long That is 4 different combinations one would normally have to represent. Not a big deal until you have to handle every combination. Suppose you are working with void arrays. They contain types but you don't know the type except after compile time. Without using this technique you have to use casts and tricks and you'll find out if you screwed up some typing stuff at runtime. Using this technique you will not have a void array but a T[] with T being any of the possible types that you specify using UDA's. if you could if (x == 0) { foo(cast(int[])a); } else if (x == 1) { foo(cast(double[])a); } else but I can do that with one line which simply generates the switch for me. Really all I'm doing is hiding the switch so it all looks like some magic is happening in one line. But the fact that it becomes really simple to do seems to open up the use of it and conceptually on can then think of "x" as a compile time variable that can take on several possibilities.
Re: how to define infix function
On Saturday, 2 June 2018 at 22:09:49 UTC, Neia Neutuladh wrote: On Saturday, 2 June 2018 at 21:44:39 UTC, greatsam4sure wrote: [...] This is a horrible abuse of D's operator overloading discovered by FeepingCreature in the distant past. You have to delimit your custom infix operator with slashes; you can't make `3 min 5` work, but you can make `3 /min/ 5` work. Observe: struct Min { MinIntermediate!T opBinaryRight(string op, T)(T value) if (op == "/") { return MinIntermediate!T(value); } } struct MinIntermediate(T) { T value; T opBinary(string op, T)(T value2) if (op == "/") { if (value < value2) return value; return value2; } } Min min; void main() { writeln(1 /min/ 2); } thanks
Re: New programming paradigm
On Saturday, 2 June 2018 at 23:12:46 UTC, DigitalDesigns wrote: On Thursday, 7 September 2017 at 22:53:31 UTC, Biotronic wrote: [...] I use something similar where I use structs behaving like enums. Each field in the struct is an "enum value" which an attribute, this is because I have not had luck with using attributes on enum values directly and that structs allow enums with a bit more power. [...] You might want to have a look at https://wiki.dlang.org/Dynamic_typing This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea.
Re: Convert a huge SQL file to CSV
On Friday, 1 June 2018 at 10:15:11 UTC, Martin Tschierschke wrote: On Friday, 1 June 2018 at 09:49:23 UTC, biocyberman wrote: I need to convert a compressed 17GB SQL dump to CSV. A workable solution is to create a temporary mysql database, import the dump, query by python, and export. But i wonder if there is something someway in D to parse the SQL file directly and query and export the data. I imagine this will envolve both parsing and querying because the data is stored in several tables. I am in the process of downloading the dump now so I can’t give excerpt of the data. You don't need python: https://michaelrigart.be/export-directly-mysql-csv/ SELECT field1, field2 FROM table1 INTO OUTFILE '/path/to/file.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' FIELDS ESCAPED BY '\' LINES TERMINATED BY '\n'; Most important: INTO OUTFILE : here you state the path where you want MySQL to store the CSV file. Keep in mind that the path needs to be writeable for the MySQL user You can write a parser for SQL in D, but even if the import into mysql would take some time, it's only compute time and not yours. Regards mt. Ah yes, thank you Martin. I forgot that we can do a "batch" SQL query where mysql server can parse and run query commands. So no need for Python. But I am still currently waiting for the import to finish the importing of mysql dump. It took 18 hours and is still counting! The whole mysql database is 68GB at the moment. Can we avoid the import and query the database dump directly?