Re: Phobos Action Items
https://lexi-lambda.github.io/blog/2016/06/12/four-months-with-haskell/#documentation-is-nearly-worthless - found this on reddit, criticizes haskell docs and praises racket docs: they are so effective because of how the prose explains what each function does, when to use it, why you’d use it, and why you wouldn’t use it.
Re: Phobos Action Items
On Monday, 20 June 2016 at 07:48:49 UTC, Andrea Fontana wrote: On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: 6. replace std.xml with something we can be proud of that is second to none in performance (Robert burner Schadek is mentoring on this https://github.com/lodo1995/experimental.xml) I hope it will support html and xpath too. Andrea The low-level parser is explicitly written to support any format which resembles XML, and will have no problem handling HTML. Regarding higher level APIs: with validations turned on, HTML would surely be rejected (it breaks many XML well-formedness constraints); with validations turned off, it may or may not work, because higher level APIs implicitly assume that their input is well-formed. As for XPath, I probably won't have time to work on it during GSoC, but if everything goes well I may work on it after the end of the project. Lodovico Giaretta
Re: Phobos Action Items
On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: Starting a new thread with this, as requested. There are things here for all skill levels and time commitments. 3. use -cov to improve code coverage of Phobos unittests 4. make sure every function in Phobos has an example 5. make sure every function in Phobos has Params: and Returns: sections http://www.digitalmars.com/d/archives/digitalmars/D/Phobos_Documentation_-_call_to_action_258777.html Maybe someone would like to go a step meta and look into automatically tracking and checking things first? Here is what autotester (or some other tool) could do: * Check for a lower bound of coverage, so we never regress. * Check documentation for Example, Params:, Returns: * Check for broken links By tracking I think: Count the stuff above and plot over time. Somebody did this for the number of pull requests, for example. This in itself does not fix anything, but it motivates. "Measure it and it will improve" –Jack Welch
Re: Phobos Action Items
On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: 6. replace std.xml with something we can be proud of that is second to none in performance (Robert burner Schadek is mentoring on this https://github.com/lodo1995/experimental.xml) I hope it will support html and xpath too. Andrea
Re: Phobos Action Items
On Sunday, 19 June 2016 at 16:21:36 UTC, Lodovico Giaretta wrote: Fibers are more of a concurrency than a parallelism tool. Do you have a link to relevant Chapel description? I am not familiar with it. Just found this, although there's probably other material out there: http://chapel.cray.com/tutorials/SC10/M10-4-TaskPar.pdf Lodovico Giaretta Thanks, that would be what I would have directed him to. They kind of use task parallelism in a way similar to concurrency. There is more here http://chapel.cray.com/docs/latest/users-guide/taskpar/taskParallelismOverview.html as well as in the language spec http://chapel.cray.com/docs/latest/language/spec.html
Re: Phobos Action Items
On Sunday, 19 June 2016 at 15:40:09 UTC, Dicebot wrote: On Sunday, 19 June 2016 at 13:32:02 UTC, jmh530 wrote: On Sunday, 19 June 2016 at 06:19:34 UTC, Dicebot wrote: Which is inherently suboptimal and is a part of Go marketing bullshit not worth spending time on. It also requires heavy runtime modifications (because TLS) unless one wants to totally screw plain fibers. Proper action item instead would be providing standard event loop and task system in Phobos. I'm really not much of a fibers user, but I have read a bit on how Chapel handles parallelism and found that very interesting. Fibers are more of a concurrency than a parallelism tool. Do you have a link to relevant Chapel description? I am not familiar with it. Just found this, although there's probably other material out there: http://chapel.cray.com/tutorials/SC10/M10-4-TaskPar.pdf Lodovico Giaretta
Re: Phobos Action Items
On Sunday, 19 June 2016 at 13:32:02 UTC, jmh530 wrote: On Sunday, 19 June 2016 at 06:19:34 UTC, Dicebot wrote: Which is inherently suboptimal and is a part of Go marketing bullshit not worth spending time on. It also requires heavy runtime modifications (because TLS) unless one wants to totally screw plain fibers. Proper action item instead would be providing standard event loop and task system in Phobos. I'm really not much of a fibers user, but I have read a bit on how Chapel handles parallelism and found that very interesting. Fibers are more of a concurrency than a parallelism tool. Do you have a link to relevant Chapel description? I am not familiar with it.
Re: Phobos Action Items
On Sunday, 19 June 2016 at 06:19:34 UTC, Dicebot wrote: Which is inherently suboptimal and is a part of Go marketing bullshit not worth spending time on. It also requires heavy runtime modifications (because TLS) unless one wants to totally screw plain fibers. Proper action item instead would be providing standard event loop and task system in Phobos. I'm really not much of a fibers user, but I have read a bit on how Chapel handles parallelism and found that very interesting.
Re: Phobos Action Items
On Sunday, 19 June 2016 at 09:50:42 UTC, John Colvin wrote: On Sunday, 19 June 2016 at 01:01:30 UTC, Nicholas Wilson wrote: On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: [...] Now that I'm on my (southern hemisphere) winter break I will be working on getting LDC to emit PTX and SPRIV and if I get time whatever metal uses to be able to use D as the source language for CUDA, OpenCL and Metal respectively. This should hopefully enable seamless integration with the use of introspection and whatnot. This would give us a huge advantage, being able to target the three major compute APIs. This would be awesome. I would love to get my hands on prototypes of this, no matter how buggy and unfinished they are at any given point. Glad you're enthusiastic! I've made a new thread in general about this. All testing appreciated.
Re: Phobos Action Items
On Sunday, 19 June 2016 at 06:28:45 UTC, deadalnix wrote: On Sunday, 19 June 2016 at 05:37:01 UTC, Walter Bright wrote: On 6/18/2016 10:22 PM, Suliman wrote: 8. create a greenthreads module that works like Goroutines But we already have fibers, I thought that they are same with Goroutines Fibers are thread local. Goroutines are distributed among fibers and threads, and can switch from one thread to another. Alright. What is the goal here ? Do people really want the threads? You don't get them with gccgo either, do you? Isn't the common request more about the channels Go provides?
Re: Phobos Action Items
On Sunday, 19 June 2016 at 06:37:03 UTC, ZombineDev wrote: I agree. BTW, what do you think about improving the compiler's understanding of Fibers (AFAIU they're purely a runtime thing), so e.g. they're usable at CTFE. See this for more details: http://lists.llvm.org/pipermail/llvm-dev/2016-June/100838.html I saw that but I'm unclear what is the cost benefit at this point. Which is an elaborate way to say I don't know.
Re: Phobos Action Items
On Sunday, 19 June 2016 at 01:01:30 UTC, Nicholas Wilson wrote: On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: 9. create a module that enables code to be run on GPUs (John Colvin is doing work on this, ask him how to help!) Now that I'm on my (southern hemisphere) winter break I will be working on getting LDC to emit PTX and SPRIV and if I get time whatever metal uses to be able to use D as the source language for CUDA, OpenCL and Metal respectively. This should hopefully enable seamless integration with the use of introspection and whatnot. This would give us a huge advantage, being able to target the three major compute APIs. This would be awesome. I would love to get my hands on prototypes of this, no matter how buggy and unfinished they are at any given point.
Re: Phobos Action Items
On Sunday, 19 June 2016 at 06:28:45 UTC, deadalnix wrote: On Sunday, 19 June 2016 at 05:37:01 UTC, Walter Bright wrote: On 6/18/2016 10:22 PM, Suliman wrote: 8. create a greenthreads module that works like Goroutines But we already have fibers, I thought that they are same with Goroutines Fibers are thread local. Goroutines are distributed among fibers and threads, and can switch from one thread to another. Alright. What is the goal here ? First, Fiber are doing some synchronized work. This must go. They can't be exchanged across thread safely so this is only make performance worse without any benefit. Second, while go move goroutine from one thread to another, the go scheduler goes to great length to avoid doing it. There is all kind of bad side effects coming from this and you don't want them. HHVM does not move requests from one thread to another. It is to be noted that go to not provide any control over scheduling, while D does, doesn't have thread local storage and the type system doesn't provide any sharing constraints. This is important because we do. If we want to make that work, there are implications. First, it become impossible to cache the TLS segment address across function calls as any function call can mean the thread has changed. Even with that deoptimization (that all code will need to pay, not only the code using these goroutines) that means the type system is now completely broken. Any goroutine can take the address of a thread local, store it on the stack and voila, it is now shared across threads. So that means you need in addition DIP30 or something alike to make sure that the only things passed to the goroutine is shared. In addition, you want to make sure these do not touch the TLS, so you either need some kind of new attribute, like pure_tls, or just make them pure altogether (which would reduce their utility greatly). TL;DR : It sucks, there are all kind of bad side effects. The only reason go does it, is because they goroutine everything and can only have an adaptive strategy as this is done in the runtime. Now on the rant. We don't need more stuff. We have plenty of stuff. We have plenty of unfinished stuff and this is what we should focus on. We have plenty of stuff that try to be 2 things at once and does both badly. We don't need anymore of this. Just remove all synchronization from fiber and be done with it. I agree. BTW, what do you think about improving the compiler's understanding of Fibers (AFAIU they're purely a runtime thing), so e.g. they're usable at CTFE. See this for more details: http://lists.llvm.org/pipermail/llvm-dev/2016-June/100838.html
Re: Phobos Action Items
On Sunday, 19 June 2016 at 05:37:01 UTC, Walter Bright wrote: On 6/18/2016 10:22 PM, Suliman wrote: 8. create a greenthreads module that works like Goroutines But we already have fibers, I thought that they are same with Goroutines Fibers are thread local. Goroutines are distributed among fibers and threads, and can switch from one thread to another. Alright. What is the goal here ? First, Fiber are doing some synchronized work. This must go. They can't be exchanged across thread safely so this is only make performance worse without any benefit. Second, while go move goroutine from one thread to another, the go scheduler goes to great length to avoid doing it. There is all kind of bad side effects coming from this and you don't want them. HHVM does not move requests from one thread to another. It is to be noted that go to not provide any control over scheduling, while D does, doesn't have thread local storage and the type system doesn't provide any sharing constraints. This is important because we do. If we want to make that work, there are implications. First, it become impossible to cache the TLS segment address across function calls as any function call can mean the thread has changed. Even with that deoptimization (that all code will need to pay, not only the code using these goroutines) that means the type system is now completely broken. Any goroutine can take the address of a thread local, store it on the stack and voila, it is now shared across threads. So that means you need in addition DIP30 or something alike to make sure that the only things passed to the goroutine is shared. In addition, you want to make sure these do not touch the TLS, so you either need some kind of new attribute, like pure_tls, or just make them pure altogether (which would reduce their utility greatly). TL;DR : It sucks, there are all kind of bad side effects. The only reason go does it, is because they goroutine everything and can only have an adaptive strategy as this is done in the runtime. Now on the rant. We don't need more stuff. We have plenty of stuff. We have plenty of unfinished stuff and this is what we should focus on. We have plenty of stuff that try to be 2 things at once and does both badly. We don't need anymore of this. Just remove all synchronization from fiber and be done with it.
Re: Phobos Action Items
On Sunday, 19 June 2016 at 05:37:01 UTC, Walter Bright wrote: On 6/18/2016 10:22 PM, Suliman wrote: 8. create a greenthreads module that works like Goroutines But we already have fibers, I thought that they are same with Goroutines Fibers are thread local. Goroutines are distributed among fibers and threads, and can switch from one thread to another. Which is inherently suboptimal and is a part of Go marketing bullshit not worth spending time on. It also requires heavy runtime modifications (because TLS) unless one wants to totally screw plain fibers. Proper action item instead would be providing standard event loop and task system in Phobos.
Re: Phobos Action Items
On 6/18/2016 10:22 PM, Suliman wrote: 8. create a greenthreads module that works like Goroutines But we already have fibers, I thought that they are same with Goroutines Fibers are thread local. Goroutines are distributed among fibers and threads, and can switch from one thread to another.
Re: Phobos Action Items
8. create a greenthreads module that works like Goroutines But we already have fibers, I thought that they are same with Goroutines
Re: Phobos Action Items
On 6/18/2016 6:01 PM, Nicholas Wilson wrote: On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: 9. create a module that enables code to be run on GPUs (John Colvin is doing work on this, ask him how to help!) Now that I'm on my (southern hemisphere) winter break I will be working on getting LDC to emit PTX and SPRIV and if I get time whatever metal uses to be able to use D as the source language for CUDA, OpenCL and Metal respectively. This should hopefully enable seamless integration with the use of introspection and whatnot. This would give us a huge advantage, being able to target the three major compute APIs. Yup. Fire away!
Re: Phobos Action Items
On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: 9. create a module that enables code to be run on GPUs (John Colvin is doing work on this, ask him how to help!) Now that I'm on my (southern hemisphere) winter break I will be working on getting LDC to emit PTX and SPRIV and if I get time whatever metal uses to be able to use D as the source language for CUDA, OpenCL and Metal respectively. This should hopefully enable seamless integration with the use of introspection and whatnot. This would give us a huge advantage, being able to target the three major compute APIs.
Re: Phobos Action Items
On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: Starting a new thread with this, as requested. There are things here for all skill levels and time commitments. There are a good number of things here that are simple to change. I might make a "How to Start Contributing to D" article with some examples of how to fix some of these. 4. make sure every function in Phobos has an example 5. make sure every function in Phobos has Params: and Returns: sections As a shameless self plug, I have two PRs open to fix some lack-luster docs: https://github.com/dlang/phobos/pull/4422 https://github.com/dlang/phobos/pull/4320
Re: Phobos Action Items
On 6/18/2016 1:47 PM, Jakob Bornecrantz wrote: Please don't use the term gorourines when describing a bunch of library functions. There is more to them then just functions. You will need compiler and GC support for the special stack that they have. I know. I just didn't want to write a detailed list of requirements in what was just a bullet point intended as a starting point.
Re: Phobos Action Items
On Saturday, 18 June 2016 at 20:04:50 UTC, Walter Bright wrote: 8. create a greenthreads module that works like Goroutines Please don't use the term gorourines when describing a bunch of library functions. There is more to them then just functions. You will need compiler and GC support for the special stack that they have. Cheers, Jakob.
Phobos Action Items
Starting a new thread with this, as requested. There are things here for all skill levels and time commitments. 1. eliminate all gratuitous use of gc 2. review all Phobos modules for compatibility with ranges - std.zip, for example, was done before ranges and does not work with them 3. use -cov to improve code coverage of Phobos unittests 4. make sure every function in Phobos has an example 5. make sure every function in Phobos has Params: and Returns: sections http://www.digitalmars.com/d/archives/digitalmars/D/Phobos_Documentation_-_call_to_action_258777.html 6. replace std.xml with something we can be proud of that is second to none in performance (Robert burner Schadek is mentoring on this https://github.com/lodo1995/experimental.xml) 7. compile a list of popular modules in other languages and see what we should have 8. create a greenthreads module that works like Goroutines 9. create a module that enables code to be run on GPUs (John Colvin is doing work on this, ask him how to help!) 10. now that D can interface to C++ templates and exceptions, create the interface code to the C++ STL, put it in core.stdcpp 11. review all of Phobos for @safe compatibility - see some of the library issues here https://issues.dlang.org/buglist.cgi?bug_status=NEW_status=ASSIGNED_status=REOPENED=druntime=phobos=safe_type=allwords_id=209021_format=advanced 12. remove dependency on autodecode from Phobos 13. there was recently a thread on Phobos desperately needing a BigDecimal module, someone helpfully posted links to a couple of free C versions, I suggest that someone build a D wrapper for one or convert one to D. 14. write articles about D 15. pick a bugzilla issue and work on it https://issues.dlang.org/buglist.cgi?bug_status=NEW_status=ASSIGNED_status=REOPENED=druntime=phobos_id=209020_format=advanced 16. this list is incomplete, it is off the top of my head I've learned to not ask for a gui library :-)
