Re: DMD Symbol Reference Analysis Pass
On Thursday, 28 May 2015 at 21:23:59 UTC, Per Nordlöw wrote: https://github.com/nordlow/justd/blob/79cc8bf0766282368f05314d00566e7d234988bd/bylinefast.d#L207 which is currently deactivated. It has worked flawlessly in my applications, so none AFAIK. Could this replace the stuck https://github.com/D-Programming-Language/phobos/pull/2794?
Re: Proof of concept - library AA
On Sunday, 24 May 2015 at 15:13:41 UTC, Vladimir Panteleev wrote: Could you elaborate on what these magic semantics are? and no easy solution exists for the ++aa[key1][key2] case. Is this specific to the pre-increment? aa[key1][key2]++ is generally a useful pattern. This applies to pre/post increment as well as assignment and opOpAssign. When an lvalue is needed the compiler will call a special runtime function GetX to obtain an lvalue for aa[key1], i.e. the entry will be default initialized iff missing. If the expression is an rvalue though (aa[key1][key2]), a missing key1 will trigger a range error. In an opIndex(Key) you have no idea whether the whole expression I an lvalue or an rvalue. IIRC the construction/assignment of a value is also handled specifically.
Re: Proof of concept - library AA
On Monday, 25 May 2015 at 23:39:18 UTC, IgorStepanov wrote: On Sunday, 24 May 2015 at 14:13:26 UTC, Martin Nowak wrote: Would be interesting to get some opinions on this. https://github.com/D-Programming-Language/druntime/pull/1282 BTW, I have one idea. We may declare the AA ABI: AA is a pointer to the next layout: This is intended as temporary solution to simplify the transition to a library aa. Extending this conversion to other types for against the goal to replace the built-in AA.
Re: shared libs for OSX
On Tuesday, 26 May 2015 at 16:25:52 UTC, bitwise wrote: Isn't it better to avoid private undocumented functions? Not only better, but mandatory, otherwise Apple will reject the app from the app store. Calling back into an unloaded image without proving a mean to deregister the callback is a bug, so you might fix it. Looking at the code [1] again there is also an interesting sImagesToNotifyAboutOtherImages. [1]: http://opensource.apple.com/source/dyld/dyld-95.3/src/dyld.cpp
Re: shared libs for OSX
On Wednesday, 27 May 2015 at 06:45:49 UTC, Jacob Carlborg wrote: I'm not sure. The ___tls_get_addr function [1] is used when accessing a TLS variable on OS X. In all native implementations, both on OS X and Linux, the parameter is not just a void* but struct containing the image header as well. On Linux you call it with an dso index and an offset. The DSO index is assigned by the runtime linker (there is a special relocation for that). Something similar could be done manually if not natively supported on OSX.
Re: shared libs for OSX
On Tuesday, 26 May 2015 at 16:25:52 UTC, bitwise wrote: Since all global functions and symbols are shared between images anyways, receiving the callback in the main image would be fine. So in this case, unregistering the callbacks is no longer needed. That only works when the host executable is linked against druntime, but falls when you load runtime dynamically, e.g. a D plugin for a C host.
Re: shared libs for OSX
On Monday, 25 May 2015 at 19:40:52 UTC, bitwise wrote: 1) _dyld_register_func_for_add_image should be taken care of with the above two fixes You still cannot unregister the callback, so it can't be used for dynamically loading druntime. Last time we talked about this problem, we found some undocumented function that could be deregistered. 2) __attribute__((constructor/destructor)) can be added to druntime when building for osx like in the file dylib_fixes.c [1] For linux we let the compiler emit comdat constructors into every D object, so you'll end up with exactly a single function for any binary containing D code. I don't think you need ctors/dtors on OSX if you already have the dylib callback. 3) copy paste rt_init/rt_term, rename them to dylib_init/dylib_term and remove everything except whats needed to initialize a shared lib's image. Not sure what you want to copy, but for sure you need to extend sections_osx to handle multiple dylibs. It should be very similar to _d_dso_registry for ELF, except that you iterate over the sections of a dylib image to get EH tables and ModuleInfo.
Re: shared libs for OSX
On Monday, 25 May 2015 at 14:33:43 UTC, bitwise wrote: At this point, my impression is that it would be very impractical, if not impossible to have separate druntimes for each shared library. Even when you do link separate runtimes, dyld still treats all the exported symbols as shared. Yes, you can't mix them with a D executable.
Proof of concept - library AA
Would be interesting to get some opinions on this. https://github.com/D-Programming-Language/druntime/pull/1282
Re: std.allocator: FreeList uses simple statistics to control number of items
On Wednesday, 20 May 2015 at 17:28:50 UTC, Andrei Alexandrescu wrote: 1. How to update Pmiss efficiently? Most allocations should be fast, so it shouldn't be update with every call to allocate(). What I have now is I update every K calls. A one-pole low pass filter is a very efficient moving average. https://github.com/D-Programming-Language/druntime/blob/6e55b7aaff7566d374c2f253f831d3489e7fd1a5/src/gc/gc.d#L1732 Feed it with 1 on hit and 0 on miss to get Phit on average.
Re: std.allocator: FreeList uses simple statistics to control number of items
On Wednesday, 20 May 2015 at 17:28:50 UTC, Andrei Alexandrescu wrote: There is a bothersome issue with freelists fronting general-purpose allocators (https://en.wikipedia.org/wiki/Free_list): they can grow indefinitely. Because they keep memory allocated in their parent, they cause fragmentation to it. You can look at jemalloc which has a GC algorithm for this problem. https://m.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919
Re: shared libs for OSX
On Thursday, 21 May 2015 at 01:31:37 UTC, bitwise wrote: I've been reading over all the bugs and conversations I can find, but I'm still confused about exactly what's going on here. What I need: -load a dynamic library on OSX using dlopen or Runtime.loadLibrary -use dlsym to retrieve my functions and set up a gc proxy Forget about the GC proxy, it's a hack to work around ODR issues. What you really need is a shared phobos library as we have on linux or freebsd. -pull interfaces of classes defined in the dynamic library into my program for use. If you want to exchange data and code across multiple shared libraries and your executable the runtime must be fully aware of all shared libraries to support casting ,exceptions and finalizers.
Re: shared libs for OSX
On Thursday, 21 May 2015 at 08:07:49 UTC, Jacob Carlborg wrote: I don't think anyone is working on this. Native TLS is a prerequisite which requires changes both to the compiler and the runtime. There's an enhancement request for native TLS [1]. One could also make emulated TLS work with shared libraries.
Re: shared libs for OSX
On Wednesday, 20 May 2015 at 21:35:38 UTC, bitwise wrote: Heh.. That's pretty useless. Any idea on the state of things? Like if there are plans to support this in the near future(few months)? I couldn't find many conversations on this. This kinda-totally ruins my plans(on OSX at least). Have you tried LDC? They recently got shared library support, maybe also on OSX.
Re: shared libs for OSX
On Thursday, 21 May 2015 at 15:51:38 UTC, bitwise wrote: I'm not sure exactly what you mean about "integrated runtime". I think he means Phobos/Druntime as shared library. Looking in /usr/share/dmd/lib, I see phobos as a static library (which I'm assuming includes druntime, which I don't see anywhere). If the runtime is linked as a static library, shouldn't it be able to link to a dynamic library and just work? I think a separate runtime is what I was expecting. Yes separate shared libraries (with multiple runtimes) work on every other platform. The problem for OSX is that onAddImage gets called for the executable and every shared library. It would be trivial to only register the image containing the current runtime, by comparing the address of a private symbol with the memory region of the images. https://github.com/D-Programming-Language/druntime/blob/6331ab1ae19f3ff82449a5734b59d81b128685f4/src/rt/sections_osx.d#L186
Re: 0 is not a power of 2
On Tuesday, 19 May 2015 at 07:56:27 UTC, Atila Neves wrote: Aren't predictable branches cheap on current architectures? Yes they are, and it seems one would rarely if ever call isPowOf2 with 0 in std.allocator. A good thing to do, is to use a good hardware event profiler like perf, and record the branch misses. Perf is also the right tool to compare the branchless version (I doubt it's significantly better, especially since the felt of the 0 branch is just a constant).
Re: [Request for ABI Breakage]: Ambiguity between extern(Pascal) vs. Template Value Parameters.
On Saturday, 16 May 2015 at 09:07:57 UTC, Iain Buclaw wrote: or have it's mangled symbol be renamed to something other than a 'V' that does not clash with other types/identifiers. Well, change it to whatever letter works, if that resolves mangling ambiguities.
Re: on the length of symbols
On Monday, 11 May 2015 at 21:18:53 UTC, Walter Bright wrote: D already does (for Win32) a generic compression on names. It works a lot better than poorly reinventing compression - it's far less complex, less buggy, easier to implement, etc. I'd support adding a Win32-like compressor. The change I'd make is having it only use identifier characters for the compressed result. Yeah, that would solve the problem. https://github.com/D-Programming-Language/dmd/blob/932e0a5c2f73b410f0a61ad721e78870ecf17510/src/backend/cgobj.c#L3760
Re: DLL symbol identity
On Friday, 8 May 2015 at 05:26:01 UTC, Benjamin Thaut wrote: And Step 2) at program start up time. This means that symbols don't have identity. If different shared libraries provide the same symbol it may exist multiple times and multiple instances might be in use. Can you elaborate a bit on that? How would you run into such an ODR violation, by linking against multiple import libraries that contain the same symbol? Any opinions on this? As both options would be quite some work I don't wan't to start blindly with one and risking it being rejected later in the PR. Last time we thought about this we came to the conclusion that global uniqueness for symbols isn't possible, even on Unix when you have 2 comdat/weak typeinfos for template classes in 2 different shared libraries but not in the executable. I suggested that we could wrap typeinfos for template types in something like TypeInfo_Comdat that would do a equality comparison based on name and type size.
Re: on the length of symbols
On Monday, 11 May 2015 at 09:33:42 UTC, weaselcat wrote: here's a single symbol from my project The underlying problem is that symbols names grow quadratically when combining templated ranges, because each range's name is a template argument to the next range. Sometimes name occur twice, as template argument and return type. An obvious solution to the problem is to adopt a compression scheme similar to C++ mangling on Windows. https://github.com/D-Programming-Language/dmd/blob/c392c80c2f04a56701fed3a61a5c4df4571a3573/src/cppmangle.c#L1712 This is a well known and important problem but has a lower priority than many other issues. You can help to solve it by opening a bugzilla issue (might already exist) and finding out how to integrate such a backreference compression with D's current mangling. http://dlang.org/abi.html#MangledName
Re: ARM Cortex-M Microcontroller startup files
On Monday, 27 April 2015 at 17:25:50 UTC, Johannes Pfau wrote: Since 2.066 the binaries on gdcproject.org are built with crosstool-NG (and an additional D script) in a docker container. That's interesting, do the Windows binaries have some binary dependency on MinGW? It currently seems that we'd to build ddmd with GDC or LDC to get a fast enough compiler. Maybe that would allow us to cross-compile release as well. Currently I'm using virtual box with 5 different OSes to build a release.
Re: ARM Cortex-M Microcontroller startup files
On Sunday, 3 May 2015 at 00:59:07 UTC, Mike wrote: I suggest refraining from requiring or preventing any feature, including garbage collection and exceptions. Rather, we can gradually make each feature available as the need arises, and the user can opt in and make their own tradeoffs. Yes exactly
Re: ARM Cortex-M Microcontroller startup files
On Sunday, 3 May 2015 at 01:57:45 UTC, Jens Bauer wrote: I'll not be working much on a malloc, but I will be thinking a little about a size-optimized / well-performing malloc could be written (approximately). Perhaps I could combine my MiniMalloc with clusters of small blocks. Newlib already ships with a size optimized malloc. https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=newlib/libc/stdlib/nano-mallocr.c;h=8d6ca5ce05a89853e40f80b583f0680a77bd4b67
Re: The hackathon week roundup
On Sunday, 3 May 2015 at 04:15:58 UTC, Mike wrote: My idea: 1. Members of the D leadership/committers form a working group. 2. The working group creates of list of bugs they are willing to work on. 3. Hackathon is announced. To motivate participants, the working group agrees to fix a bug of the winner's choosing. 4. After the hackathon, the working group subjectively chooses a winner from the participants. 5. The winner discloses which bug they would like fixed, and the qualified members of the working group agrees to fix it for the next release. 7. Wash, rinse, repeat. Good idea
Re: ARM Cortex-M Microcontroller startup files
On Saturday, 2 May 2015 at 15:15:50 UTC, Jens Bauer wrote: Will it be possible to have associative arrays without garbage collection ? You can write an AA container. A RefCounted AA implementation might allow unsafe escaping though. What about dynamic strings and dynamic arrays, don't they need GC ? Same here array slices require a GC to be safe, but one could implement them like std::vector. While built-in arrays and AAs are nice to have it's trivial to replace them with other containers, no need for GC. I never even needed dynamic memory allocation on a microcontroller.
Re: if(arr) now a warning
On Friday, 1 May 2015 at 19:45:18 UTC, Walter Bright wrote: I suspect this kind of change and check would be appropriate for a D linter, and not be part of the core language. Yes it would fit nicely into a static analyzer, but DMD is currently the only semantic analyzer we have.
Re: ARM Cortex-M Microcontroller startup files
On Saturday, 2 May 2015 at 04:53:51 UTC, Jens Bauer wrote: Is it possible to write the malloc so it's "garbage collector-friendly" ? Garbage collection on microcontrollers doesn't make sense, because the memory consumption will always be significantly higher than with deterministic memory management.
Re: ARM Cortex-M Microcontroller startup files
On Saturday, 2 May 2015 at 04:53:51 UTC, Jens Bauer wrote: The problem I've seen with most C-solutions, is that once someone uses printf, the binary suddenly grows out of proportions. (It may be because the programmer included a line for debugging only, and that causes the otherwise 1K program to be huge and almost not fit the MCU). Granted printf isn't small, but it's not that large either, sure you're not including %f float formatting? https://launchpadlibrarian.net/200699979/readme.txt
Re: ARM Cortex-M Microcontroller startup files
On Saturday, 2 May 2015 at 08:46:56 UTC, Timo Sintonen wrote: Std.format, as suggested, would be too big. I tis easty to copy the printf formatter from libc sources. Or just write an own. No need to rewrite libc, just link against it and use whatever is needed. It is a matter of taste if it is a c like printf function or a formatter class. Or both if we like. Only the used ones are picked from the library. You could implement more high level application code like a formatting library as dub package.
Re: ARM Cortex-M Microcontroller startup files
On Saturday, 2 May 2015 at 08:33:34 UTC, Timo Sintonen wrote: It is ok for me and it is used in our production code that does not yet use D. I am still open for other solutions. These functions should be at least extern C because library code written in C may use them. Newlib already comes with a malloc implementation, you just need to implement _sbrk_r. https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=newlib/libc/stdlib/nano-mallocr.c;h=8d6ca5ce05a89853e40f80b583f0680a77bd4b67
Re: ARM Cortex-M Microcontroller startup files
On Friday, 1 May 2015 at 06:57:08 UTC, Timo Sintonen wrote: * Is dynamic memory allocation a requirement of D, or a library feature? We should agree whether we are making only yet another C compiler or do we want the D compiler. The ability to use object oriented features was the reason I started with D. I think we do npot need full gc but I want to create objects at least at start. they will live trough the program so I have no need to delete then. You can use malloc+emplace, scoped!Class, or put the instance in the data segment.
Re: ARM Cortex-M Microcontroller startup files
On 04/30/2015 08:43 AM, Timo Sintonen wrote: > Printf is a little tricky. It is actually a file operation to stdout and > that is actually a syscall to kernel. No, you usually have to implement some hook for outputting yourself, e.g. putc or write, printf solely takes care of the formatting.
Re: ARM Cortex-M Microcontroller startup files
On 04/30/2015 01:30 PM, Mike wrote: > While this may seem simple to achieve, I think it will raise a few > questions that will need answering. > > * Can ModuleInfo be leveraged, without introducing overhead, to call > module constructors and static constructors? They might be useful for > hardware initialization. Currently D sorts modules by initialization order,which requires every moduleinfo to contain an array of imported modules. Quite a lot of RAM for a nice to have feature, so we should drop it or at least provide a -fno-moduleinfo switch. > * Is dynamic memory allocation a requirement of D, or a library feature? Definitely a library feature, though many language features, like array appending, won't work without it (delegate closures won't even work without a GC). > * Does D need the C runtime, or can it init on its own? It shouldn't need a C runtime. > * Should the C standard library bindings be part of the runtime, or > exist as an external Deimos library? Either way they can still be used > by the runtime, I'm just suggesting that they should be encapsulated. It doesn't cost you anything to include the bindings in a release, they could be maintained in a separate project if that helps. > * What will be done about TypeInfo for now? It's causing me some > serious code-bloat problems. See > http://forum.dlang.org/post/quemhwpgijwmqtpxu...@forum.dlang.org Implement a -fno-rtti switch for GDC. > * Is data and bss initialization part of the runtime, or delegated to > toolchain, silicon, and BSP vendors? We should provide appropriate linker scripts and do the initialization.
Re: if(arr) now a warning
On 04/29/2015 09:15 PM, Jonathan M Davis wrote: > Yeah, but I think that it's safe to say that std.allocator is not the > normal case std.allocator really isn't a general example. I'm exclusively using if (ary.length) or if (!ary.empty) in my code to avoid the problem. Occasionally I'm using if (auto ary = func()), despite the fact that the semantics are wrong, but it's nice and short and works as long a func always returns null instead of empty slices. But it's very fragile, hard to spot during debugging, and I already spent too many hours on that. So I'd expect any code analyzer to fault such usage and any D book to teach people not to use arrays as boolean, at which point we'd be better off to slowly remove it from the language.
Re: ARM Cortex-M Microcontroller startup files
On 04/27/2015 03:42 PM, Timo Sintonen wrote: > > The basic idea has been to make as little changes as possible. I started > by compiling object.d and then added files and modified them one by one > until there were no compile or link errors. Then I added other files > that could be compiled without errors. It is not guaranteed that all > features work and the list of files have changed from version to version. Wonder if it makes more sense to start from zero and add as few files as possible. Druntime doesn't do much useful stuff for a µC anyhow. - GC and rt.lifetime (new, arrays) - Moduleinfo - EH unwind support - AA implementation - vectorized array ops - core.time/sync/thread/demangle What might be interesting is this. - core.bitop - maybe core.atomic - some gcc simd module - libc bindings for core.stdc.math and core.stdc.stdio for printf
Re: [hackathon] My and Walter's ideas
On Monday, 27 April 2015 at 10:56:17 UTC, Steven Schveighoffer wrote: Everything to alter is in lifetime.d. It would be trivial to create this. https://issues.dlang.org/show_bug.cgi?id=13988 The only thing is to have a malloc-based AA for tracking https://github.com/D-Programming-Language/druntime/blob/18d57ffe3eed8674ca2052656bb3f410084379f6/src/rt/util/container/hashtab.d However, note that this wouldn't track allocations that the compiler did for closures. Plain _d_allocmemory.
Re: Adding Radix Sort into Phobos
On 04/27/2015 09:52 AM, "Per =?UTF-8?B?Tm9yZGzDtnci?=
" wrote:
> Does someone have any answers to these questions or should I wait until
> the prel. pull request is done?:
>
> •Figure out a way to template-parameterize radixSortImpl to make it
> work on aggregate element types when the comparison function can be
> expressed as an data-member-accessor of the aggregate. For example if
> ElementType is a struct S { int x, y; } and comparison function "a.x <
> b.x".
I think an alias transform function that defaults to identity makes most
sense.
That's a bit similar to
http://dlang.org/phobos/std_algorithm_sorting.html#schwartzSort.
And it doesn't look like you support custom predicates.
> •If possible implement a generic sorting algorithm that automatically
> selects the best suitable in-Place sorting algorithm based on types of
> the input parameters (range and comparsion/accessor function). This
> currently means isIntegral, float, double, and as above mentioned
> aggregates sorted on a member or combination of members whose bijection
> can fit into 8, 16, 32 or 64 bits.
That's a tough problem => treat it separately.
Re: ARM Cortex-M Microcontroller startup files
On Monday, 27 April 2015 at 05:30:55 UTC, Timo Sintonen wrote: One of the biggest issues has been the multilib build. If it is solved now we are one step closer to be able to build binaries. Great, I tried to find out how GDC binaries are build, but couldn't find any script. How much stuff do you strip out of runtime for minilibd? It still seems to contain fat typeinfo and moduleinfo. We'd probably need the equivalent of -no-rtti and -fno-exceptions and add -fno-moduleinfo. Maybe building a few different configurations of minilibd makes sense. The ARM toolchain comes with a nano.spec to select newlib-nano and size optimized libc++. https://launchpadlibrarian.net/200699979/readme.txt
Re: Quit running foreign unittests >_
https://issues.dlang.org/show_bug.cgi?id=13454
Re: Quit running foreign unittests >_
A fairly simple solution to this would be to compile unittests only for root modules (the ones that are part of the compilation), not for imported modules. Then everyone can decide for which modules to use -unittest.
Re: ARM Cortex-M Microcontroller startup files
On 04/26/2015 07:29 PM, Jens Bauer wrote: > I may be able to put together some kind of recipe for building the > GCC+GDC I have. > Unfortunately, I have not been able to build with multilib yet, so my > setup cannot build code for Cortex-M0; it keeps stuffing Cortex-M3 and > Cortex-M4 instructions in there. The wiki says to disable multilib (http://wiki.dlang.org/Bare_Metal_ARM_Cortex-M_GDC_Cross_Compiler#Build_GCC), what's the problem? Maybe ask Iain/Johannes if it's GDC specific. > -But if I build GCC without GDC, then I can build with multilib, and > make code for Cortex-M0. So I cannot have both. > > I know Mike have been working on this. Hopefully one of us will find a > solution. What libc are you using? The gcc-arm-embedded project uses newlib and newlib-nano. https://launchpad.net/gcc-arm-embedded
Re: Adding Radix Sort into Phobos
On 04/26/2015 09:16 AM, "Per =?UTF-8?B?Tm9yZGzDtnci?= " wrote: > I have a radix sort implementation at > > https://github.com/nordlow/justd/blob/master/intsort.d#L92intsort.d > > which beats Phobos own Quicksort by a factor 1.5 to 4 depending on > element type (Intergral or FloatingPoint). > > Anyone up for the job of adapting and merging it into Phobos' > std.algorithm.sorting? Code seems to be pretty done (although there are lots of TODOs). Why not simply convert it into a pull request?
Re: [OT] Nim is usable with PNaCl
On 04/26/2015 08:18 AM, Rikki Cattermole wrote: > > I already asked on D.ldc. No reply. > I even included links to the differences between what PNaCL supports and > LLVM IR. What are they supposed to say? Obviously doable but requires lots of work. This is one of those particular interests that won't get implemented unless a stakeholder writes the code.
Re: AA Performance in Benchmarks
On 04/26/2015 12:58 PM, Daniel Murphy wrote: > > Yes, if we had an AA in phobos. Yes, I think phobos should get a few more optimized containers. SparseSet DenseSet SparseHash DenseHash They should be configurable w.r.t. load factor, equality comparison, and allocation policy (when std.allocator is available). http://sparsehash.googlecode.com/svn/trunk/doc/index.html As the builtin druntime AA will likely always remain a general purpose hash table we can't do all the possible optimizations.
Re: ARM Cortex-M Microcontroller startup files
On 04/26/2015 05:55 PM, Jens Bauer wrote: > Done. > http://wiki.dlang.org/Microcontroller_startup_files > > -This is my first successful Wiki page, BTW. :) Nice, minilibd seems to be maintained as well, you happen to know the author? I'd really like to see binary releases of GDC for arm-none-eabi that ship with a patched compiler (iff necessary) and minilibd.
Re: AA Performance in Benchmarks
On 04/23/2015 05:29 PM, Steven Schveighoffer wrote: > > https://github.com/D-Programming-Language/druntime/pull/1229 > > It would be interesting to know how the new AA performs in this test. Went from 11s down to 9s, ~20% improvement.
Re: Range of chars (narrow string ranges)
On 04/24/2015 10:44 PM, Walter Bright wrote: > 4. Autodecoding is inefficient, especially considering that few > algorithms actually need decoding. Re-encoding the result back to UTF8 > is another inefficiency. > > I'm afraid we are stuck with autodecoding, as taking it out may be far > too disruptive. > > But all is not lost. The Phobos algorithms can all be fixed to not care > about autodecoding. The changes I've made to std.string all reflect that. It probably won't be too disruptive to optimize algorithms such as filter to return a range of chars, but only if we support such ranges as narrow strings everywhere.
Range of chars (narrow string ranges)
Just want to make this a bit more visible. https://github.com/D-Programming-Language/phobos/pull/3206#issuecomment-95681812 We just added entabber to std.phobos, and AFAIK, it's the first range algorithm that transforms narrow strings to a range of chars, instead of decoding the original string and returning a range of dchars. Most of phobos can't handle such ranges like strings and you'd have to decode them using byDchar to work with them.
Re: code.dlang community transfer repository
On 04/23/2015 05:43 PM, Jesse Phillips wrote: > > In order to keep the projects in code.dlang.org relevant, I think it is > important that we provide a way to have the primary project change > hands, rather than require the fork be placed on to code.dlang.org too[2]. Write a mail to Söhnke, he can change the project owner. In the mid-term we want to have admins for dub-registry (https://github.com/D-Programming-Language/dub-registry/issues/90).
Re: code.dlang community transfer repository
On 04/23/2015 08:49 PM, Steven Schveighoffer wrote: > If you are going to fork, you need to give it a new name. This is > standard practice for open source projects. It's a common practice on github to take over small but no longer maintained projects. You give credit to the original author and things should be fine.
Re: code.dlang community transfer repository
On 04/24/2015 12:08 PM, extrawurst wrote: > > let's just use the github stars system. i guess the github API allows us > to query those too. maybe we can even star a project from the registry > too ?! Interesting idea, but github centric. We already have download numbers and would just need to cache and show them in a prominent place. You can already add a badge to your project with it's download numbers, see http://shields.io/.
Re: WTF: dmd 2.066 vs. dmd 2.067 really dangerous code breakage
On Friday, 24 April 2015 at 02:09:06 UTC, Walter Bright wrote: On 4/23/2015 6:26 AM, Steven Schveighoffer wrote: I agree it should have been done, not saying it's OK to break the process in some cases. I'm just explaining why it probably happened the way it did. Yes, it should have been done. We screwed up. It's time that we agree on/document an official deprecation approach and rigorously enforce it, making as few exceptions as possible. As it stands now, everyone follows their own policy. Any volunteer to put this in a DIP?
Re: SDC needs you -- redux
On 04/18/2015 07:21 PM, Shammah Chancellor wrote: > 1) SDC, libd-llvm, and libd together are ~ 30KLOC of idiomatic and very > intelligible D code. Compare this with the 300KLOC for the DMD frontend. > 1.1) The 30KLOC is clean and easy to get into. It took me less than > 3 days to get into it and start implementing language features. > 1.2) This is enabled by the use of a scheduler to easily handle > complex Dlang features. > 1.3) This will enable advanced compiler normally be very difficult > to implement > 2) SDC supports a majority of the language at this point, the remaining > work is mostly trivial. (When I say majority, I mean majority in terms > of LOC needed to support, not in terms of implemented pieces of syntatic > sugar) > 2.1) We won't need another 250KLOC to get to 100% feature parity. > We're already at something like 80% feature parity by my estimation. > 3) SDC is designed also to be a library that can used for tooling in an > easy fashion! > 3.1) I work with Golang on a daily basis professionally. The > tooling for golang is a major reason for it's adoption. This tooling > looks like gofix, gofmt, govet, etc. We need this tooling to be able to > succeed. > 4) SDC use of fibers and a scheduler allows us to have a very robust > error handling mechanism that DMD will never be able to implement. We > emit beautiful clang-style errors telling you EXACTLY what is wrong with > your code. > 5) SDC is fast. It is actually faster than DMD when you move into > template and CTFE land. > 6) SDC leverages all the work done by the HUGE llvm group. > 7) SDC uses the LLVM JIT for CTFE, this means no double-implementation > of language features for CTFE, and for normal compilation. > 8) Because SDC is written in D, is clean, and uses a JIT this opens up > the door for actually exposing compiler AST Nodes *directly* to the code > being compiled. (! No more __traits horror when doing > metaprogramming) > 9) Because SDC is clean, easy to understand, and easy to implement > language feature on top of, we have been able to find quite a few bugs > in DMD. I have been submitting bug reports back to DMD and actually > causing the mainline D compiler to do a *better* job! It's a really impressive list and it's very much how I'd like dmd to be written. I could offer to work on dub and travis-ci integration, but there doesn't even seem to exists a cmdline interface? bin/sdc --help HELP ! core.exception.RangeError@sdc/src/sdc/main.d(51): Range violation Also some of the TODO items really look very feasible. https://github.com/deadalnix/SDC/issues/125 What I'd like to see though, is a prioritized list of stuff that needs to be done to reach a working D style hello world. And at some point you need to get rid of stack traces of the compiler on compile errors.
Re: WTF: dmd 2.066 vs. dmd 2.067 really dangerous code breakage
On 04/22/2015 01:36 PM, Dicebot wrote: > > -d is your enemy, If you remove that, there will be a clear warning > "Deprecation: variable XXX.S.FLAG_ON immutable field with initializer > should be static, __gshared, or an enum". You decided to ignore and hide > it, why the surprise about the breakage? Maybe using -d should itself emit a warning?
Re: SDC needs you -- redux
On 04/22/2015 08:49 AM, Dan Olson wrote: > Mind if I add it then? I've lived with etags for so long that my > fingers automatically reach for " M-." for D names too. Please do, it would be really helpful, at least to me.
Re: if(arr) now a warning
Interesting trivia, I even found this bug in the documentation of the first dmd release http://downloads.dlang.org/releases/0.x/0.100/dmd.100.zip. Checking For Empty Strings C++ strings use a function to determine if a string is empty: string str; if (str.empty()) // string is empty In D, an empty string is just null: char[] str; if (!str) // string is empty Apparently it was later fixed to if (!str.length) http://www.digitalmars.com/d/1.0/cppstrings.html.
Re: I have made a discovery
On Saturday, 18 April 2015 at 15:24:27 UTC, w0rp wrote:
@nogc
void main() {
throw new Foo("Oh no!");
}
Though until https://issues.dlang.org/show_bug.cgi?id=14119 is
resolved the tracehandler GC allocates anyhow.
Why are malloc exceptions better then gc exceptions? Are you
throwing so many of them that the GC becomes a bottleneck :)?
Re: D Hackathon: April 25 - May 1
On 04/09/2015 04:44 AM, Andrei Alexandrescu wrote: > The idea is to spend a week concentrating efforts on writing code for > all aspects of D (with a focus on, but not limited to, D's github > repos). Should be fun! At the end of the week we'll count the bugs fixed > and pulled PRs and compare them against the weekly average over the past > few months. If it was productive in addition to fun, so much the better! Nice idea, I recently had the idea that we should spent one week fixing bugs together with D first-time contributors. Think that fits nicely in here to also invite new contributors to participate in the hackathon.
Re: Mitigating the attribute proliferation - attribute inference for functions
On Sunday, 12 April 2015 at 22:45:37 UTC, Zach the Mystic wrote: The basic suggestion is that D has a function attribute which expressly indicates that a function is separately compiled, thus eliminating all ambiguity and mystery about what can and can't be inferred. Yeah, as you already seem to know, we want to get `export` in shape to be usable for this and other purposes. See http://wiki.dlang.org/DIP45.
Re: DIP77 - Fix unsafe RC pass by 'ref'
On Sunday, 12 April 2015 at 07:17:07 UTC, Walter Bright wrote: On 4/11/2015 4:33 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= " wrote: On Saturday, 11 April 2015 at 09:41:07 UTC, Walter Bright wrote: A quick read of this suggests it is doing the Rust model of only one mutable reference at a time. Is this really viable with D? It's a very good ownership model IMO, because it has no runtime overhead and often sharing/escaping isn't needed. Would at least be interesting to explore this in more detail for scoped variables. As a side note, I find that unique ownership is strangely missing from our discussions, yet it's a perfect ownership model for resources such as a File. https://github.com/D-Programming-Language/phobos/pull/3171
Re: DIP77 - Fix unsafe RC pass by 'ref'
On Sunday, 12 April 2015 at 12:06:50 UTC, bearophile wrote: I think D Zen asks for safety on default and opt-out on request. Marc talked about unique ownership vs. shared ownership, not about safe vs. unsafe.
Re: Mitigating the attribute proliferation - attribute inference for functions
On Sunday, 12 April 2015 at 09:42:19 UTC, Daniel N wrote:
FYI, I wrote an enhancement request for this already. In case
the discussion reaches a conclusion this time around, you might
want to update the status:
https://issues.dlang.org/show_bug.cgi?id=13567
Oh great, can you help me to turn this into a DIP?
We need to define a sane behavior for when public templates call
private functions.
One idea I'm carrying around for some time is the following.
For functions with inferred attributes the compiler could emit an
additional alias symbol with the mangling of the function before
the inference.
That would allow to link against that function without running
inference and still use inference for the same function
internally.
Note that this relies on covariant attributes, i.e.
void foo() @safe @nogc pure nothrow
is implicitly convertible to
void foo()
This could work out quite well, you only need to add attributes
to your private functions to support public templates, but
typically this should only be few entry points of your library.
One problem arising from that is testing your API.
void tmpl(T)(T t) { return func(t.foo); }
private void func(size_t v) { return v; }
unittest @safe @nogc pure nothrow
{
static struct S { size_t foo; }
tmpl(S()); // works here, but not outside of this module
}
Re: Mitigating the attribute proliferation - attribute inference for functions
On Sunday, 12 April 2015 at 07:12:47 UTC, Walter Bright wrote: At one point, I had implemented it for auto return functions, because the source must exist for them, too, but it got a lot of resistance and was dropped. I thought that was already in, but it isn't. https://github.com/D-Programming-Language/dmd/pull/1877#issuecomment-16353774 The resistance to the pull this isn't directed against attribute inference but against the problems that may arise. Looks like we need to come up with a comprehensive proposal, that addresses those problems. Anyone wants to help with a DIP?
Re: Mitigating the attribute proliferation - attribute inference for functions
On Sunday, 12 April 2015 at 08:55:17 UTC, Walter Bright wrote: There's still an issue of a private function being called by a template, and the definition of that private function is compiled separately. That's the problem of the author of that template. He would still have to annotate that private function manually like today. But everything called by that function doesn't need to be attributed.
Re: Mitigating the attribute proliferation - attribute inference for functions
On Sunday, 12 April 2015 at 02:07:54 UTC, weaselcat wrote: I thought as much, that's unsettling because dmd is already starting to feel dog slow. We might have some sort of performance bug in the latest release. https://issues.dlang.org/show_bug.cgi?id=14431 If you know how to use a profiler, I'd be glad for more info.
Re: Mitigating the attribute proliferation - attribute inference for functions
On 04/12/2015 12:35 AM, weaselcat wrote: > How does this differ from e.g, if the function was templated? It doesn't, if you decide to analyze each imported function to perform attribute inference, then that's just like treating all of them like template instances. This approach doesn't scale to bigger projects, because the compiler always has to analyze the complete source code. That's also the reason why an optimization was added to dmd whereby it doesn't instantiate templates, that are already instantiated in an imported module.
Re: Yes, you can help std.allocator!
On 04/05/2015 05:16 AM, Andrei Alexandrescu wrote: > Right now the allocator design works well as a flexible malloc/free > implementation, but has no support for tracing-based garbage collection. > > I am evaluating whether I should add tracing capabilities to > std.allocator, or stop here. Tracing would most definitely require new > primitives such as embedding some sort of type information and resolving > internal pointers. Just to make sure we're on the same page here. You're talking about implementing the GC in terms of std.allocator, not about making std.allocator GCable, right? This would be fairly interesting for us too experiment with, as we could easily try out different allocator architectures. But then again the GC needs to use highly optimized data structures to lookup metadata, we'll likely only choose a single allocator architecture, and we already know a few good ones https://code.dawg.eu/talks/2015-03-20-garbage_collection_dmeetup/#9. So while the heap layer idea is interesting and would facilitate to try different allocators, it's a substantial amount of work, to add the marking and metadata to std.allocator. We already started to modularize the GC a little and can try out different architectures by hand, so I don't think it's worth the effort. > What would really help this evaluation and any other work on garbage > collection would be a modularization of the tracing process. It's already pretty much like that. http://dlang.org/phobos/core_thread.html#.thread_suspendAll http://dlang.org/phobos/core_thread.html#.thread_scanAll http://dlang.org/phobos/core_thread.html#.thread_resumeAll Scanning is not a range but callback based and misses GC.addRoots and GC.addRanges, wouldn't be too much work to add that. https://github.com/D-Programming-Language/druntime/blob/7877f65a6baf53b0c53c1babf9a0f0a5d1893940/src/gc/gc.d#L2205 There is also no TypeInfo yet. -Martin
Mitigating the attribute proliferation - attribute inference for functions
Sorry to open yet another topic.
I'm repeatedly finding myself in situations where I write functions like
this.
private @property bool empty() const @safe pure nothrow @nogc
{
return impl is null || !impl.count;
}
This is obviously a joke, because the compiler very well knows the
attributes and I don't need to guarantee them as part of an API.
The situation is getting somewhat out of hands and we need to find a way
to make attribute inference for functions feasible.
At the last D Meetup [¹] I had a nice chat with Dicebot about this.
We both had the same notion of how things should be. People should put
attributes on their exported API functions (or main() for apps), thereby
making API guarantees and transitively enforcing them, but the compiler
should infer attributes for internal functions.
This wouldn't help with separate compilation of modules, because the
compiler only knows the attributes of a function after semantically
analyzing it, but it works extremely well for libraries that are
compiled as batch.
And given that separate compilation is inherently inefficient anyhow,
it's well worth to look at this approach in more detail.
The main problem of attribute inference for functions presents as
follows on API borders.
/// An API function that guarantees certain attributes
public size_t foo(string a) @safe @nogc pure nothrow
{
// and calls an internal function
return impl(a);
}
/// An API template function with the same guarantees
public size_t bar(R)(R a) @safe @nogc pure nothrow
{
// that calls an internal function
return impl("something");
}
/// A function with inferred attributes
private size_t impl(string a)
{
return a.length;
}
Applying attribute inference to `impl` would work for `foo` because code
calling `foo` doesn't need to check the attributes of `impl`, except
when foo gets inlined.
This doesn't work for the template function bar, because attributes are
checked when the template is instantiated, leaving us with the following
options on how the compiler could deal with missing attributes of a
function.
1) If that function has source code available, it gets semantically
analyzed and additional attributes are inferred.
2) The compiler errors because of missing attributes.
3) Don't inline functions when that would require to infer attributes of
a private/package function.
4) ???
Option 1 isn't really viable because it turns into a whole program
compilation model.
Option 2 is basically what we have today. Without a clear border of
where the compiler stops to analyze your implementation details, it'd
still be necessary to annotate every function.
Option 3 defines such a border which allows to use attribute inference
for internal functions. In the above example `bar` wouldn't work without
inference on `impl`, but it's possible to manually annotate the private
functions directly used by a template.
Those ideas aren't thought through too well, but it's an important issue
that requires a solution.
-Martin
[¹]: http://www.meetup.com/Berlin-D-Programmers/events/220702368/
Re: if(arr) now a warning
On 04/11/2015 01:03 PM, w0rp wrote: > At the moment, auto-decoding isn't part of the language That won't change anytime soon.
Re: DIP77 - Fix unsafe RC pass by 'ref'
On 04/09/2015 01:10 AM, Walter Bright wrote: > http://wiki.dlang.org/DIP77 It's a very interesting proposal to tackle this specific problem. As with all the scope/lifetime related stuff, I find it extremely difficult to anticipate all the needs and foresee how this will integrate with the rest of the language, or even the other scope related DIPs. At this point it might be better to make a dmd branch and throw up a prototype for all the scope/ref-counted work. The litmus test for a successful design would be safe, efficient, @nogc containers.
Re: DIP77 - Fix unsafe RC pass by 'ref'
On Friday, 10 April 2015 at 21:26:14 UTC, Walter Bright wrote: This would be a bad design of an RCO. RCO's must be constructed to not allow pointers to the payload other than by ref. And taking the address of that is already unsafe.
Re: Associative Arrays in the data segment
On 04/10/2015 09:54 AM, Daniel Murphy wrote: > > Is anybody else interested in seeing this in the next release? Give me at least a few days. I have a new idea how to make that library AA happen. Might be able to implement that over the weekend.
Re: "make std/concurrency.test" fails but others don't... why?
On 04/11/2015 01:04 AM, Andrei Alexandrescu wrote: > What makes MessageBox special? > > > Andrei For some reason the linker drags in another copy of the concurrency module (part of it to be precise concurrency_329_3ee.o). That collides with concurrency.o the to be tested module. I don't have time to debug this on OSX, but looking at the verbose linker output or at the symbols in both object files might give you an idea, why it was pulled in. ar x generated/osx/release/64/libphobos2.a concurrency_329_3ee.o nm concurrency_329_3ee.o ../dmd/src/dmd -conf= -I../druntime/import -w -dip25 -m64 -O -release -main -unittest -c std/concurrency.d nm concurrency.o | grep ' U ' It's probably because of some undefined ModuleInfoZ or InitZ symbol in the concurrency.o object.
Re: DIP77 - Fix unsafe RC pass by 'ref'
On 04/10/2015 11:29 PM, Walter Bright wrote: > > The latter. Can you update that part in the DIP, it wasn't clear that the temporary selectively pins RCO fields of a normal struct passed by ref.
Reminder, use stable branches for important bug fixes
A small reminder, as the next point release isn't that far away, we're now using stable branches to incorporate important bug fixes. This means I won't look through master trying to identify relevant commits. If there is anything that needs to go into 2.067.1 but was accidentally merged into master, please make a pull request for stable that cherry-picks this change. -Martin
Re: DIP77 - Fix unsafe RC pass by 'ref'
On 04/10/2015 12:29 PM, Walter Bright wrote: >> >> So someone passes an RCO via ref to avoid the inc/dec, and because >> that imposes >> safety issues we turn it into some sort of pass by value under the hood, >> defeating the purpose, and provide an opt-out via @system opAssign. > > Or you could pass it by const ref (which is what Rust essentially does). Maybe I'm missing something, but this proposal seems to make `ref RCO` fairly useless, because it creates a copy anyhow. >> Wouldn't it more straightforward to make pass-by-ref unsafe (@system) >> for RCOs? > > That's what we have now. It's not good enough. Assigning a RefCounted is marked @system, pass-by-ref is @safe. What's missing, you want to be able to use RefCounted in @safe code? Why not pass it by value then? That would pin the object and you could elide the additional inc/dec just like you propose to elide the temporary copies. It's more efficient to pass a smart pointer like RefCounted by-value anyhow.
Re: DIP77 - Fix unsafe RC pass by 'ref'
On 04/09/2015 01:10 AM, Walter Bright wrote:
> http://wiki.dlang.org/DIP77
In the first problem example:
struct S {
RCArray!T array;
}
void main() {
auto s = S(RCArray!T([T()])); // s.array's refcount is now 1
foo(s, s.array[0]); // pass by ref
}
void foo(ref S s, ref T t) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
What do you do to pin s.array?
auto tmp = s;
or
auto tmp = s.array;
Re: if(arr) now a warning
On 04/10/2015 07:28 PM, Steven Schveighoffer wrote: >> Also empty should work for AAs. > > How should "abc".front work? Do you want to move unicode decoding of > char and wchar arrays into object.d? We already have unicode decoding in druntime, the old and slow version though. Wouldn't cost much to move those 2 functions. Also we now have core.internal for more contrived stuff, works like this. https://github.com/D-Programming-Language/druntime/blob/master/src/object.di#L697
Re: Associative Arrays in the data segment
On Friday, 10 April 2015 at 07:54:44 UTC, Daniel Murphy wrote: It only works with integral types of at most 32-bits at the moment, but most built-in types are fairly easy to support. The downside is requires re-implementing druntime's AA and hashing algorithms in the compiler, and keeping them in sync when either changes. It might make sense as an intermediate step. After all it's fairly simple and who knows how long we'll take to change the AA implementation. But what I find unacceptable is a lousy intermediate solution that supports int, but not short and only if the value isn't a const string or a class starting with the letter K. I don't want to add another facet to the already long AA story. So if you can fix the problem and are willing to do the work, go on. Just adding a tiny hack makes the situation worse IMO.
Re: Associative Arrays in the data segment
On Friday, 10 April 2015 at 13:44:27 UTC, Daniel Murphy wrote: Who wouldn't? But realistically, how many more years until that happens? I'll better change it right now, have been cooking up some stuff.
Re: Wanted: Review manager for std.data.json
On 04/09/2015 10:59 AM, Sönke Ludwig wrote: > As far as the profiler results can be trusted, a good chunk of the time > gets spent for reading individual bytes from memory, but there must be > something else low-level going on that make things this bad. However, > there is nothing fundamental in the structure/design that would cause > this, so I think spending more time with the profiler is the only > logical step now. Unfortunately my VTune license has expired and perf on > Linux makes the task quite a bit more involved. I didn't found too many issues. Most of the time is spent in parseJSONValue (looks like there are some expansive struct copies) https://github.com/s-ludwig/std_data_json/blob/1da3f828ae6c4fd7cac7f7e13ae9e51ec93e6f02/source/stdx/data/json/parser.d#L148 and skipWhitespace. This function could really take some optimization, e.g. avoid UTF decoding. https://github.com/s-ludwig/std_data_json/blob/1da3f828ae6c4fd7cac7f7e13ae9e51ec93e6f02/source/stdx/data/json/lexer.d#L345 Libdparse has some optimized ASM function, might be useful. https://github.com/Hackerpilot/libdparse/blob/51b7d9d321aac0fcc4a9be99bbbed5db3158326c/src/std/d/lexer.d#L2233
Re: Wanted: Review manager for std.data.json
On 04/08/2015 03:56 PM, Sönke Ludwig wrote: >> > > The problem is that even the pull parser alone is relatively slow. Also, > for some reason the linker reports unresolved symbols as soon as I build > without the -debug flag... The review hasn't yet started and I'm already against the "stream" parser, because it hardly deserves the names parser, it's more like a lexer. Because the benchmark code by tcha was a very specific hack for the used data structure, I tried to write a proper stream parser to have a fair comparison. This is where I stopped (it doesn't work). http://dpaste.dzfl.pl/8282d70a1254 The biggest problem with that interface is, that you have to count matching start/end markers for objects and arrays in order to skip an entry, not much fun and definitely needs a dedicated skip value function. There are 2 very nice alternative approaches in the benchmark repo. https://github.com/kostya/benchmarks/blob/master/json/test_pull.cr https://github.com/kostya/benchmarks/blob/master/json/test_schema.cr
Re: Wanted: Review manager for std.data.json
On 04/09/2015 02:10 PM, John Colvin wrote: > > Still getting trounced across the board by rapidjson. Yep, anyone knows why? They don't even use a lazy parser.
Re: Wanted: Review manager for std.data.json
On 04/08/2015 08:32 PM, tcha wrote: Now with release numbers. > D new - debug - 14.98s, 1782.0Mb 8.53s, 1786.8Mb > D new Gdc - debug - 29.08s, 1663.9Mb GDC still misses @nogc support. > D new Ldc - 16.99s, 1663.0Mb 18.76s, 1664.1Mb > D new lazy - debug - 11.50s, 213.2Mb 4.57s, 206Mb > D new lazy Gdc - 13.66s, 206.1Mb Can't compile stdx.data.json with gdc-4.9.0 which doesn't yet support @nogc. > D new lazy Ldc - 3.59s, 205.4Mb 4.0s, 205.4Mb LDC doesn't yet have the GC improvements, therefor is much slower for the DOM parsing benchmarks.
Re: Wanted: Review manager for std.data.json
On Wednesday, 8 April 2015 at 09:58:31 UTC, Sönke Ludwig wrote: Initial numbers that I just collected have not been as good as expected. I'll have to take a closer look at the compiler output. I made a note, will see if I time to help with that. Algebraic might be a problem as it's based on typeinfo not tags, just a wild guess though.
Re: Which D IDE do you use?(survey)
On Tuesday, 7 April 2015 at 22:58:44 UTC, weaselcat wrote: http://goo.gl/forms/MmsuInzDL0 Emacs
Re: Fun project - faster associative array algorithm
On Wednesday, 8 April 2015 at 09:12:00 UTC, Martin Nowak wrote: The biggest problems in writing an AA library implementation sorted by difficulty are: There is a clear acceptance list for a good AA here. https://github.com/D-Programming-Language/druntime/pull/934#issuecomment-65916801
Re: Fun project - faster associative array algorithm
On Tuesday, 7 April 2015 at 22:33:52 UTC, Steven Schveighoffer wrote: Until that point, all these discussions on AA updates are useless. I really don't that all or nothing attitude, it condemns an important step, just because something better might be possible. It's also very comfortable, because this way nobody will ever have to do anything. Improving the AA implementation has a big immediate effect, and will also help anyone writing a library implementation, because any candidate up to this date was still based on that crappy bucket list implementation. The biggest problems in writing an AA library implementation sorted by difficulty are: - deprecation of all magic AA behaviors (attributes, as[i][j]++) - get the lowering right - efficient construction and value insertion (rvalue moving)
Re: Fun project - faster associative array algorithm
On Tuesday, 7 April 2015 at 23:38:21 UTC, Andrei Alexandrescu wrote: Ah, I thought the array embeds KeyValue directly. Thx! -- Andrei It should if they are relatively small compared to the pointer, or at least store the key inline. As we recently discussed about the freeing bug in the AA, it should be fine to store values inline, as long as the bucket array is never deleted.
Re: Fun project - faster associative array algorithm
On Tuesday, 7 April 2015 at 19:07:01 UTC, w0rp wrote: On Tuesday, 7 April 2015 at 18:35:27 UTC, Martin Nowak wrote: One thing I was wondering about, which you might know more about, is that I had to set my load factor to be half the size of the array, as quadratic probing seems to fail when more than half the buckets are filled. Is that correct, or did I make a mistake? You made a mistake somewhere, the sweet spot should be in the range of 0.6-0.8. Quadratic probing with triangular numbers is guaranteed to fail only when your buckets are completely full.
Re: Wanted: Review manager for std.data.json
On Wednesday, 8 April 2015 at 07:14:32 UTC, Iain Buclaw wrote: With this lib it gets to [2]: D - 7.48s, 1794.0Mb Gdc and Ldc cannot build it with release (debug works) [3] and [4] Have you tried to use the pull/stream parser?
Re: Why I'm Excited about D
On Tuesday, 7 April 2015 at 22:22:35 UTC, Andrei Alexandrescu wrote: On 4/7/15 11:42 AM, Martin Nowak wrote: What's still missing is a faster AST interpreter for CTFE though. A JIT would be nice. -- Andrei Maxine understood the point, see http://dconf.org/2013/talks/chevalier_boisvert.pdf p. 61 ff, an AST interpreter is something you can write over a long weekend, a JIT is rather unlikely to happen any time soon. Even a bytecode interpreter might hardly be worth the effort.
Re: DDMD is now in the master branch
On Wednesday, 8 April 2015 at 06:26:04 UTC, Daniel Murphy wrote: If we're lucky this will happen before the 2.068 release. That would be great, though being able to compile ddmd with ldc and/or gdc is necessary IMO to make it releasable. What's missing to make that work?
Re: Why I'm Excited about D
On 04/07/2015 06:06 PM, Adam D. Ruppe wrote: > In my web projects. I used compile time stuff sometimes too, but the > runtime loading ultimately won out for the ease of editing by me and by > the frontend team - they can edit html too without needing to worry > about rerunning a compiler. That's why I spent quite some time reducing that cycle and worked on a dub watch command. https://github.com/D-Programming-Language/dub/pull/388 https://code.dawg.eu/reducing-vibed-turnaround-time-part-2-less-compiling.html https://github.com/D-Programming-Language/dub/pull/446 What's still missing is a faster AST interpreter for CTFE though.
Re: Fun project - faster associative array algorithm
On 04/07/2015 07:24 PM, Walter Bright wrote: > https://github.com/D-Programming-Language/dmd/blob/master/src/root/stringtable.c > > uses quadratic probing instead of linked lists, but this is not cache > friendly, either. Quite to the contrary, it's very cache friendly. The triangular numbers are 1 3 6 10 15 21 With 8 byte per entry as in stringtable you need 2 collisions on the same "bucket" to land 48 bytes away, which might still be the same cacheline. The problem with linear probing is primary clustering, especially when your hash isn't too good, which can lead to long streaks of neighbouring buckets that are filled. Any insert that hashes into such a lump (chances are increasing the bigger it becomes) will make it grow even further. For higher load factor such as 0.75 this can lead to a huge variance of the probe sequence length and extreme lengths on the upper 95/99 percentiles. Here is the ticket for open addressing, btw. https://issues.dlang.org/show_bug.cgi?id=14385
Re: json parsing performance
On 04/06/2015 11:09 PM, Brad Anderson wrote: > > We actually have a JSON parser meant to replace std.json that should be > very high performance. You can try it out now using dub: > > http://code.dlang.org/packages/std_data_json It also includes a stream parser for the highest performance requirements. http://s-ludwig.github.io/std_data_json/stdx/data/json/parser/parse_json_stream.html
Re: Why I'm Excited about D
On 04/07/2015 08:28 AM, Jacob Carlborg wrote: >> > > vibe.d has a template system. It's based on Jade, which seems to be > based on Haml. There is also a runtime template system, http://code.dlang.org/packages/mustache-d. See https://github.com/D-Programming-GDC/gdcproject for an example.
Re: Mid-term vision review
On 04/06/2015 03:39 AM, Dennis Ritchie wrote: > > Thanks. I hope that in the near future will be a powerful D macrosystem. Don't expect too much enthusiasm about adding macros. - D already has very good metaprogramming capabilities and most stuff macros can do, can already be achieved using string mixins - Macros in Scala turned out very complex (not sure about std.meta). - DSLs can fracture the community (http://winestockwebdesign.com/Essays/Lisp_Curse.html) Might be a approach to write a good, fast, and CTFEable parsing library for.
Re: I submitted my container library to code.dlang.org
On 04/04/2015 05:12 PM, Kagamin wrote: > > I think, leave the seed zero and only provide a function to change it: > extern(C) void _d_setHashSeed(int seed); Sounds good, accepting patches :). https://issues.dlang.org/show_bug.cgi?id=14414
Re: I submitted my container library to code.dlang.org
On 04/02/2015 02:10 PM, Kagamin wrote: > > The vulnerability presentation suggests perl solution (random hash seed) > is good enough, it doesn't slow down anything. The seed can be left zero > and initialized by an application as needed. One can also use a longer > key and add more its bits every, say, 10 bytes of hashed data, not sure > if it will make any difference. A global random hash seed would work, but it needs to be accessible for reproducing test cases (druntime DRT option or in core.runtime).
