Re: Why is sformat and formattedWrite (appender) allocating GC mem here?
On 03.09.19 16:03, James Blachly wrote: For my own learning, why was a unittest to ensure no GC added to sformat instead of a @nogc annotation? `sformat` uses the GC less now, but it's not @nogc. It can still throw GC-allocated exceptions, e.g. when the arguments don't match the format string.
Re: Why is sformat and formattedWrite (appender) allocating GC mem here?
On 8/31/19 5:12 PM, ag0aep6g wrote: I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163 Wonderful! For my own learning, why was a unittest to ensure no GC added to sformat instead of a @nogc annotation?
Re: Why is sformat and formattedWrite (appender) allocating GC mem here?
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote: I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163 Thanks for the responses, very cool seeing these updates happen so fluidly.
Re: Why is sformat and formattedWrite (appender) allocating GC mem here?
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote: I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163 Merged.
Re: Why is sformat and formattedWrite (appender) allocating GC mem here?
On 31.08.19 14:07, cc wrote: I'm guessing it's allocating a string first to write the contents of the array and then inserting that string into the buffer I supplied. Is there no way to have it skip this step and just write each element (plus the joining punctuation, etc) directly into the buffer? `formatElement` does something like that. It writes the string into a temporary buffer while looking for invalid Unicode. When it finds some, the temporary is discarded and the whole string is formatted differently. When the string is a-ok, the data is copied over to the actual destination. I'm not sure if that's the best approach, though. The temporary buffer and the string copy are costly. There is also a closure being allocated for no reason in `sformat` itself. The compiler isn't smart enough to see that it's not really needed. I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163
Re: Why is sformat and formattedWrite (appender) allocating GC mem here?
On Saturday, 31 August 2019 at 12:07:56 UTC, cc wrote: And what, if anything, can I do to avoid it? import core.stdc.stdio : printf; There are no @nogc versions of the Phobos write* and format functions.
Why is sformat and formattedWrite (appender) allocating GC mem here?
And what, if anything, can I do to avoid it? string[] arr = ["abcd", "efgh", "ijkl"]; char[4096] buf; char[] res; writeln(GC.stats); res = sformat(buf, "%s", arr); assert(res.ptr == buf.ptr); writeln(res); writeln(GC.stats); res = sformat(buf, "%s", arr); assert(res.ptr == buf.ptr); writeln(res); writeln(GC.stats); // Results: Stats(64, 1048512, 80) ["abcd", "efgh", "ijkl"] Stats(272, 1048304, 272) ["abcd", "efgh", "ijkl"] Stats(464, 1048112, 464) I get similar behavior trying to use formattedWrite with an appender: string[] arr = ["abcd", "efgh", "ijkl"]; auto formatBuffer = appender!(char[])(); formatBuffer.reserve(4096); formatBuffer.clear(); writeln(GC.stats); formatBuffer.formattedWrite!"%s"(arr); writeln(formatBuffer.data); writeln(GC.stats); formatBuffer.clear(); formatBuffer.formattedWrite!"%s"(arr); writeln(formatBuffer.data); writeln(GC.stats); // Results: Stats(12288, 5230592, 4208) ["abcd", "efgh", "ijkl"] Stats(12432, 5230448, 4352) ["abcd", "efgh", "ijkl"] Stats(12576, 5230304, 4496) Same thing if I use a format string like "%(%s,%)" rather than just "%s". I'm guessing it's allocating a string first to write the contents of the array and then inserting that string into the buffer I supplied. Is there no way to have it skip this step and just write each element (plus the joining punctuation, etc) directly into the buffer?
Re: When to prefer formatValue over formattedWrite?
On Tuesday, 19 February 2019 at 11:40:09 UTC, Per Nordlöw wrote: Is `formatValue` more lightweight to the compiler? I already posted a very similar question at https://forum.dlang.org/post/xsovipkjyjyumtyzm...@forum.dlang.org but in this thread I'm interested in knowing if is there is any significant difference in compilation-speed and if formatValue and/or formattedWrite needs to allocate using the GC when we are writing a array of `char`s or a long, directly to standard output. In theory the string could be written directly to stdout without an extra memory buffer and the `long` could be formatted to a temporarily (stack) allocated array of `char`s before written to standard output.
When to prefer formatValue over formattedWrite?
Is `formatValue` also preferred over `formattedWrite` when
writing a single value?
Code example:
import std.array : appender;
auto writer1 = appender!string();
writer1.formattedWrite("%08b", 42);
auto writer2 = appender!string();
auto f = singleSpec("%08b");
writer2.formatValue(42, f);
Is `formatValue` more lightweight to the compiler?
Re: @nogc formattedWrite
On Saturday, 7 October 2017 at 18:14:00 UTC, Nordlöw wrote:
Is it currently possible to somehow do @nogc formatted output
to string?
I'm currently using my `pure @nogc nothrow` array-container
`CopyableArray` as
@safe pure /*TODO nothrow @nogc*/ unittest
{
import std.format : formattedWrite;
const x = "42";
alias A = CopyableArray!(char);
A a;
a.formattedWrite!("x : %s")(x);
assert(a == "x : 42");
}
but I can't tag the unittest as `nothrow @nogc` because of the
call to `formattedWrite`. Is this because `formattedWrite`
internally uses the GC for buffer allocations or because it may
throw?
It would be nice to be able to formatted output in -betterC...
Phobos code is not @nogc (can be fixed), but it will not nothrow.
PRs with @nogc formatting functionality are welcome in Mir
Algorithm.
Cheers,
Ilya
Re: @nogc formattedWrite
On Saturday, 7 October 2017 at 18:27:36 UTC, jmh530 wrote: On Saturday, 7 October 2017 at 18:14:00 UTC, Nordlöw wrote: It would be nice to be able to formatted output in -betterC... Agreed. If you know the size of the buffer, you can use sformat, which might be @nogc, but I don't know if it's compatible with betterC. Also, you might check out Ocean, which might have nogc formatting. https://github.com/sociomantic-tsunami/ocean As far as I know sformat is still not @nogc because it may throw an exception if buffer is not large enough, and throwing exceptions requires allocation.
Re: @nogc formattedWrite
On Saturday, 7 October 2017 at 18:14:00 UTC, Nordlöw wrote: It would be nice to be able to formatted output in -betterC... Agreed. If you know the size of the buffer, you can use sformat, which might be @nogc, but I don't know if it's compatible with betterC. Also, you might check out Ocean, which might have nogc formatting. https://github.com/sociomantic-tsunami/ocean
@nogc formattedWrite
Is it currently possible to somehow do @nogc formatted output to
string?
I'm currently using my `pure @nogc nothrow` array-container
`CopyableArray` as
@safe pure /*TODO nothrow @nogc*/ unittest
{
import std.format : formattedWrite;
const x = "42";
alias A = CopyableArray!(char);
A a;
a.formattedWrite!("x : %s")(x);
assert(a == "x : 42");
}
but I can't tag the unittest as `nothrow @nogc` because of the
call to `formattedWrite`. Is this because `formattedWrite`
internally uses the GC for buffer allocations or because it may
throw?
It would be nice to be able to formatted output in -betterC...
Re: Should formattedWrite take the outputrange by ref?
On Wednesday, 3 September 2014 at 07:43:20 UTC, Guillaume Chatelet wrote: +1 I've been bitten by this also. Same here, +1
Re: Should formattedWrite take the outputrange by ref?
On Thursday, 4 September 2014 at 17:06:00 UTC, Márcio Martins wrote: On Wednesday, 3 September 2014 at 07:43:20 UTC, Guillaume Chatelet wrote: +1 I've been bitten by this also. Same here, +1 It's still on my radar, but it's actually not that trivial of a change, especially if we want to avoid breaking code, and binary bloat...
Re: Should formattedWrite take the outputrange by ref?
+1 I've been bitten by this also.
formattedWrite writes nothing
class MyClass {
Appender!string _stringBuilder;
this() {
_stringBuilder = Appender!string(null);
_stringBuilder.clear();
}
@property string str() {
return _stringBuilder.data;
}
void append(string s) {
formattedWrite(_stringBuilder, %s, s);
}
}
MyClass c = new MyClass();
c.append(text 1);
c.append(__222);
writeln(c.str); //in this case nothing is printed out
Following workarounds work:
1) call _stringBuilder.put() instead of formattedWrite()
2) if _stringBuilder.clear() is omitted in the constructor,
formattedWrite(...) will work as expected.
Is it a bug or is there a reason for such behaviour?
Re: formattedWrite writes nothing
On 5/2/14, ref2401 via Digitalmars-d-learn
digitalmars-d-learn@puremagic.com wrote:
class MyClass {
Appender!string _stringBuilder;
this() {
_stringBuilder = Appender!string(null);
_stringBuilder.clear();
Ouch, ouch, ouch! What's happening is that the 'clear' Appender method
is only compiled-in if the data is mutable, otherwise you end up
calling the object.clear UFCS function. So don't use clear here.
I don't know if this is a case of poor method naming or another
downside of UFCS. Luckily 'clear' is being renamed to 'destroy' in the
object module, so this specific case will not become a problem in the
future.
Re: formattedWrite writes nothing
On Friday, 2 May 2014 at 10:23:03 UTC, Andrej Mitrovic via Digitalmars-d-learn wrote: Ouch, ouch, ouch! What's happening is that the 'clear' Appender method is only compiled-in if the data is mutable, otherwise you end up calling the object.clear UFCS function. So don't use clear here. I don't know if this is a case of poor method naming or another downside of UFCS. Luckily 'clear' is being renamed to 'destroy' in the object module, so this specific case will not become a problem in the future. I'd say clear should be @disabled in Appender for non-mutable data.
Re: formattedWrite writes nothing
I logged this bug a while ago: https://issues.dlang.org/show_bug.cgi?id=10291 On Friday, 2 May 2014 at 10:06:43 UTC, ref2401 wrote: Is it a bug or is there a reason for such behaviour?
Should formattedWrite take the outputrange by ref?
I want to print a tree structure and need to keep track of the indention for different parts of the tree. My idea was to write a generic wrapper for an output range that outputs tabs when it encounters a newline. This wrapper has internal state and if I want to use formattedWrite with this wrapper than state changes don't propagate to the calling context because by value semantics. Short solution: I'm using a class now. But shouldn't formattedWrite take it's output range by ref? Does anyone else have a use case for this and might this cause any problems?
Re: Should formattedWrite take the outputrange by ref?
monarch_dodra: *I* think it should. File a report, and I'll see what I can do about it. The problem with these kinds of things though might be breaking existing code... Given the frequency of bugs caused by such functions that require a pointer to the data, I think that a breaking change is the smaller problem. Bye, bearophile
Re: Should formattedWrite take the outputrange by ref?
On Saturday, 18 January 2014 at 21:55:54 UTC, Tobias Pankrath wrote: I want to print a tree structure and need to keep track of the indention for different parts of the tree. My idea was to write a generic wrapper for an output range that outputs tabs when it encounters a newline. This wrapper has internal state and if I want to use formattedWrite with this wrapper than state changes don't propagate to the calling context because by value semantics. Short solution: I'm using a class now. But shouldn't formattedWrite take it's output range by ref? Does anyone else have a use case for this and might this cause any problems? *I* think it should. File a report, and I'll see what I can do about it. The problem with these kinds of things though might be breaking existing code...
Re: Should formattedWrite take the outputrange by ref?
On Saturday, 18 January 2014 at 21:55:54 UTC, Tobias Pankrath wrote: I want to print a tree structure and need to keep track of the indention for different parts of the tree. My idea was to write a generic wrapper for an output range that outputs tabs when it encounters a newline. This wrapper has internal state and if I want to use formattedWrite with this wrapper than state changes don't propagate to the calling context because by value semantics. Short solution: I'm using a class now. Alternatively, pass a pointer: formattedWrite(writer, ...)
Re: Should formattedWrite take the outputrange by ref?
On Sunday, 19 January 2014 at 15:03:13 UTC, monarch_dodra wrote: So, my conclusion, * might be a workable solution. But simply taking by ref would be cleaner, and make more sense as a whole. Or a special template constraint path for T*. foo(T)(T t) if (is(T = Q*)) manyOtherChecks!(Q) foo(T)(T t) if (!is(T = Q*)) manyOtherChecks!(T) Maybe that will break less code.
Re: Should formattedWrite take the outputrange by ref?
On Saturday, 18 January 2014 at 23:06:42 UTC, Tobias Pankrath wrote: I actually didn't think that a ptr (to output range) would work. This way we can have best of both worlds and I'm happy with it. A fun fact is that since . notation works with pointers, more often than not, if T verifies some trait isOutputRange, more often than not, so will T*. But this is more of a by-product than an actual rule, and, as a rule of thumb, may not be something you want to rely on in a general sense. Limitations include: Type checking: If R is a random access range, R* will only be an input range, because p.save will return an R, and not an R* :/ Also, it is limited to member functions, and not generic UFCS: For example, while in the general sense, R is input range = R* is input range, this will fail for T[]*, because slices have a non-member popFront, so p.popFront() will not actually match a function, and T[]* will fail the input range validation :/ So, my conclusion, * might be a workable solution. But simply taking by ref would be cleaner, and make more sense as a whole.
Re: Should formattedWrite take the outputrange by ref?
On Sunday, 19 January 2014 at 15:12:07 UTC, Tobias Pankrath wrote: On Sunday, 19 January 2014 at 15:03:13 UTC, monarch_dodra wrote: So, my conclusion, * might be a workable solution. But simply taking by ref would be cleaner, and make more sense as a whole. Or a special template constraint path for T*. foo(T)(T t) if (is(T = Q*)) manyOtherChecks!(Q) foo(T)(T t) if (!is(T = Q*)) manyOtherChecks!(T) Maybe that will break less code. But maybe we really should go for taking the output range by reference. See https://d.puremagic.com/issues/show_bug.cgi?id=10291 https://d.puremagic.com/issues/show_bug.cgi?id=9102 Created a new report for this: https://d.puremagic.com/issues/show_bug.cgi?id=11951
Re: Should formattedWrite take the outputrange by ref?
On Sunday, 19 January 2014 at 15:28:04 UTC, Tobias Pankrath wrote: On Sunday, 19 January 2014 at 15:12:07 UTC, Tobias Pankrath wrote: On Sunday, 19 January 2014 at 15:03:13 UTC, monarch_dodra wrote: So, my conclusion, * might be a workable solution. But simply taking by ref would be cleaner, and make more sense as a whole. Or a special template constraint path for T*. foo(T)(T t) if (is(T = Q*)) manyOtherChecks!(Q) foo(T)(T t) if (!is(T = Q*)) manyOtherChecks!(T) Maybe that will break less code. But maybe we really should go for taking the output range by reference. See https://d.puremagic.com/issues/show_bug.cgi?id=10291 https://d.puremagic.com/issues/show_bug.cgi?id=9102 Created a new report for this: https://d.puremagic.com/issues/show_bug.cgi?id=11951 Thanks! I'll look into these.
formattedWrite
I cannot find any string format() method in phobos. Having to do: auto a = appender!string(); formattedWrite(a, , ...); a.data seems unnecessary in many cases.
Re: formattedWrite
On Mon, 08 Aug 2011 14:37:26 +0300, simendsjo simend...@gmail.com wrote: I cannot find any string format() method in phobos. What's wrong with format() from std.string? -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: formattedWrite
On 08.08.2011 13:37, Vladimir Panteleev wrote: On Mon, 08 Aug 2011 14:37:26 +0300, simendsjo simend...@gmail.com wrote: I cannot find any string format() method in phobos. What's wrong with format() from std.string? Thanks, that was the function I was looking for. I just expected it to be in std.format (and I also looked in std.stdio).
Re: formattedWrite
On 08.08.2011 13:41, simendsjo wrote: On 08.08.2011 13:37, Vladimir Panteleev wrote: On Mon, 08 Aug 2011 14:37:26 +0300, simendsjo simend...@gmail.com wrote: I cannot find any string format() method in phobos. What's wrong with format() from std.string? Thanks, that was the function I was looking for. I just expected it to be in std.format (and I also looked in std.stdio). Seems std.string.format is missing positional arguments..
Re: formattedWrite
On 08.08.2011 14:07, simendsjo wrote: On 08.08.2011 13:41, simendsjo wrote: On 08.08.2011 13:37, Vladimir Panteleev wrote: On Mon, 08 Aug 2011 14:37:26 +0300, simendsjo simend...@gmail.com wrote: I cannot find any string format() method in phobos. What's wrong with format() from std.string? Thanks, that was the function I was looking for. I just expected it to be in std.format (and I also looked in std.stdio). Seems std.string.format is missing positional arguments.. http://d.puremagic.com/issues/show_bug.cgi?id=6455
