core.exception.InvalidMemoryOperationError
A code I'm working on stops working and starts printing an infinite loop of core.exception.InvalidMemoryOperationError to the command line output. The code is quite complex and the bug seems to present itself almost in random situation so I would like to try to understand the issue better before looking for the wrong line of code hiding somewhere. I've read it might be that something is trying to allocate during a destructor call, but it sounds really strange to me that there's a neverending amount of exceptions being thrown. This is the first exception being thrown (nothing is thrown before the infinite loop begins). Anyone has suggestions/ideas/heard of a similar stuff before?
Re: core.exception.InvalidMemoryOperationError
On Thursday, 10 July 2014 at 15:36:53 UTC, francesco cattoglio wrote: A code I'm working on stops working and starts printing an infinite loop of core.exception.InvalidMemoryOperationError to the command line output. The code is quite complex and the bug seems to present itself almost in random situation so I would like to try to understand the issue better before looking for the wrong line of code hiding somewhere. I've read it might be that something is trying to allocate during a destructor call, but it sounds really strange to me that there's a neverending amount of exceptions being thrown. This is the first exception being thrown (nothing is thrown before the infinite loop begins). Anyone has suggestions/ideas/heard of a similar stuff before? I had the same issue with Derelict bindings. Bindings symbols could be already unloaded when a destructor tries to use them.
Re: core.exception.InvalidMemoryOperationError
On Thursday, 10 July 2014 at 15:36:53 UTC, francesco cattoglio wrote: A code I'm working on stops working and starts printing an infinite loop of core.exception.InvalidMemoryOperationError to the command line output. The code is quite complex and the bug seems to present itself almost in random situation so I would like to try to understand the issue better before looking for the wrong line of code hiding somewhere. I've read it might be that something is trying to allocate during a destructor call, but it sounds really strange to me that there's a neverending amount of exceptions being thrown. This is the first exception being thrown (nothing is thrown before the infinite loop begins). Anyone has suggestions/ideas/heard of a similar stuff before? If you look at the source for the garbage collector, the only place that error is called is if the gc is trying to malloc or execute other memory operations while the collector is running. I ran across this myself because an assert was getting triggered in a destructor. Since an assert tries to malloc and the destructor is called by the GC, I got an InvalidMemoryOperationError which swallowed up the message from the original assert. By putting printfs in the code path in druntime, I was able to track it down to that destructor, otherwise I had no idea where the invalid memory error was getting triggered. You can probably do the same, but you can be sure it's a GC issue, and I would guess tied to allocating in a destructor (unless you happen to be calling InvalidMemoryOperationErrors somewhere in your own code or some library that you're using, which is unlikely).
Re: core.exception.InvalidMemoryOperationError
On Friday, 11 July 2014 at 11:43:44 UTC, Joakim wrote: On Thursday, 10 July 2014 at 15:36:53 UTC, francesco cattoglio wrote: A code I'm working on stops working and starts printing an infinite loop of core.exception.InvalidMemoryOperationError to the command line output. The code is quite complex and the bug seems to present itself almost in random situation so I would like to try to understand the issue better before looking for the wrong line of code hiding somewhere. I've read it might be that something is trying to allocate during a destructor call, but it sounds really strange to me that there's a neverending amount of exceptions being thrown. This is the first exception being thrown (nothing is thrown before the infinite loop begins). Anyone has suggestions/ideas/heard of a similar stuff before? If you look at the source for the garbage collector, the only place that error is called is if the gc is trying to malloc or execute other memory operations while the collector is running. I ran across this myself because an assert was getting triggered in a destructor. Since an assert tries to malloc and the destructor is called by the GC, I got an InvalidMemoryOperationError which swallowed up the message from the original assert. By putting printfs in the code path in druntime, I was able to track it down to that destructor, otherwise I had no idea where the invalid memory error was getting triggered. You can probably do the same, but you can be sure it's a GC issue, and I would guess tied to allocating in a destructor (unless you happen to be calling InvalidMemoryOperationErrors somewhere in your own code or some library that you're using, which is unlikely). It's unfortunate that you wrote this only 4 hours ago, because I already spent the morning doing more-or-less the same thing, and finaly realized what was happening and WHO was allocating during a destructor. :o) It's even somewhat told in the docs of core.exception module. What I really don't understand is how the hell was it possible that something managed to either recurse or loop to generate an infinite WOE (Wall Of Exceptions).
core.exception.InvalidMemoryOperationError@(0)
This problem is a tough one. I've been getting this error when I run my unittests, and apparently it is caused by attempting an allocation in the destructor from what little I could find online about the subject. The error is triggered once all my tests are complete, so I'm assuming the garbage collector is running before termination, but I tried placing logging messages in all my destructors and I didn't get anything, so none of them are being called. Is there any other possible reason to get this error?
Tracing down core.exception.InvalidMemoryOperationError
Hi, at the end of my program it throws InvalidMemoryOperationError. Looking at the documentation and past forum questions I learned that it is probably because of allocations in destructors. However, I have no such thing in my code (at least not intentionally). I am suspecting the std.logger package, because it throwed on me a memory error on occasions. But regardless of the source, I would like to trace it and deal with it. But I do not have much of an idea where to start. Could you give me an advice? Thanks, Drasha
Re: core.exception.InvalidMemoryOperationError@(0)
On 01/24/2015 04:16 AM, Bayan Rafeh wrote: This problem is a tough one. I've been getting this error when I run my unittests, and apparently it is caused by attempting an allocation in the destructor from what little I could find online about the subject. The error is triggered once all my tests are complete, so I'm assuming the garbage collector is running before termination, but I tried placing logging messages in all my destructors and I didn't get anything, so none of them are being called. Is there any other possible reason to get this error? The allocations may be indirect. For example, formatting a string to log a message may allocate. Try marking your destructors as @nogc to see whether the compiler agrees. I am not sure whether it would cause the same error but another reason for the error is touching reference members of a class, which may have been finalized before the object. Ali
Re: core.exception.InvalidMemoryOperationError@(0)
On Saturday, 24 January 2015 at 22:42:51 UTC, Ali Çehreli wrote: The allocations may be indirect. For example, formatting a string to log a message may allocate. Try marking your destructors as @nogc to see whether the compiler agrees. I am not sure whether it would cause the same error but another reason for the error is touching reference members of a class, which may have been finalized before the object. Ali Strange, I have no explicit destructors declared in my code (classes/structs). I'm making rich usage of std.container.Array, thought...
Re: core.exception.InvalidMemoryOperationError@(0)
On Saturday, 24 January 2015 at 12:16:38 UTC, Bayan Rafeh wrote: This problem is a tough one. I've been getting this error when I run my unittests, and apparently it is caused by attempting an allocation in the destructor from what little I could find online about the subject. The error is triggered once all my tests are complete, so I'm assuming the garbage collector is running before termination, but I tried placing logging messages in all my destructors and I didn't get anything, so none of them are being called. Is there any other possible reason to get this error? Hi, I created a wiki page which I hope will help you solve this problem: http://wiki.dlang.org/InvalidMemoryOperationError Hope this helps.
Re: core.exception.InvalidMemoryOperationError@(0)
On Sunday, 25 January 2015 at 00:43:43 UTC, Vladimir Panteleev
wrote:
On Saturday, 24 January 2015 at 12:16:38 UTC, Bayan Rafeh wrote:
This problem is a tough one. I've been getting this error when
I run my unittests, and apparently it is caused by attempting
an allocation in the destructor from what little I could find
online about the subject.
The error is triggered once all my tests are complete, so I'm
assuming the garbage collector is running before termination,
but I tried placing logging messages in all my destructors and
I didn't get anything, so none of them are being called.
Is there any other possible reason to get this error?
Hi,
I created a wiki page which I hope will help you solve this
problem:
http://wiki.dlang.org/InvalidMemoryOperationError
Hope this helps.
Thank you very much, this will certainly be useful in debugging
it.
The allocations may be indirect. For example, formatting a
string to log a message may allocate. Try marking your
destructors as @nogc to see whether the compiler agrees.
I tried what you said and I think I see the problem. I managed to
create an example program that duplicates the problem:
import std.stdio;
class A {
string path;
this(string p) {
path = p;
}
~this() {
}
void a(){}
void b(){}
}
class B {
A a;
this() {
this.a = new A("laladiv");
}
~this() {
delete a;
}
}
void main() {
B a = new B();
B b = new B();
delete b;
}
Re: core.exception.InvalidMemoryOperationError@(0)
On Sunday, 25 January 2015 at 08:39:42 UTC, Bayan Rafeh wrote:
On Sunday, 25 January 2015 at 00:43:43 UTC, Vladimir Panteleev
wrote:
On Saturday, 24 January 2015 at 12:16:38 UTC, Bayan Rafeh
wrote:
This problem is a tough one. I've been getting this error
when I run my unittests, and apparently it is caused by
attempting an allocation in the destructor from what little I
could find online about the subject.
The error is triggered once all my tests are complete, so I'm
assuming the garbage collector is running before termination,
but I tried placing logging messages in all my destructors
and I didn't get anything, so none of them are being called.
Is there any other possible reason to get this error?
Hi,
I created a wiki page which I hope will help you solve this
problem:
http://wiki.dlang.org/InvalidMemoryOperationError
Hope this helps.
Thank you very much, this will certainly be useful in debugging
it.
The allocations may be indirect. For example, formatting a
string to log a message may allocate. Try marking your
destructors as @nogc to see whether the compiler agrees.
I tried what you said and I think I see the problem. I managed
to create an example program that duplicates the problem:
import std.stdio;
class A {
string path;
this(string p) {
path = p;
}
~this() {
}
void a(){}
void b(){}
}
class B {
A a;
this() {
this.a = new A("laladiv");
}
~this() {
delete a;
}
}
void main() {
B a = new B();
B b = new B();
delete b;
}
The solution was just to remove the "delete a" from the
destructor if someone comes across this later. Could someone tell
me why though?
Re: core.exception.InvalidMemoryOperationError@(0)
On Sunday, 25 January 2015 at 08:41:25 UTC, Bayan Rafeh wrote: The solution was just to remove the "delete a" from the destructor if someone comes across this later. Could someone tell me why though? The "non-reentrant" bit applies to all GC operations, really - not just allocations. Explicit deletions are forbidden as well. Use of the "delete" keyword is discouraged, and in that context, it is also not used in a safe way. By the time B's destructor is called, the A instance might have been already collected by the garbage collector, causing a double free bug (one of the reasons to not use delete).
Re: core.exception.InvalidMemoryOperationError@(0)
On Sun, 25 Jan 2015 08:41:24 +, Bayan Rafeh wrote:
>> I tried what you said and I think I see the problem. I managed to
>> create an example program that duplicates the problem:
>>
>>
>> import std.stdio;
>>
>> class A {
>> string path;
>>
>> this(string p) {
>> path = p;
>> }
>> ~this() {
>> }
>> void a(){}
>>
>> void b(){}
>> }
>>
>> class B {
>> A a;
>> this() {
>> this.a = new A("laladiv");
>> }
>> ~this() {
>> delete a;
>> }
>> }
>>
>> void main() {
>> B a = new B(); B b = new B(); delete b;
>> }
>
> The solution was just to remove the "delete a" from the destructor if
> someone comes across this later. Could someone tell me why though?
there is no guarantees on destruction order. and GC will not nullify any
references to collected data. so, `a` can be already collected and
finalized when `B.~this()` is called. yet reference is still there, so
`delete a;` will try to delete already dead object. this will lead to
crash.
without precise GC collector is not able to automatically nullify all
"dead" references. and even if there will be such possibility, it can
slow down collections alot (GC will nullifying alot of references that
aren't used anyway), so i don't think that it do nullifying.
there is a simple rule: "dtor should not touch GC-managed resources".
this will not give you "predictable destruction order" (that's why most
people try to manually delete something in dtor), and this simply will
not work at all.
if you want predictable destruction order, don't use GC at all, use
manual memory management. it doesn't matter which hack you will invent to
force destruction order, any hack will either be very fragile, or will
not work. this is due to nature of GC-manged memory.
so: don't use GC-managed resources in dtors. don't use in any way -- this
including accessing 'em. i.e. reading `a.path` in dtor is invalid too. it
will not necessarily crash, but it's the source of "use after free" error.
and don't even think that you can trick GC using checks from
`core.memory`! this will not work too. sure, you can check if memory used
by `a` is still alive, but that memory can be used by completely
different object!
tl;dr:
1. don't use GC-managed objects in dtors. not even try to access 'em.
2. don't try to trick GC. either don't use it, or cooperate with it.
signature.asc
Description: PGP signature
Re: core.exception.InvalidMemoryOperationError@(0)
On Sunday, 25 January 2015 at 19:15:54 UTC, ketmar wrote:
On Sun, 25 Jan 2015 08:41:24 +, Bayan Rafeh wrote:
I tried what you said and I think I see the problem. I
managed to
create an example program that duplicates the problem:
import std.stdio;
class A {
string path;
this(string p) {
path = p;
}
~this() {
}
void a(){}
void b(){}
}
class B {
A a;
this() {
this.a = new A("laladiv");
}
~this() {
delete a;
}
}
void main() {
B a = new B(); B b = new B(); delete b;
}
The solution was just to remove the "delete a" from the
destructor if
someone comes across this later. Could someone tell me why
though?
there is no guarantees on destruction order. and GC will not
nullify any
references to collected data. so, `a` can be already collected
and
finalized when `B.~this()` is called. yet reference is still
there, so
`delete a;` will try to delete already dead object. this will
lead to
crash.
without precise GC collector is not able to automatically
nullify all
"dead" references. and even if there will be such possibility,
it can
slow down collections alot (GC will nullifying alot of
references that
aren't used anyway), so i don't think that it do nullifying.
there is a simple rule: "dtor should not touch GC-managed
resources".
this will not give you "predictable destruction order" (that's
why most
people try to manually delete something in dtor), and this
simply will
not work at all.
if you want predictable destruction order, don't use GC at all,
use
manual memory management. it doesn't matter which hack you will
invent to
force destruction order, any hack will either be very fragile,
or will
not work. this is due to nature of GC-manged memory.
so: don't use GC-managed resources in dtors. don't use in any
way -- this
including accessing 'em. i.e. reading `a.path` in dtor is
invalid too. it
will not necessarily crash, but it's the source of "use after
free" error.
and don't even think that you can trick GC using checks from
`core.memory`! this will not work too. sure, you can check if
memory used
by `a` is still alive, but that memory can be used by completely
different object!
tl;dr:
1. don't use GC-managed objects in dtors. not even try to
access 'em.
2. don't try to trick GC. either don't use it, or cooperate
with it.
All right, I removed all my destructors(turns out I don't really
need them), but I'm still running into this very same error.
This is another problematic example program:
import std.stdio;
void main(){
auto a = new A("/tmp/invalid");
}
class A {
File f;
string path;
this(string path) {
this.path = path;
//f = File(path, "r");
}
invariant() {
File test = File(path, "r");
}
}
is invariant() called during the destruction phase?
Re: core.exception.InvalidMemoryOperationError@(0)
On 01/25/2015 02:06 PM, Bayan Rafeh wrote:
is invariant() called during the destruction phase?
Something is fishy.
import std.stdio;
void main(){
writeln("entered main");
auto a = new A();
writeln("leaving main");
}
class A {
File file;
this() {
writeln("this");
}
~this() {
writeln("~this");
}
invariant() {
writeln("invariant");
}
}
The program produces the following output:
entered main
this
invariant
leaving main
invariant
invariant<-- ?
~this
invariant<-- ?
Removing the File member changes the output. Now two of the invariant
calls are missing:
entered main
this
invariant
leaving main
invariant
~this
However, the last invariant is still there and I think by design: The
destructor should see a valid state.
Ali
Re: core.exception.InvalidMemoryOperationError@(0)
On Sunday, 25 January 2015 at 22:46:56 UTC, Ali Çehreli wrote:
On 01/25/2015 02:06 PM, Bayan Rafeh wrote:
is invariant() called during the destruction phase?
Something is fishy.
import std.stdio;
void main(){
writeln("entered main");
auto a = new A();
writeln("leaving main");
}
class A {
File file;
this() {
writeln("this");
}
~this() {
writeln("~this");
}
invariant() {
writeln("invariant");
}
}
The program produces the following output:
entered main
this
invariant
leaving main
invariant
invariant<-- ?
~this
invariant<-- ?
Removing the File member changes the output. Now two of the
invariant calls are missing:
entered main
this
invariant
leaving main
invariant
~this
However, the last invariant is still there and I think by
design: The destructor should see a valid state.
Ali
Could this be a bug in D?
Re: core.exception.InvalidMemoryOperationError@(0)
On Sun, 25 Jan 2015 22:06:26 +, Bayan Rafeh wrote:
> This is another problematic example program:
> import std.stdio;
>
> void main(){
> auto a = new A("/tmp/invalid");
> }
>
> class A {
> File f;
> string path;
>
> this(string path) {
> this.path = path;
> //f = File(path, "r");
> }
>
> invariant() {
> File test = File(path, "r");
> }
> }
>
> is invariant() called during the destruction phase?
the thing is that your invariant is not a correct invariant at all.
invariants are meant to check *internal* object consistency, not external
conditions. compiler is free to call invariant block at any time after
object is properly initialised (i.e. after ctor is complete) and is not
executing member method. so it's perfectly legal to call invariant before
dtor, as you should not check anything that is not belonging to the
object itself in in.
in other words: you can't check any contents of any reference-typed
variables in your invariant block. `string` is reference-typed (it's a
dynamic array managed by GC in your case), so you can't check it contents
in invariant. you CAN, however, use `f` methods in your invariant, as `f`
is a struct which lives *inside* your object, and not a reference var.
but note that it's a bad practice anyway, as some struct can use some GC-
managed objects which can't be safely used in invariant block.
signature.asc
Description: PGP signature
Re: core.exception.InvalidMemoryOperationError@(0)
On Sun, 25 Jan 2015 22:06:26 +, Bayan Rafeh wrote: p.s. yet creating new `File` in invariant is wrong nevertheless, as it changes the program state. invariant checks SHOULD NEVER CHANGE THE PROGRAM STATE. signature.asc Description: PGP signature
Re: core.exception.InvalidMemoryOperationError@(0)
the thing is that your invariant is not a correct invariant at all. invariants are meant to check *internal* object consistency, not external conditions. compiler is free to call invariant block at any time after object is properly initialised (i.e. after ctor is complete) and is not executing member method. so it's perfectly legal to call invariant before dtor, as you should not check anything that is not belonging to the object itself in in. in other words: you can't check any contents of any reference-typed variables in your invariant block. `string` is reference-typed (it's a dynamic array managed by GC in your case), so you can't check it contents in invariant. you CAN, however, use `f` methods in your invariant, as `f` is a struct which lives *inside* your object, and not a reference var. but note that it's a bad practice anyway, as some struct can use some GC- managed objects which can't be safely used in invariant block. There are 2 problems here: 1. By definition, after the destructor is called the object state is destroyed. It makes no sense to check the invariant after the destructor is called because there is no state for us to check. 2. Invariants theoretically describe the legal states the object can be in which includes variables that are GC managed, therefore I should be able to check the invariant on GC managed objects as well. What's the point of having this feature if I can't even check the invariants of a simple data structure like a set, let alone classes involving files?
Re: core.exception.InvalidMemoryOperationError@(0)
On Mon, 26 Jan 2015 05:25:48 +, Bayan Rafeh wrote: > There are 2 problems here: > > 1. By definition, after the destructor is called the object state is > destroyed. It makes no sense to check the invariant after the destructor > is called because there is no state for us to check. i agree that calling invariant after destruction is invalid. i believe that this is a bug. > 2. Invariants theoretically describe the legal states the object can be > in which includes variables that are GC managed, therefore I should be > able to check the invariant on GC managed objects as well. What's the > point of having this feature if I can't even check the invariants of a > simple data structure like a set, let alone classes involving files? checking something that is not owned by an object is invalid. invariants MUST NOT be dependent on external data or system state. invariants that tries to check for GC objects ARE dependent of external state. let alone invariants that checking for files, which can be removed while the object is still alive. object can own another object, but it can't own another object *state*. if you want to check something that involves checking something like that, you should use `assert()`s in function body and/or in/out function parts. you are trying to use invariants for the things that invariants can't (and must not) check. invariants are meant for checking *internal* *object* *consistency*. NOT correctness. NOT applicability. ONLY consistency. object can be in "inapplicable" state, but still consistent. signature.asc Description: PGP signature
Re: core.exception.InvalidMemoryOperationError@(0)
you are trying to use invariants for the things that invariants can't (and must not) check. invariants are meant for checking *internal* *object* *consistency*. NOT correctness. NOT applicability. ONLY consistency. object can be in "inapplicable" state, but still consistent. Then I must have misunderstood the documentation, I apologize, I thought the point of invariant was to ensure correctness of the object's state. I'll go ahead and report the bug.
Re: core.exception.InvalidMemoryOperationError@(0)
Bug report at https://issues.dlang.org/show_bug.cgi?id=14051
Re: core.exception.InvalidMemoryOperationError@(0)
On Monday, 26 January 2015 at 15:54:12 UTC, Bayan Rafeh wrote: I apologize, I thought the point of invariant was to ensure correctness of the object's state It is simply matter of "internal correctness" vs "in-program correctness". Sorry, documentation should probably mention that explicitly. I have created a documentation PR : https://github.com/D-Programming-Language/dlang.org/pull/851
Re: core.exception.InvalidMemoryOperationError@(0)
On Mon, 26 Jan 2015 11:37:12 +, Bayan Rafeh wrote: >> you are trying to use invariants for the things that invariants can't >> (and must not) check. invariants are meant for checking *internal* >> *object* *consistency*. NOT correctness. NOT applicability. ONLY >> consistency. object can be in "inapplicable" state, but still >> consistent. > > Then I must have misunderstood the documentation, I apologize, I thought > the point of invariant was to ensure correctness of the object's state. > I'll go ahead and report the bug. but please, take my words with a grain of salt. i'm in no way a representative of D devs. that is how *i* understand invariants. it seems to be consistent with the cases where invariant works, but the designers may have different opinion, and don't have time to properly fix invariant handling. signature.asc Description: PGP signature
Re: core.exception.InvalidMemoryOperationError@(0)
but please, take my words with a grain of salt. i'm in no way a representative of D devs. that is how *i* understand invariants. it seems to be consistent with the cases where invariant works, This is the first serious project I do with D, so I'm kind of discovering the language as I write my code. I found the contracts page and got overly excited about it, so you're probably right. but the designers may have different opinion, and don't have time to properly fix invariant handling. That's fair. Useful as it may be for debugging, it's something I can live without.
Re: core.exception.InvalidMemoryOperationError@(0)
On Tue, 27 Jan 2015 06:46:20 +, Bayan Rafeh wrote: > This is the first serious project I do with D and now you're lost to other C-like languages, methinks. ;-) signature.asc Description: PGP signature
Re: Tracing down core.exception.InvalidMemoryOperationError
On Monday, 28 July 2014 at 09:38:35 UTC, Martin Drasar via Digitalmars-d-learn wrote: Hi, at the end of my program it throws InvalidMemoryOperationError. Looking at the documentation and past forum questions I learned that it is probably because of allocations in destructors. However, I have no such thing in my code (at least not intentionally). I am suspecting the std.logger package, because it throwed on me a memory error on occasions. But regardless of the source, I would like to trace it and deal with it. But I do not have much of an idea where to start. Could you give me an advice? More broadly speaking, it is thrown whenever certain memory operations are attempted while the GC is running, 6 in all, as you can see here: https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L458 I believe I stuck in printfs till I found out which one was run before the error was thrown, and then traced that back with more printfs to where it was getting called. I didn't have a debugger available, you may be able to trace faster with one.
Re: Tracing down core.exception.InvalidMemoryOperationError
On 28.7.2014 14:09, Joakim via Digitalmars-d-learn wrote: > More broadly speaking, it is thrown whenever certain memory operations > are attempted while the GC is running, 6 in all, as you can see here: > > https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L458 > > > I believe I stuck in printfs till I found out which one was run before > the error was thrown, and then traced that back with more printfs to > where it was getting called. I didn't have a debugger available, you > may be able to trace faster with one. Hi, thanks for the tip. I have a debugger at hand and I am would prefer to use it. However, I don't really know where and how to start. I would like to break at core/exception.d when onInvalidMemoryOperationError is called, but I am not sure how to build druntime with debug information. There does not seem to be some flag in the makefile like for dmd. Is there some document describing how to do this? Thanks, Drasha
Re: Tracing down core.exception.InvalidMemoryOperationError
On Monday, 28 July 2014 at 13:31:08 UTC, Martin Drasar via Digitalmars-d-learn wrote: On 28.7.2014 14:09, Joakim via Digitalmars-d-learn wrote: More broadly speaking, it is thrown whenever certain memory operations are attempted while the GC is running, 6 in all, as you can see here: https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L458 I believe I stuck in printfs till I found out which one was run before the error was thrown, and then traced that back with more printfs to where it was getting called. I didn't have a debugger available, you may be able to trace faster with one. Hi, thanks for the tip. I have a debugger at hand and I am would prefer to use it. However, I don't really know where and how to start. I would like to break at core/exception.d when onInvalidMemoryOperationError is called, but I am not sure how to build druntime with debug information. There does not seem to be some flag in the makefile like for dmd. Is there some document describing how to do this? It's not in the makefile; I simply added the -g or -gc flag where it compiled and then the debug symbols showed up in the debugger. You may also want to experiment with the -debug flag, which will turn on various kinds of log output depending on which module and flag you use it with.
Re: Tracing down core.exception.InvalidMemoryOperationError
On Monday, 28 July 2014 at 22:13:56 UTC, Joakim wrote: On Monday, 28 July 2014 at 13:31:08 UTC, Martin Drasar via Digitalmars-d-learn wrote: On 28.7.2014 14:09, Joakim via Digitalmars-d-learn wrote: More broadly speaking, it is thrown whenever certain memory operations are attempted while the GC is running, 6 in all, as you can see here: https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L458 I believe I stuck in printfs till I found out which one was run before the error was thrown, and then traced that back with more printfs to where it was getting called. I didn't have a debugger available, you may be able to trace faster with one. Hi, thanks for the tip. I have a debugger at hand and I am would prefer to use it. However, I don't really know where and how to start. I would like to break at core/exception.d when onInvalidMemoryOperationError is called, but I am not sure how to build druntime with debug information. There does not seem to be some flag in the makefile like for dmd. Is there some document describing how to do this? It's not in the makefile; I simply added the -g or -gc flag where it compiled and then the debug symbols showed up in the debugger. You may also want to experiment with the -debug flag, which will turn on various kinds of log output depending on which module and flag you use it with. I can tell you it is the logger, for sure. I have had similar problems in the past because I was trying to print a string in a destructor, and even just using the string concatenation is enough for an allocation to happen and for the exception to ruin everything. As a bonus, the exception is thrown from another thread :P In fact, now that we have @nogc I really think we could use *at least a warning* if the compiler determines that allocation might happen inside a destructor... (btw: a debug strategy might be: fire up dmd beta 2.066, add @nogc at all destructors and see where the compiler complains)
core.exception.InvalidMemoryOperationError@(0) on File Reading.
I am getting an core.exception.InvalidMemoryOperationError@(0)
auto recs = f // Open for reading
.byLineCopy();
.array; //Here is where is appears to
be happening.
I have narrowed it down to the .array. I am wondering if there
is some common issue I may be experiencing or if there is a
better solution?
Here is what I am overall trying to achieve.
auto recs = f // Open for reading
.byLineCopy;
.array();
.map!(a=>
assocArray(zip(h1.fullHeader, splitter(a,','
.map!(a=>a.byKeyValue()
.filter!(a=>h1.filteredHeader.canFind(a.key)));
foreach(item;recs){
auto item = Json.emptyObject;
each!((k)=> item[k.key] =
k.value)(item);
insert(item, dateIndex);
//ElasticSearch
}
Also any suggestions on the code are greatly appreciated :)
Re: core.exception.InvalidMemoryOperationError@(0) on File Reading.
Where is your code found? InvalidMemoryOperationError usually
props up in code running from a destructor.
If the call to .array isn't in a destructor, it might also be
that the array is triggering a GC cycle... which calls something
in a destructor.
Since that's so commonly the cause I encourage you to audit your
classes and make sure none of them try to new any memory in their
~this() {}...
Re: core.exception.InvalidMemoryOperationError@(0) on File Reading.
On Monday, 22 June 2015 at 20:30:40 UTC, David DeWitt wrote: I am getting an core.exception.InvalidMemoryOperationError@(0) auto recs = f // Open for reading .byLineCopy(); .array; //Here is where is appears to be happening. I have narrowed it down to the .array. I am wondering if there is some common issue I may be experiencing or if there is a better solution? You may be hitting issue 14005/14578 with the underlying issue 13856. https://issues.dlang.org/show_bug.cgi?id=14005 https://issues.dlang.org/show_bug.cgi?id=14578 https://issues.dlang.org/show_bug.cgi?id=13856 You can try using the current development version of phobos from git. I think the InvalidMemoryOperationError shouldn't happen with that anymore. If it still happens with git head phobos, it's probably not 14005/14578. Alternatively, you can use std.stdio.readln directly, but be aware of issue 13856.
Re: core.exception.InvalidMemoryOperationError@(0) on File Reading.
On Monday, 22 June 2015 at 20:30:40 UTC, David DeWitt wrote: I am getting an core.exception.InvalidMemoryOperationError@(0) auto recs = f // Open for reading .byLineCopy(); .array; //Here is where is appears to be happening. [...] I have had a chance to try the new phobos. Dealing with the errors related to: http://forum.dlang.org/thread/mm1fdo$q5u$1...@digitalmars.com in debian. Gonna mess with it some more when I have a chance.
Re: core.exception.InvalidMemoryOperationError@(0) on File Reading.
On Tuesday, 23 June 2015 at 18:49:59 UTC, David DeWitt wrote: On Monday, 22 June 2015 at 20:30:40 UTC, David DeWitt wrote: I am getting an core.exception.InvalidMemoryOperationError@(0) auto recs = f // Open for reading .byLineCopy(); .array; //Here is where is appears to be happening. [...] I have had a chance to try the new phobos. Dealing with the errors related to: http://forum.dlang.org/thread/mm1fdo$q5u$1...@digitalmars.com in debian. Gonna mess with it some more when I have a chance. I pulled down the latest version on Github and everything ran fine.
foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
Hi,
I'm trying to read a file line by line, and I get a
core.exception.InvalidMemoryOperationError@(0), even after
reducing the program to:
import std.stdio;
void main()
{
File f = File("testfile");
foreach(line; f.byLine)
{
}
}
The file is a simple table of ascii characters, 811 columns and
it fails on the second line. Taking any subset of the columns and
the program runs fine so I don't think it can by any particular
file corruption.
In this simple example I can get it to work by changing:
foreach(line; f.byLine)
to
foreach(char[] line; f.byLine)
but in my more complicated program this still fails with the same
error:
foreach (char[] lineVar; inFile.byLine)
{
lineVar.split.indexed(places).joiner("\t").writeln;
}
(as does the range version I originally wrote:
inFile.byLine.map!(a =>
a.split.indexed(places).joiner("\t")).joiner("\n").writeln;)
Is this a bug, gdc on version 2.066 seems to have no problems
with it? I'd be happy to fill in a bug report, but I can't share
the file as it's sensitive genetic data and I haven't been able
to reduce it to anything innocuous.
This has me very puzzled, any suggestions would be much
appreciated.
Thanks very much
Andrew
Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
On Tue, 15 Sep 2015 13:56:36 +
Andrwe Brown via Digitalmars-d-learn
wrote:
> Hi,
>
> I'm trying to read a file line by line, and I get a
> core.exception.InvalidMemoryOperationError@(0), even after
> reducing the program to:
>
> import std.stdio;
>
> void main()
> {
>File f = File("testfile");
>foreach(line; f.byLine)
>{
>}
> }
>
> The file is a simple table of ascii characters, 811 columns and
> it fails on the second line. Taking any subset of the columns and
> the program runs fine so I don't think it can by any particular
> file corruption.
>
> In this simple example I can get it to work by changing:
>
> foreach(line; f.byLine)
>
> to
>
> foreach(char[] line; f.byLine)
>
> but in my more complicated program this still fails with the same
> error:
>
> foreach (char[] lineVar; inFile.byLine)
> {
>lineVar.split.indexed(places).joiner("\t").writeln;
> }
>
> (as does the range version I originally wrote:
> inFile.byLine.map!(a =>
> a.split.indexed(places).joiner("\t")).joiner("\n").writeln;)
>
> Is this a bug, gdc on version 2.066 seems to have no problems
> with it? I'd be happy to fill in a bug report, but I can't share
> the file as it's sensitive genetic data and I haven't been able
> to reduce it to anything innocuous.
>
> This has me very puzzled, any suggestions would be much
> appreciated.
>
> Thanks very much
>
> Andrew
Which OS?
Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
On Tuesday, 15 September 2015 at 13:56:37 UTC, Andrwe Brown wrote: I'm trying to read a file line by line, and I get a core.exception.InvalidMemoryOperationError@(0), even after https://issues.dlang.org/show_bug.cgi?id=13856 Try DMD 2.068, it has got fixed byLine implementation.
Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
On Tuesday, 15 September 2015 at 14:19:13 UTC, Daniel Kozák wrote: Which OS? It's CentOS release 6.5 (Final), I tried dmd 2.068.1 and the problem has disappeared. Thanks very much for the advice, I can stick to old gdc for speed until ldc catches up to 2.068. Best Andrew
Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
For reference, it was this PR: https://github.com/D-Programming-Language/phobos/pull/3089 which fixed the same issue for me.
Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
On Tuesday, 15 September 2015 at 14:55:42 UTC, Martin Krejcirik wrote: For reference, it was this PR: https://github.com/D-Programming-Language/phobos/pull/3089 which fixed the same issue for me. A very naive question: would it be possible in this case to backport it into gdc/ldc by copying the pull request and building the compiler from source, or would this get me into a world of pain?
Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
On Tuesday, 15 September 2015 at 15:28:23 UTC, Andrew Brown wrote: A very naive question: would it be possible in this case to backport it into gdc/ldc by copying the pull request and building the compiler from source, or would this get me into a world of pain? Cherry-picking should work and merge cleanly. I have done it for DMD 2.067. I don't know how difficult it is to recompile Phobos and Druntime with LDC/GDC though.
Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066
Thanks very much for your help, it seemed to work a treat (I hope :))! Compiling ldc wasn't too bad, make the changes to runtime/phobos/std/stdio.d and then just building as normal was no problem. Unittests are passing and it handles that file perfectly. On Tuesday, 15 September 2015 at 16:11:06 UTC, Martin Krejcirik wrote: On Tuesday, 15 September 2015 at 15:28:23 UTC, Andrew Brown wrote: A very naive question: would it be possible in this case to backport it into gdc/ldc by copying the pull request and building the compiler from source, or would this get me into a world of pain? Cherry-picking should work and merge cleanly. I have done it for DMD 2.067. I don't know how difficult it is to recompile Phobos and Druntime with LDC/GDC though.
