Re: Table exists in same JVM after Derby is shutdown
On 09.08.11 17:14, Pavel Bortnovskiy wrote: Thank you, Kristian. Can you please clarify some of the points you made (or point me to an online primer that describes it): - what is the difference between shutting down a specific db and the whole engine? Shutting down a specific database will close all open handles to that database and free up resources associated with the database. The next connection attempt to this database will cause it to be booted again. When a database isn't booted, i.e. it has been shut down, it can also be safely copied using OS utilities [1]. Shutting down the Derby engine usually makes the Derby classes eligible [2] for garbage collection. You won't be able to obtain a new connection to a Derby database before you reload the engine/driver [3]. Shutting down the engine also shuts down all booted databases. - what does it mean that shutting down the Derby engine which runs an in-memory-only DB doesn't drop tables? It means that the in-memory database will be kept in memory, just as an on-disk database will be kept on disk. The in-memory databases are stored in a static variable. The ways to get rid of them are to explicitly drop them, to have the Derby classes garbage collected [2] , or to exit the JVM. Are you saying that if that engine were to be restarted, the tables would be accessible again? Yes, all in-memory databases would be accessible again. Does it also mean that the memory taken by the engine and its tables doesn't get reclaimed until it's shutdown (or dropped)? Shutting down the engine will free up resources like the page cache, and most, but not all, instances of Derby classes will be garbage collected. If in-memory databases have been created, but not dropped, these will remain in memory. Regards, -- Kristian [1] There are other ways to safely copy a database, for instance freezing it while booted or using the Derby backup routines. [2] To actually unload the Derby classes, the class loader that loaded them has to be eligible for gc too. Simply shutting down the engine is not sufficient (see also [3]). Note the difference between garbage collecting instances of Derby classes and unloading the Derby classes themselves. [3] This changed somewhat with the addition of the deregister=*true*|false attribute. If false, the driver will be left registered even if the engine is shut down. Regards, Pavel. From: Kristian Waagan kristian.waa...@oracle.com To: derby-user@db.apache.org Date: 08/09/2011 01:42 AM Subject:Re: Table exists in same JVM after Derby is shutdown On 08.08.11 16:41, Pavel Bortnovskiy wrote: Thank you, Dag. It looks like in order to get this to work, I had to make two calls to DriverManager.getConnection. Combining /shutdown=true;drop=true/ didn't work. So, is this the correct implementation (if in-line image doesn't work, please see attachment): Hello Pavel, To simply drop an in-memory database, you only need the drop attribute. Shutdown of the specified database happens as part of the drop operation. It is correct that you need two calls to DriverManager.getConnection if you want to drop a database and additionally shut down the Derby engine. For clarity, note that there is a difference between shutting down a specific database and the Derby engine/system. Further, shutting down an in-memory database does not drop it, neither does shutting down the Derby engine. I don't know, maybe it would make sense to drop all in-memory databases when the Derby system is shut down - but that's not being done currently. Hope this helps, -- Kristian From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 06:36 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: 1) I create an in-memory database and then a table in it. Then the database is shut down. I would expect that the shutdown effectively cleans everything up. Is it not so? Ah, I misunderstood you Pavel. Have a look here: http://wiki.apache.org/db-derby/InMemoryBackEndPrimer As you can see, you need to provide the drop=true connection attribute to wipe the data. Thanks, Dag Thanks, Pavel. From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 05:57 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: Hello, all: While executing a bunch of JUnit tests within the same JVM (all executed by IntelliJ IDEA) I started seeing strange and unexpected errors occurring . Upon closer inspection, I noticed that in many
Re: Table exists in same JVM after Derby is shutdown
Thank you, Kristian. Can you please clarify some of the points you made (or point me to an online primer that describes it): - what is the difference between shutting down a specific db and the whole engine? - what does it mean that shutting down the Derby engine which runs an in-memory-only DB doesn't drop tables? Are you saying that if that engine were to be restarted, the tables would be accessible again? Does it also mean that the memory taken by the engine and its tables doesn't get reclaimed until it's shutdown (or dropped)? Regards, Pavel. From: Kristian Waagan kristian.waa...@oracle.com To: derby-user@db.apache.org Date: 08/09/2011 01:42 AM Subject: Re: Table exists in same JVM after Derby is shutdown On 08.08.11 16:41, Pavel Bortnovskiy wrote: Thank you, Dag. It looks like in order to get this to work, I had to make two calls to DriverManager.getConnection. Combining /shutdown=true;drop=true/ didn't work. So, is this the correct implementation (if in-line image doesn't work, please see attachment): Hello Pavel, To simply drop an in-memory database, you only need the drop attribute. Shutdown of the specified database happens as part of the drop operation. It is correct that you need two calls to DriverManager.getConnection if you want to drop a database and additionally shut down the Derby engine. For clarity, note that there is a difference between shutting down a specific database and the Derby engine/system. Further, shutting down an in-memory database does not drop it, neither does shutting down the Derby engine. I don't know, maybe it would make sense to drop all in-memory databases when the Derby system is shut down - but that's not being done currently. Hope this helps, -- Kristian From: dag.wan...@oracle.com (Dag H. Wanvik) To:Derby Discussion derby-user@db.apache.org Date: 08/05/2011 06:36 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: 1) I create an in-memory database and then a table in it. Then the database is shut down. I would expect that the shutdown effectively cleans everything up. Is it not so? Ah, I misunderstood you Pavel. Have a look here: http://wiki.apache.org/db-derby/InMemoryBackEndPrimer As you can see, you need to provide the drop=true connection attribute to wipe the data. Thanks, Dag Thanks, Pavel. From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 05:57 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: Hello, all: While executing a bunch of JUnit tests within the same JVM (all executed by IntelliJ IDEA) I started seeing strange and unexpected errors occurring . Upon closer inspection, I noticed that in many of those tests tables with the same names are attempted to be created. Then I realized that although Derby is shutdown and then re-created, the tables remain, thus causing collisions. The create=true connection attribute is ignored (with a warning) if the database with the same name alrady exists. I've created a digest (attached) which is executed as one JUnit test to illustrate what I'm seeing. The behavior I would expect is that once Derby is shutdown, no tables would remain in the JVM, and if a database (with the same name) is re-created, it would be a tabula rasa. The tables are no longer in memory (or should not be unless you found a bug), but they are not erased from the disk image of the database. As per the above, one would need to explicitly delete it using OS file system tools for the data to be cleared. Some JUnit tests delete tables in TestCase#tearDown, others use singleUseDatabaseDecorator to isolate itself from the rest of the tests. The tests not necessarily very consistent in their patterns for this.. Dag Can you please let me know whether my expectations are erroneous and whether I should find workarounds (albeit trivial to implement). However, what would concern me in that case is that the tables and the data remain in the JVM, thus consuming memory (and, if unused, creating memory leaks). Thank you, Pavel. snip Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation
Re: Table exists in same JVM after Derby is shutdown
Thank you, Dag. It looks like in order to get this to work, I had to make two calls to DriverManager.getConnection. Combining shutdown=true;drop=true didn't work. So, is this the correct implementation (if in-line image doesn't work, please see attachment): From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 06:36 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: 1) I create an in-memory database and then a table in it. Then the database is shut down. I would expect that the shutdown effectively cleans everything up. Is it not so? Ah, I misunderstood you Pavel. Have a look here: http://wiki.apache.org/db-derby/InMemoryBackEndPrimer As you can see, you need to provide the drop=true connection attribute to wipe the data. Thanks, Dag Thanks, Pavel. From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 05:57 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: Hello, all: While executing a bunch of JUnit tests within the same JVM (all executed by IntelliJ IDEA) I started seeing strange and unexpected errors occurring . Upon closer inspection, I noticed that in many of those tests tables with the same names are attempted to be created. Then I realized that although Derby is shutdown and then re-created, the tables remain, thus causing collisions. The create=true connection attribute is ignored (with a warning) if the database with the same name alrady exists. I've created a digest (attached) which is executed as one JUnit test to illustrate what I'm seeing. The behavior I would expect is that once Derby is shutdown, no tables would remain in the JVM, and if a database (with the same name) is re-created, it would be a tabula rasa. The tables are no longer in memory (or should not be unless you found a bug), but they are not erased from the disk image of the database. As per the above, one would need to explicitly delete it using OS file system tools for the data to be cleared. Some JUnit tests delete tables in TestCase#tearDown, others use singleUseDatabaseDecorator to isolate itself from the rest of the tests. The tests not necessarily very consistent in their patterns for this.. Dag Can you please let me know whether my expectations are erroneous and whether I should find workarounds (albeit trivial to implement). However, what would concern me in that case is that the tables and the data remain in the JVM, thus consuming memory (and, if unused, creating memory leaks). Thank you, Pavel. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection
Re: Table exists in same JVM after Derby is shutdown
On 08.08.11 16:41, Pavel Bortnovskiy wrote: Thank you, Dag. It looks like in order to get this to work, I had to make two calls to DriverManager.getConnection. Combining /shutdown=true;drop=true/ didn't work. So, is this the correct implementation (if in-line image doesn't work, please see attachment): Hello Pavel, To simply drop an in-memory database, you only need the drop attribute. Shutdown of the specified database happens as part of the drop operation. It is correct that you need two calls to DriverManager.getConnection if you want to drop a database and additionally shut down the Derby engine. For clarity, note that there is a difference between shutting down a specific database and the Derby engine/system. Further, shutting down an in-memory database does not drop it, neither does shutting down the Derby engine. I don't know, maybe it would make sense to drop all in-memory databases when the Derby system is shut down - but that's not being done currently. Hope this helps, -- Kristian From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 06:36 PM Subject:Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: 1) I create an in-memory database and then a table in it. Then the database is shut down. I would expect that the shutdown effectively cleans everything up. Is it not so? Ah, I misunderstood you Pavel. Have a look here: http://wiki.apache.org/db-derby/InMemoryBackEndPrimer As you can see, you need to provide the drop=true connection attribute to wipe the data. Thanks, Dag Thanks, Pavel. From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 05:57 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: Hello, all: While executing a bunch of JUnit tests within the same JVM (all executed by IntelliJ IDEA) I started seeing strange and unexpected errors occurring . Upon closer inspection, I noticed that in many of those tests tables with the same names are attempted to be created. Then I realized that although Derby is shutdown and then re-created, the tables remain, thus causing collisions. The create=true connection attribute is ignored (with a warning) if the database with the same name alrady exists. I've created a digest (attached) which is executed as one JUnit test to illustrate what I'm seeing. The behavior I would expect is that once Derby is shutdown, no tables would remain in the JVM, and if a database (with the same name) is re-created, it would be a tabula rasa. The tables are no longer in memory (or should not be unless you found a bug), but they are not erased from the disk image of the database. As per the above, one would need to explicitly delete it using OS file system tools for the data to be cleared. Some JUnit tests delete tables in TestCase#tearDown, others use singleUseDatabaseDecorator to isolate itself from the rest of the tests. The tests not necessarily very consistent in their patterns for this.. Dag Can you please let me know whether my expectations are erroneous and whether I should find workarounds (albeit trivial to implement). However, what would concern me in that case is that the tables and the data remain in the JVM, thus consuming memory (and, if unused, creating memory leaks). Thank you, Pavel. snip Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
Table exists in same JVM after Derby is shutdown
Hello, all: While executing a bunch of JUnit tests within the same JVM (all executed by IntelliJ IDEA) I started seeing strange and unexpected errors occurring . Upon closer inspection, I noticed that in many of those tests tables with the same names are attempted to be created. Then I realized that although Derby is shutdown and then re-created, the tables remain, thus causing collisions. I've created a digest (attached) which is executed as one JUnit test to illustrate what I'm seeing. The behavior I would expect is that once Derby is shutdown, no tables would remain in the JVM, and if a database (with the same name) is re-created, it would be a tabula rasa. Can you please let me know whether my expectations are erroneous and whether I should find workarounds (albeit trivial to implement). However, what would concern me in that case is that the tables and the data remain in the JVM, thus consuming memory (and, if unused, creating memory leaks). Thank you, Pavel. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority. DerbyTest.java Description: Binary data
Re: Table exists in same JVM after Derby is shutdown
Pavel Bortnovskiy pbortnovs...@jefferies.com writes: Hello, all: While executing a bunch of JUnit tests within the same JVM (all executed by IntelliJ IDEA) I started seeing strange and unexpected errors occurring . Upon closer inspection, I noticed that in many of those tests tables with the same names are attempted to be created. Then I realized that although Derby is shutdown and then re-created, the tables remain, thus causing collisions. The create=true connection attribute is ignored (with a warning) if the database with the same name alrady exists. I've created a digest (attached) which is executed as one JUnit test to illustrate what I'm seeing. The behavior I would expect is that once Derby is shutdown, no tables would remain in the JVM, and if a database (with the same name) is re-created, it would be a tabula rasa. The tables are no longer in memory (or should not be unless you found a bug), but they are not erased from the disk image of the database. As per the above, one would need to explicitly delete it using OS file system tools for the data to be cleared. Some JUnit tests delete tables in TestCase#tearDown, others use singleUseDatabaseDecorator to isolate itself from the rest of the tests. The tests not necessarily very consistent in their patterns for this.. Dag Can you please let me know whether my expectations are erroneous and whether I should find workarounds (albeit trivial to implement). However, what would concern me in that case is that the tables and the data remain in the JVM, thus consuming memory (and, if unused, creating memory leaks). Thank you, Pavel. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
Re: Table exists in same JVM after Derby is shutdown
Pavel Bortnovskiy pbortnovs...@jefferies.com writes: 1) I create an in-memory database and then a table in it. Then the database is shut down. I would expect that the shutdown effectively cleans everything up. Is it not so? Ah, I misunderstood you Pavel. Have a look here: http://wiki.apache.org/db-derby/InMemoryBackEndPrimer As you can see, you need to provide the drop=true connection attribute to wipe the data. Thanks, Dag Thanks, Pavel. From: dag.wan...@oracle.com (Dag H. Wanvik) To: Derby Discussion derby-user@db.apache.org Date: 08/05/2011 05:57 PM Subject: Re: Table exists in same JVM after Derby is shutdown Pavel Bortnovskiy pbortnovs...@jefferies.com writes: Hello, all: While executing a bunch of JUnit tests within the same JVM (all executed by IntelliJ IDEA) I started seeing strange and unexpected errors occurring . Upon closer inspection, I noticed that in many of those tests tables with the same names are attempted to be created. Then I realized that although Derby is shutdown and then re-created, the tables remain, thus causing collisions. The create=true connection attribute is ignored (with a warning) if the database with the same name alrady exists. I've created a digest (attached) which is executed as one JUnit test to illustrate what I'm seeing. The behavior I would expect is that once Derby is shutdown, no tables would remain in the JVM, and if a database (with the same name) is re-created, it would be a tabula rasa. The tables are no longer in memory (or should not be unless you found a bug), but they are not erased from the disk image of the database. As per the above, one would need to explicitly delete it using OS file system tools for the data to be cleared. Some JUnit tests delete tables in TestCase#tearDown, others use singleUseDatabaseDecorator to isolate itself from the rest of the tests. The tests not necessarily very consistent in their patterns for this.. Dag Can you please let me know whether my expectations are erroneous and whether I should find workarounds (albeit trivial to implement). However, what would concern me in that case is that the tables and the data remain in the JVM, thus consuming memory (and, if unused, creating memory leaks). Thank you, Pavel. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
