Re: Derby client fails to connect to server every time after updating to 10.8.1.2
On 02.09.11 07:40, Trejkaz wrote: Hi all. We have tests (and apps, naturally) which start up the network server and then test that connecting to it works. One test runs our wrapping around the server and uses bare JDBC code to verify that it's connectable. The other test runs our wrapping around the client and uses bare NetworkServerControl calls to start the server. Both of these tests fail after updating from 10.6.1.0 - 10.8.1.2. Hi, The code below works for me with 10.8.1.2. I also added code to select from a system table - just to get some output. I don't have to set the user either (I think a default will be used). So why is it working for me, but not for you: o which Derby jars are you running off? o what are typical values of the scratch variable? o which platform are you running on? o which JVM is being used? o anything else in the test framework (or apps) that may cause this? (i.e., is configuration settings being picked up from somewhere?) Regards, -- Kristian The error I am getting: Fri Sep 02 15:21:40 EST 2011 : Execution failed because of a Distributed Protocol Error: DRDA_Proto_SYNTAXRM; CODPNT arg = 115a; Error Code Value = 9. Plaintext connection attempt from an SSL enabled client? org.apache.derby.impl.drda.DRDAProtocolException: Execution failed because of a Distributed Protocol Error: DRDA_Proto_SYNTAXRM; CODPNT arg = 115a; Error Code Value = 9. Plaintext connection attempt from an SSL enabled client? at org.apache.derby.impl.drda.DRDAConnThread.throwSyntaxrm(DRDAConnThread.java:517) at org.apache.derby.impl.drda.DRDAConnThread.tooBig(DRDAConnThread.java:8160) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSAT(DRDAConnThread.java:1583) at org.apache.derby.impl.drda.DRDAConnThread.exchangeServerAttributes(DRDAConnThread.java:1148) at org.apache.derby.impl.drda.DRDAConnThread.sessionInitialState(DRDAConnThread.java:672) at org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:283) I joined the unit tests to form one test which doesn't use any of our own code, and the same error occurs. The test does require a 'scratch' field to be set up... which is unfortunately still part of our test framework. :/ @Test public void test() throws Exception { File dbDir = new File(scratch, db); EmbeddedConnectionPoolDataSource ds1 = new EmbeddedConnectionPoolDataSource(); ds1.setDatabaseName(dbDir.getAbsolutePath()); ds1.setCreateDatabase(create); ds1.getConnection().close(); NetworkServerControl server = new NetworkServerControl(InetAddress.getByName(127.0.0.1), 27000); try { server.start(null); ClientConnectionPoolDataSource ds = new ClientConnectionPoolDataSource(); ds.setServerName(127.0.0.1); ds.setPortNumber(27000); ds.setDatabaseName(dbDir.getAbsolutePath()); ds.setUser(bob); Connection connection = ds.getConnection(); connection.close(); } finally { server.shutdown(); // clean up after ourselves EmbeddedConnectionPoolDataSource ds2 = new EmbeddedConnectionPoolDataSource(); ds2.setDatabaseName(dbDir.getAbsolutePath()); ds2.setShutdownDatabase(shutdown); try { ds2.getConnection(); } catch (SQLException e) { /* expected */ } } } The same code still works fine on 10.6.1.0. Additionally, on 10.6.1.0 I don't have to set the user. I searched the changelog and couldn't see anything about a user now being mandatory, but I assume this has changed for security reasons? TX
Re: Derby client fails to connect to server every time after updating to 10.8.1.2
On Fri, Sep 2, 2011 at 4:18 PM, Kristian Waagan kristian.waa...@oracle.com wrote: On 02.09.11 07:40, Trejkaz wrote: Hi all. We have tests (and apps, naturally) which start up the network server and then test that connecting to it works. One test runs our wrapping around the server and uses bare JDBC code to verify that it's connectable. The other test runs our wrapping around the client and uses bare NetworkServerControl calls to start the server. Both of these tests fail after updating from 10.6.1.0 - 10.8.1.2. Hi, The code below works for me with 10.8.1.2. I also added code to select from a system table - just to get some output. I don't have to set the user either (I think a default will be used). So why is it working for me, but not for you: o which Derby jars are you running off? I figured it out, and it was related to this. We have some minor changes in our vendor copy of Derby, so we build our own jars. The changes themselves are nowhere near the client-server area, *but* Derby's build.xml seems to have this new thing which runs svnrevision and puts the result into the source code. In the best case, it would show our revision number. In the worst case, someone might have vendored it to a git repository instead, and then I have no idea what it would display. Anyway, what it actually displays for us is this: Fri Sep 02 15:21:40 EST 2011 : Apache Derby Network Server - 10.8.1.2 - (cygwin warning: MS-DOS style path detected: C:\Data\Projects\trunk\vendor\derby\build\work/.svn/entries Preferred POSIXexported equivalent is: /cygdrive/c/Data/Projects/trunk/vendor/derby/build/work/.svn/entries CYGWIN environment variable option nodosfilewarning turns off this warning. Consult the user's guide for more details about POSIX paths: http://cygwin.com/cygwin-ug-net/using.html#using-pathnames) started and ready to accept connections on port 27000 I hacked the build so that it doesn't run svnversion and instead just puts in 0... I hope that won't break anything else, but tests pass at least. TX
Derby and database names containing a semicolon
Hi all. I am getting an error like this when the database directory has a semicolon in it: java.sql.SQLException: The URL 'jdbc:derby:C:\Users\Me\Desktop\one;two' is not properly formed. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:142) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:148) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:227) at org.apache.derby.jdbc.InternalDriver.getAttributes(InternalDriver.java:376) at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:190) at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:480) at org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(EmbedPooledConnection.java:178) at org.apache.derby.jdbc.EmbedPooledConnection.init(EmbedPooledConnection.java:119) at org.apache.derby.jdbc.EmbedPooledConnection40.init(EmbedPooledConnection40.java:54) at org.apache.derby.jdbc.Driver40.getNewPooledConnection(Driver40.java:178) at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.createPooledConnection(EmbeddedConnectionPoolDataSource.java:129) at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.getPooledConnection(EmbeddedConnectionPoolDataSource.java:75) However, I am using EmbeddedConnectionPoolDataSource, so I am not the one providing the URL. It seems like Derby is generating a URL which it subsequently decides is invalid. Q1. Shouldn't it be escaping it or something? Q2. Is there some way to get around this? TX
[no subject]
Hi My application (Anthillpro) is currently Apache Derby - 10.1.3.1. I am looking at various ways to improve performance. I would like to try setting the following property and am looking for some assurance from experienced users that there will be no ill affect. Any comments would be much appreciated. derby.storage.pageCacheSize Function Defines the size, in number of pages, of the database's data page cache (data pages kept in memory). The actual amount of memory the page cache will use depends on the following: * the size of the cache (configured with this property, derby.storage.pageCacheSize) * the size of the pages (configured with the derby.storage.pageSizehttp://db.apache.org/derby/docs/10.2/tuning/rtunproper40688.html#rtunproper40688 property) * overhead (varies with JVMs) When increasing the size of the page cache, you typically have to allow more memory for the Java heap when starting the embedding application (taking into consideration, of course, the memory needs of the embedding application as well). For example, using the default page size of 4K, a page cache size of 2000 pages will require at least 8 MB of memory (and probably more, given the overhead). For a simple application (no GUI), using the Sun 1.1.7 JVM on Windows NT and using the -mx96m option (which allows 96 MB for the Java heap), it is possible to have a page cache size of 10,000 pages (approximately 40 MB). The minimum value is 40 pages. If you specify a lower value, Derby uses the default value. John Godschall Release Engineer | First Marblehead 781.658.5028 (direct) 617.513.1912 (mobile) jgodsch...@fmd.commailto:jgodsch...@fmd.com *This e-mail and any attachments may contain content protected under federal law and is also confidential and proprietary in nature. If you received this message in error, please notify the sender immediately and delete the original and destroy all copies of the message and any attachments. Any other use of this e-mail by you including retaining, using, copying, distributing, or otherwise disclosing this information in any manner is prohibited.
Re:
Increasing the pageCacheSize will of course increase the heap memory used by your running application. If you have the memory to spare, I'm not aware of any harm to increasing it. As for performance and if increasing it would help; it would most likely be determined by how you are using the database: -Few updates/deletes, Data mostly static vs. Highly dynamic Data constantly changing -Selecting the Same Data Sets or Large Data Sets Multiple times vs. Selecting highly selective rows (1 row only) in a Large Database. I've found that with the latter of the two, increasing the page cache size didn't really help out a whole lot, But if you have either of the former, then it might be good. On Fri, Sep 2, 2011 at 1:12 PM, Godschall, John jgodsch...@firstmarblehead.com wrote: Hi My application (Anthillpro) is currently Apache Derby - 10.1.3.1. I am looking at various ways to improve performance. ** ** I would like to try setting the following property and am looking for some assurance from experienced users that there will be no ill affect. Any comments would be much appreciated. ** ** ** ** derby.storage.pageCacheSize Function Defines the size, in number of pages, of the database's data page cache (data pages kept in memory). The actual amount of memory the page cache will use depends on the following: - the size of the cache (configured with this property, derby.storage.pageCacheSize) - the size of the pages (configured with the derby.storage.pageSizehttp://db.apache.org/derby/docs/10.2/tuning/rtunproper40688.html#rtunproper40688property) - overhead (varies with JVMs) When increasing the size of the page cache, you typically have to allow more memory for the Java heap when starting the embedding application (taking into consideration, of course, the memory needs of the embedding application as well). For example, using the default page size of 4K, a page cache size of 2000 pages will require at least 8 MB of memory (and probably more, given the overhead). For a simple application (no GUI), using the Sun 1.1.7 JVM on Windows NT and using the -mx96m option (which allows 96 MB for the Java heap), it is possible to have a page cache size of 10,000 pages (approximately 40 MB).* *** The minimum value is 40 pages. If you specify a lower value, Derby uses the default value. ** ** John Godschall Release Engineer | First Marblehead 781.658.5028 (direct) 617.513.1912 (mobile) jgodsch...@fmd.com *This e-mail and any attachments may contain content protected under federal law and is also confidential and proprietary in nature. If you received this message in error, please notify the sender immediately and delete the original and destroy all copies of the message and any attachments. Any other use of this e-mail by you including retaining, using, copying, distributing, or otherwise disclosing this information in any manner is prohibited.
Opinions on new security feature requested
Hi folks, we are always working to make Derby more secure; in this day and age, security is ever more on people's minds for obvious reasons; IT systems are everywhere and the bad guys never tire of finding new holes to break them. Up till now, Derby creates database files and logs using the default operating system permission in effect. On Unix/linux this is controlled by the umask of the process starting the Derby engine, be in embedded or a standalone Derby network server. Similarly on Windows, NTFS has a security model can be configured to give various default permissions. Now, often the defaults will allow other (than the one starting the VM) OS users the permissions to read and possibly write to the database files. This can be intention to allow several users to boot the same (shared) database), or it can be accidental. In DERBY-5363 we have been discussion use cases and scenarios for when it would be desirable to let Derby be more secure than the default permissions. Other database also do this, e.g. PostGreSQL, MS SQL server. Typically, only the OS user creating the database would have access (default behavior) unless one told the database to be lax and not worry about tightening up the default OS permissions. Obviously, one can achieve the same restrictive permissions, by setting the umask to 0077 on Unix, or tweak the NTFS settings similarly (Windows), but this requires some care and presumes that the users remembers to do so (many people don't grok the NTFS security model..) To be clear, one would be able to enable/disable this extra security by providing Derby with a property setting, so the question is really what is the msot appropriate default: use lax permissions (rely on the user having tightened up be fore starting the VM), or use the new proactive secure settings proposed in DERBY-5363. Secure default pros: - users will get better security by default. If one needs to share the database files, one can use a property to get old, lax behavior. - no need to change startup scripts to get better security Secure default cons: - upwards compatibility: if an installation relies on sharing database files, on would have to start using a property after upgrading. - requires at least Java 6 (on Unix), Java 7 on Windows/NTFS to work (an incentive to upgrade, though :-) In the discussion it as been suggested that many deployments, especially of embedded Derby, rely on several OS users having permissions, so changing the default Derby behavior would cause upgrade issues. Probably for most client/server deployments, where the server is started from the command line, it would be the same OS user starting the Derby server every time. In mixed deployments (embedded, but the server is sometimes started via the API), the latter may not hold true. A possible trade-off between the concerns would thus be to start embedded with the exisiting, lax permissions by default, but start the server from the command line with a secure (restrictive) default. In both cases, one would get the opposite behavior by providing a system property on VM startup. Before we settle the discussion on this, it would be good to hear what you think! Thanks! Dag
Re:
Thanks David From: David Zanter [mailto:dzan...@gmail.com] Sent: Friday, September 02, 2011 01:31 PM To: Derby Discussion derby-user@db.apache.org Subject: Re: Increasing the pageCacheSize will of course increase the heap memory used by your running application. If you have the memory to spare, I'm not aware of any harm to increasing it. As for performance and if increasing it would help; it would most likely be determined by how you are using the database: -Few updates/deletes, Data mostly static vs. Highly dynamic Data constantly changing -Selecting the Same Data Sets or Large Data Sets Multiple times vs. Selecting highly selective rows (1 row only) in a Large Database. I've found that with the latter of the two, increasing the page cache size didn't really help out a whole lot, But if you have either of the former, then it might be good. On Fri, Sep 2, 2011 at 1:12 PM, Godschall, John jgodsch...@firstmarblehead.commailto:jgodsch...@firstmarblehead.com wrote: Hi My application (Anthillpro) is currently Apache Derby - 10.1.3.1. I am looking at various ways to improve performance. I would like to try setting the following property and am looking for some assurance from experienced users that there will be no ill affect. Any comments would be much appreciated. derby.storage.pageCacheSize Function Defines the size, in number of pages, of the database's data page cache (data pages kept in memory). The actual amount of memory the page cache will use depends on the following: * the size of the cache (configured with this property, derby.storage.pageCacheSize) * the size of the pages (configured with the derby.storage.pageSizehttp://db.apache.org/derby/docs/10.2/tuning/rtunproper40688.html#rtunproper40688 property) * overhead (varies with JVMs) When increasing the size of the page cache, you typically have to allow more memory for the Java heap when starting the embedding application (taking into consideration, of course, the memory needs of the embedding application as well). For example, using the default page size of 4K, a page cache size of 2000 pages will require at least 8 MB of memory (and probably more, given the overhead). For a simple application (no GUI), using the Sun 1.1.7 JVM on Windows NT and using the -mx96m option (which allows 96 MB for the Java heap), it is possible to have a page cache size of 10,000 pages (approximately 40 MB). The minimum value is 40 pages. If you specify a lower value, Derby uses the default value. John Godschall Release Engineer | First Marblehead 781.658.5028tel:781.658.5028 (direct) 617.513.1912tel:617.513.1912 (mobile) jgodsch...@fmd.commailto:jgodsch...@fmd.com *This e-mail and any attachments may contain content protected under federal law and is also confidential and proprietary in nature. If you received this message in error, please notify the sender immediately and delete the original and destroy all copies of the message and any attachments. Any other use of this e-mail by you including retaining, using, copying, distributing, or otherwise disclosing this information in any manner is prohibited. *This e-mail and any attachments may contain content protected under federal law and is also confidential and proprietary in nature. If you received this message in error, please notify the sender immediately and delete the original and destroy all copies of the message and any attachments. Any other use of this e-mail by you including retaining, using, copying, distributing, or otherwise disclosing this information in any manner is prohibited.
Re: Opinions on new security feature requested
+1 for more restrictive permissions. Actually when I run Derby on Unix it runs under own user+group and database files are not accessible by others. On Fri, Sep 2, 2011 at 7:31 PM, Dag H. Wanvik dag.wan...@oracle.com wrote: Hi folks, we are always working to make Derby more secure; in this day and age, security is ever more on people's minds for obvious reasons; IT systems are everywhere and the bad guys never tire of finding new holes to break them. Up till now, Derby creates database files and logs using the default operating system permission in effect. On Unix/linux this is controlled by the umask of the process starting the Derby engine, be in embedded or a standalone Derby network server. Similarly on Windows, NTFS has a security model can be configured to give various default permissions. Now, often the defaults will allow other (than the one starting the VM) OS users the permissions to read and possibly write to the database files. This can be intention to allow several users to boot the same (shared) database), or it can be accidental. In DERBY-5363 we have been discussion use cases and scenarios for when it would be desirable to let Derby be more secure than the default permissions. Other database also do this, e.g. PostGreSQL, MS SQL server. Typically, only the OS user creating the database would have access (default behavior) unless one told the database to be lax and not worry about tightening up the default OS permissions. Obviously, one can achieve the same restrictive permissions, by setting the umask to 0077 on Unix, or tweak the NTFS settings similarly (Windows), but this requires some care and presumes that the users remembers to do so (many people don't grok the NTFS security model..) To be clear, one would be able to enable/disable this extra security by providing Derby with a property setting, so the question is really what is the msot appropriate default: use lax permissions (rely on the user having tightened up be fore starting the VM), or use the new proactive secure settings proposed in DERBY-5363. Secure default pros: - users will get better security by default. If one needs to share the database files, one can use a property to get old, lax behavior. - no need to change startup scripts to get better security Secure default cons: - upwards compatibility: if an installation relies on sharing database files, on would have to start using a property after upgrading. - requires at least Java 6 (on Unix), Java 7 on Windows/NTFS to work (an incentive to upgrade, though :-) In the discussion it as been suggested that many deployments, especially of embedded Derby, rely on several OS users having permissions, so changing the default Derby behavior would cause upgrade issues. Probably for most client/server deployments, where the server is started from the command line, it would be the same OS user starting the Derby server every time. In mixed deployments (embedded, but the server is sometimes started via the API), the latter may not hold true. A possible trade-off between the concerns would thus be to start embedded with the exisiting, lax permissions by default, but start the server from the command line with a secure (restrictive) default. In both cases, one would get the opposite behavior by providing a system property on VM startup. Before we settle the discussion on this, it would be good to hear what you think! Thanks! Dag
Re: Opinions on new security feature requested
Unless it is completely unavoidable, it should be possible to install a later version AND NOT BREAK AN EXISTING APPLICATION. To do so is rude and can be very disruptive. - Original Message - From: Dag H. Wanvik dag.wan...@oracle.com To: Derby Discussion derby-user@db.apache.org Sent: Friday, September 2, 2011 1:31:46 PM Subject: Opinions on new security feature requested Hi folks, we are always working to make Derby more secure; in this day and age, security is ever more on people's minds for obvious reasons; IT systems are everywhere and the bad guys never tire of finding new holes to break them. Up till now, Derby creates database files and logs using the default operating system permission in effect. On Unix/linux this is controlled by the umask of the process starting the Derby engine, be in embedded or a standalone Derby network server. Similarly on Windows, NTFS has a security model can be configured to give various default permissions. Now, often the defaults will allow other (than the one starting the VM) OS users the permissions to read and possibly write to the database files. This can be intention to allow several users to boot the same (shared) database), or it can be accidental. In DERBY-5363 we have been discussion use cases and scenarios for when it would be desirable to let Derby be more secure than the default permissions. Other database also do this, e.g. PostGreSQL, MS SQL server. Typically, only the OS user creating the database would have access (default behavior) unless one told the database to be lax and not worry about tightening up the default OS permissions. Obviously, one can achieve the same restrictive permissions, by setting the umask to 0077 on Unix, or tweak the NTFS settings similarly (Windows), but this requires some care and presumes that the users remembers to do so (many people don't grok the NTFS security model..) To be clear, one would be able to enable/disable this extra security by providing Derby with a property setting, so the question is really what is the msot appropriate default: use lax permissions (rely on the user having tightened up be fore starting the VM), or use the new proactive secure settings proposed in DERBY-5363. Secure default pros: - users will get better security by default. If one needs to share the database files, one can use a property to get old, lax behavior. - no need to change startup scripts to get better security Secure default cons: - upwards compatibility: if an installation relies on sharing database files, on would have to start using a property after upgrading. - requires at least Java 6 (on Unix), Java 7 on Windows/NTFS to work (an incentive to upgrade, though :-) In the discussion it as been suggested that many deployments, especially of embedded Derby, rely on several OS users having permissions, so changing the default Derby behavior would cause upgrade issues. Probably for most client/server deployments, where the server is started from the command line, it would be the same OS user starting the Derby server every time. In mixed deployments (embedded, but the server is sometimes started via the API), the latter may not hold true. A possible trade-off between the concerns would thus be to start embedded with the exisiting, lax permissions by default, but start the server from the command line with a secure (restrictive) default. In both cases, one would get the opposite behavior by providing a system property on VM startup. Before we settle the discussion on this, it would be good to hear what you think! Thanks! Dag