Re: Derby client fails to connect to server every time after updating to 10.8.1.2

2011-09-02 Thread Kristian Waagan

On 02.09.11 07:40, Trejkaz wrote:

Hi all.

We have tests (and apps, naturally) which start up the network server
and then test that connecting to it works.  One test runs our wrapping
around the server and uses bare JDBC code to verify that it's
connectable.  The other test runs our wrapping around the client and
uses bare NetworkServerControl calls to start the server.  Both of
these tests fail after updating from 10.6.1.0 -  10.8.1.2.


Hi,

The code below works for me with 10.8.1.2. I also added code to select 
from a system table - just to get some output.

I don't have to set the user either (I think a default will be used).

So why is it working for me, but not for you:
 o which Derby jars are you running off?
 o what are typical values of the scratch variable?
 o which platform are you running on?
 o which JVM is being used?
 o anything else in the test framework (or apps) that may cause this?
   (i.e., is configuration settings being picked up from somewhere?)


Regards,
--
Kristian



The error I am getting:

Fri Sep 02 15:21:40 EST 2011 : Execution failed because of a
Distributed Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT arg  = 115a;
Error Code Value = 9. Plaintext connection attempt from an SSL enabled
client?
org.apache.derby.impl.drda.DRDAProtocolException: Execution failed
because of a Distributed Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT
arg  = 115a; Error Code Value = 9. Plaintext connection attempt from
an SSL enabled client?
at 
org.apache.derby.impl.drda.DRDAConnThread.throwSyntaxrm(DRDAConnThread.java:517)
at 
org.apache.derby.impl.drda.DRDAConnThread.tooBig(DRDAConnThread.java:8160)
at 
org.apache.derby.impl.drda.DRDAConnThread.parseEXCSAT(DRDAConnThread.java:1583)
at 
org.apache.derby.impl.drda.DRDAConnThread.exchangeServerAttributes(DRDAConnThread.java:1148)
at 
org.apache.derby.impl.drda.DRDAConnThread.sessionInitialState(DRDAConnThread.java:672)
at 
org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:283)

I joined the unit tests to form one test which doesn't use any of our
own code, and the same error occurs.  The test does require a
'scratch' field to be set up... which is unfortunately still part of
our test framework. :/

 @Test
 public void test() throws Exception
 {
 File dbDir = new File(scratch, db);

 EmbeddedConnectionPoolDataSource ds1 = new
EmbeddedConnectionPoolDataSource();
 ds1.setDatabaseName(dbDir.getAbsolutePath());
 ds1.setCreateDatabase(create);
 ds1.getConnection().close();

 NetworkServerControl server = new
NetworkServerControl(InetAddress.getByName(127.0.0.1), 27000);
 try
 {
 server.start(null);

 ClientConnectionPoolDataSource ds = new
ClientConnectionPoolDataSource();
 ds.setServerName(127.0.0.1);
 ds.setPortNumber(27000);
 ds.setDatabaseName(dbDir.getAbsolutePath());
 ds.setUser(bob);

 Connection connection = ds.getConnection();
 connection.close();
 }
 finally
 {
 server.shutdown();

 // clean up after ourselves
 EmbeddedConnectionPoolDataSource ds2 = new
EmbeddedConnectionPoolDataSource();
 ds2.setDatabaseName(dbDir.getAbsolutePath());
 ds2.setShutdownDatabase(shutdown);
 try { ds2.getConnection(); } catch (SQLException e) { /*
expected */ }
 }
 }

The same code still works fine on 10.6.1.0.

Additionally, on 10.6.1.0 I don't have to set the user.  I searched
the changelog and couldn't see anything about a user now being
mandatory, but I assume this has changed for security reasons?

TX




Re: Derby client fails to connect to server every time after updating to 10.8.1.2

2011-09-02 Thread Trejkaz
On Fri, Sep 2, 2011 at 4:18 PM, Kristian Waagan
kristian.waa...@oracle.com wrote:
 On 02.09.11 07:40, Trejkaz wrote:

 Hi all.

 We have tests (and apps, naturally) which start up the network server
 and then test that connecting to it works.  One test runs our wrapping
 around the server and uses bare JDBC code to verify that it's
 connectable.  The other test runs our wrapping around the client and
 uses bare NetworkServerControl calls to start the server.  Both of
 these tests fail after updating from 10.6.1.0 -  10.8.1.2.

 Hi,

 The code below works for me with 10.8.1.2. I also added code to select from
 a system table - just to get some output.
 I don't have to set the user either (I think a default will be used).

 So why is it working for me, but not for you:
  o which Derby jars are you running off?

I figured it out, and it was related to this.

We have some minor changes in our vendor copy of Derby, so we build
our own jars.  The changes themselves are nowhere near the
client-server area, *but*

Derby's build.xml seems to have this new thing which runs svnrevision
and puts the result into the source code.  In the best case, it would
show our revision number.  In the worst case, someone might have
vendored it to a git repository instead, and then I have no idea what
it would display.

Anyway, what it actually displays for us is this:

Fri Sep 02 15:21:40 EST 2011 : Apache Derby Network Server - 10.8.1.2
- (cygwin warning:  MS-DOS style path detected:
C:\Data\Projects\trunk\vendor\derby\build\work/.svn/entries  Preferred
POSIXexported equivalent is:
/cygdrive/c/Data/Projects/trunk/vendor/derby/build/work/.svn/entries
CYGWIN environment variable option nodosfilewarning turns off this
warning.  Consult the user's guide for more details about POSIX paths:
   http://cygwin.com/cygwin-ug-net/using.html#using-pathnames) started
and ready to accept connections on port 27000

I hacked the build so that it doesn't run svnversion and instead just
puts in 0... I hope that won't break anything else, but tests pass at
least.

TX


Derby and database names containing a semicolon

2011-09-02 Thread Trejkaz
Hi all.

I am getting an error like this when the database directory has a
semicolon in it:

java.sql.SQLException: The URL
'jdbc:derby:C:\Users\Me\Desktop\one;two' is not properly formed.
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:142)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:148)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:227)
at 
org.apache.derby.jdbc.InternalDriver.getAttributes(InternalDriver.java:376)
at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:190)
at 
org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:480)
at 
org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(EmbedPooledConnection.java:178)
at 
org.apache.derby.jdbc.EmbedPooledConnection.init(EmbedPooledConnection.java:119)
at 
org.apache.derby.jdbc.EmbedPooledConnection40.init(EmbedPooledConnection40.java:54)
at 
org.apache.derby.jdbc.Driver40.getNewPooledConnection(Driver40.java:178)
at 
org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.createPooledConnection(EmbeddedConnectionPoolDataSource.java:129)
at 
org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.getPooledConnection(EmbeddedConnectionPoolDataSource.java:75)


However, I am using EmbeddedConnectionPoolDataSource, so I am not the
one providing the URL.  It seems like Derby is generating a URL which
it subsequently decides is invalid.

Q1. Shouldn't it be escaping it or something?

Q2. Is there some way to get around this?

TX


[no subject]

2011-09-02 Thread Godschall, John
Hi
My application (Anthillpro) is currently Apache Derby - 10.1.3.1.  I am looking 
at various ways to improve performance.

I would like to try setting the following property and am looking for some 
assurance from experienced users that there will be no ill affect.
Any comments would be much appreciated.


derby.storage.pageCacheSize
Function

Defines the size, in number of pages, of the database's data page cache (data 
pages kept in memory).
The actual amount of memory the page cache will use depends on the following:

  *   the size of the cache (configured with this property, 
derby.storage.pageCacheSize)
  *   the size of the pages (configured with the 
derby.storage.pageSizehttp://db.apache.org/derby/docs/10.2/tuning/rtunproper40688.html#rtunproper40688
 property)
  *   overhead (varies with JVMs)

When increasing the size of the page cache, you typically have to allow more 
memory for the Java heap when starting the embedding application (taking into 
consideration, of course, the memory needs of the embedding application as 
well). For example, using the default page size of 4K, a page cache size of 
2000 pages will require at least 8 MB of memory (and probably more, given the 
overhead).

For a simple application (no GUI), using the Sun 1.1.7 JVM on Windows NT and 
using the -mx96m option (which allows 96 MB for the Java heap), it is possible 
to have a page cache size of 10,000 pages (approximately 40 MB).

The minimum value is 40 pages. If you specify a lower value, Derby uses the 
default value.

John Godschall
Release Engineer | First Marblehead
781.658.5028 (direct)
617.513.1912 (mobile)
jgodsch...@fmd.commailto:jgodsch...@fmd.com
*This e-mail and any attachments may contain content protected 
under federal law and is also confidential and proprietary in nature. 
If you received this message in error, please notify the sender 
immediately and delete the original and destroy all copies of the 
message and any attachments. Any other use of this e-mail by you 
including retaining, using, copying, distributing, or otherwise 
disclosing this information in any manner is prohibited.



Re:

2011-09-02 Thread David Zanter
Increasing the pageCacheSize will of course increase the heap memory used by
your running application.  If you have the memory to spare, I'm not aware of
any harm to increasing it.

As for performance and if increasing it would help; it would most likely be
determined by how you are using the database:

-Few updates/deletes, Data mostly static  vs. Highly dynamic Data constantly
changing
-Selecting the Same Data Sets or Large Data Sets Multiple times vs.
Selecting highly selective rows (1 row only) in a Large Database.

I've found that with the latter of the two, increasing the page cache size
didn't really help out a whole lot, But if you have either of the former,
then it might be good.

On Fri, Sep 2, 2011 at 1:12 PM, Godschall, John 
jgodsch...@firstmarblehead.com wrote:

  Hi

 My application (Anthillpro) is currently Apache Derby - 10.1.3.1.  I am
 looking at various ways to improve performance.

 ** **

 I would like to try setting the following property and am looking for some
 assurance from experienced users that there will be no ill affect.

 Any comments would be much appreciated.

 ** **

 ** **
  derby.storage.pageCacheSize Function

 Defines the size, in number of pages, of the database's data page cache
 (data pages kept in memory).

 The actual amount of memory the page cache will use depends on the
 following: 

- the size of the cache (configured with this property,
derby.storage.pageCacheSize) 
- the size of the pages (configured with the 
 derby.storage.pageSizehttp://db.apache.org/derby/docs/10.2/tuning/rtunproper40688.html#rtunproper40688property)

- overhead (varies with JVMs) 

 When increasing the size of the page cache, you typically have to allow
 more memory for the Java heap when starting the embedding application
 (taking into consideration, of course, the memory needs of the embedding
 application as well). For example, using the default page size of 4K, a page
 cache size of 2000 pages will require at least 8 MB of memory (and probably
 more, given the overhead).

 For a simple application (no GUI), using the Sun 1.1.7 JVM on Windows NT
 and using the -mx96m option (which allows 96 MB for the Java heap), it is
 possible to have a page cache size of 10,000 pages (approximately 40 MB).*
 ***

 The minimum value is 40 pages. If you specify a lower value, Derby uses the
 default value.

 ** **

 John Godschall

 Release Engineer | First Marblehead 

 781.658.5028 (direct) 

 617.513.1912 (mobile) 

 jgodsch...@fmd.com
  *This e-mail and any attachments may contain content protected
 under federal law and is also confidential and proprietary in nature.
 If you received this message in error, please notify the sender
 immediately and delete the original and destroy all copies of the
 message and any attachments. Any other use of this e-mail by you
 including retaining, using, copying, distributing, or otherwise
 disclosing this information in any manner is prohibited.



Opinions on new security feature requested

2011-09-02 Thread Dag H. Wanvik

Hi folks,

we are always working to make Derby more secure; in this day and age, 
security is ever more on people's minds for obvious reasons;
IT systems are everywhere and the bad guys never tire of finding new 
holes to break them. Up till now, Derby creates database files and logs 
using the default operating system permission in effect. On Unix/linux 
this is controlled by the umask of the process starting the Derby 
engine, be in embedded or a standalone Derby network server. Similarly 
on Windows, NTFS has a security model can be configured to give various 
default permissions.


Now, often the defaults will allow other (than the one starting the VM) 
OS users the permissions to read and possibly write to the database 
files. This can be intention to allow several users to boot the same 
(shared) database), or it can be accidental. In DERBY-5363 we have been 
discussion use cases and scenarios for when it would be desirable to let 
Derby be more secure than the default permissions. Other database also 
do this, e.g. PostGreSQL, MS SQL server.


Typically, only the OS user creating the database would have access 
(default behavior) unless one told the database to be lax and not worry 
about tightening up the default OS permissions.
Obviously, one can achieve the same restrictive permissions, by setting 
the umask to 0077 on Unix, or tweak the NTFS settings similarly 
(Windows), but this requires some care and presumes that the users 
remembers to do so (many people don't grok the NTFS security model..)


To be clear, one would be able to enable/disable this extra security by 
providing Derby with a property setting, so the question is really what 
is the msot appropriate default: use lax permissions (rely on the user 
having tightened up be fore starting the VM), or use the new proactive 
secure settings proposed in DERBY-5363.


Secure default pros:
- users will get better security by default. If one needs to share the 
database files, one can use a property to get old, lax behavior.

- no need to change startup scripts to get better security

Secure default cons:
- upwards compatibility: if an installation relies on sharing database 
files, on would have to start using a property after upgrading.
- requires at least Java 6 (on Unix), Java 7 on Windows/NTFS to work (an 
incentive to upgrade, though :-)


In the discussion it as been suggested that many deployments, especially 
of embedded Derby, rely on several OS users having permissions, so 
changing the default Derby behavior would cause upgrade issues. Probably 
for most client/server deployments, where the server is started from the 
command line, it would be the same OS user starting the Derby server 
every time. In mixed deployments (embedded, but the server is sometimes 
started via the API), the latter may not hold true.


A possible trade-off between the concerns would thus be to start 
embedded with the exisiting, lax permissions by default, but start the 
server from the command line with a secure (restrictive) default. In 
both cases, one would get the opposite behavior by providing a system 
property on VM startup.


Before we settle the discussion on this, it would be good to hear what 
you think! Thanks!


Dag



Re:

2011-09-02 Thread Godschall, John
Thanks David

From: David Zanter [mailto:dzan...@gmail.com]
Sent: Friday, September 02, 2011 01:31 PM
To: Derby Discussion derby-user@db.apache.org
Subject: Re:


Increasing the pageCacheSize will of course increase the heap memory used by 
your running application.  If you have the memory to spare, I'm not aware of 
any harm to increasing it.

As for performance and if increasing it would help; it would most likely be 
determined by how you are using the database:

-Few updates/deletes, Data mostly static  vs. Highly dynamic Data constantly 
changing
-Selecting the Same Data Sets or Large Data Sets Multiple times vs. Selecting 
highly selective rows (1 row only) in a Large Database.

I've found that with the latter of the two, increasing the page cache size 
didn't really help out a whole lot, But if you have either of the former, then 
it might be good.

On Fri, Sep 2, 2011 at 1:12 PM, Godschall, John 
jgodsch...@firstmarblehead.commailto:jgodsch...@firstmarblehead.com wrote:
Hi
My application (Anthillpro) is currently Apache Derby - 10.1.3.1.  I am looking 
at various ways to improve performance.

I would like to try setting the following property and am looking for some 
assurance from experienced users that there will be no ill affect.
Any comments would be much appreciated.


derby.storage.pageCacheSize
Function

Defines the size, in number of pages, of the database's data page cache (data 
pages kept in memory).
The actual amount of memory the page cache will use depends on the following:

  *   the size of the cache (configured with this property, 
derby.storage.pageCacheSize)
  *   the size of the pages (configured with the 
derby.storage.pageSizehttp://db.apache.org/derby/docs/10.2/tuning/rtunproper40688.html#rtunproper40688
 property)
  *   overhead (varies with JVMs)

When increasing the size of the page cache, you typically have to allow more 
memory for the Java heap when starting the embedding application (taking into 
consideration, of course, the memory needs of the embedding application as 
well). For example, using the default page size of 4K, a page cache size of 
2000 pages will require at least 8 MB of memory (and probably more, given the 
overhead).

For a simple application (no GUI), using the Sun 1.1.7 JVM on Windows NT and 
using the -mx96m option (which allows 96 MB for the Java heap), it is possible 
to have a page cache size of 10,000 pages (approximately 40 MB).

The minimum value is 40 pages. If you specify a lower value, Derby uses the 
default value.

John Godschall
Release Engineer | First Marblehead
781.658.5028tel:781.658.5028 (direct)
617.513.1912tel:617.513.1912 (mobile)
jgodsch...@fmd.commailto:jgodsch...@fmd.com
*This e-mail and any attachments may contain content protected
under federal law and is also confidential and proprietary in nature.
If you received this message in error, please notify the sender
immediately and delete the original and destroy all copies of the
message and any attachments. Any other use of this e-mail by you
including retaining, using, copying, distributing, or otherwise
disclosing this information in any manner is prohibited.

*This e-mail and any attachments may contain content protected 
under federal law and is also confidential and proprietary in nature. 
If you received this message in error, please notify the sender 
immediately and delete the original and destroy all copies of the 
message and any attachments. Any other use of this e-mail by you 
including retaining, using, copying, distributing, or otherwise 
disclosing this information in any manner is prohibited.



Re: Opinions on new security feature requested

2011-09-02 Thread Peter Ondruška
+1 for more restrictive permissions. Actually when I run Derby on Unix
it runs under own user+group and database files are not accessible by
others.

On Fri, Sep 2, 2011 at 7:31 PM, Dag H. Wanvik dag.wan...@oracle.com wrote:
 Hi folks,

 we are always working to make Derby more secure; in this day and age,
 security is ever more on people's minds for obvious reasons;
 IT systems are everywhere and the bad guys never tire of finding new holes
 to break them. Up till now, Derby creates database files and logs using the
 default operating system permission in effect. On Unix/linux this is
 controlled by the umask of the process starting the Derby engine, be in
 embedded or a standalone Derby network server. Similarly on Windows, NTFS
 has a security model can be configured to give various default permissions.

 Now, often the defaults will allow other (than the one starting the VM) OS
 users the permissions to read and possibly write to the database files. This
 can be intention to allow several users to boot the same (shared) database),
 or it can be accidental. In DERBY-5363 we have been discussion use cases and
 scenarios for when it would be desirable to let Derby be more secure than
 the default permissions. Other database also do this, e.g. PostGreSQL, MS
 SQL server.

 Typically, only the OS user creating the database would have access (default
 behavior) unless one told the database to be lax and not worry about
 tightening up the default OS permissions.
 Obviously, one can achieve the same restrictive permissions, by setting the
 umask to 0077 on Unix, or tweak the NTFS settings similarly (Windows), but
 this requires some care and presumes that the users remembers to do so (many
 people don't grok the NTFS security model..)

 To be clear, one would be able to enable/disable this extra security by
 providing Derby with a property setting, so the question is really what is
 the msot appropriate default: use lax permissions (rely on the user having
 tightened up be fore starting the VM), or use the new proactive secure
 settings proposed in DERBY-5363.

 Secure default pros:
 - users will get better security by default. If one needs to share the
 database files, one can use a property to get old, lax behavior.
 - no need to change startup scripts to get better security

 Secure default cons:
 - upwards compatibility: if an installation relies on sharing database
 files, on would have to start using a property after upgrading.
 - requires at least Java 6 (on Unix), Java 7 on Windows/NTFS to work (an
 incentive to upgrade, though :-)

 In the discussion it as been suggested that many deployments, especially of
 embedded Derby, rely on several OS users having permissions, so changing the
 default Derby behavior would cause upgrade issues. Probably for most
 client/server deployments, where the server is started from the command
 line, it would be the same OS user starting the Derby server every time. In
 mixed deployments (embedded, but the server is sometimes started via the
 API), the latter may not hold true.

 A possible trade-off between the concerns would thus be to start embedded
 with the exisiting, lax permissions by default, but start the server from
 the command line with a secure (restrictive) default. In both cases, one
 would get the opposite behavior by providing a system property on VM
 startup.

 Before we settle the discussion on this, it would be good to hear what you
 think! Thanks!

 Dag




Re: Opinions on new security feature requested

2011-09-02 Thread Roy . Minet
Unless it is completely unavoidable, it should be possible to install a later 
version AND NOT BREAK AN EXISTING APPLICATION.  To do so is rude and can be 
very disruptive. 

- Original Message -
From: Dag H. Wanvik dag.wan...@oracle.com 
To: Derby Discussion derby-user@db.apache.org 
Sent: Friday, September 2, 2011 1:31:46 PM 
Subject: Opinions on new security feature requested 

Hi folks, 

we are always working to make Derby more secure; in this day and age, 
security is ever more on people's minds for obvious reasons; 
IT systems are everywhere and the bad guys never tire of finding new 
holes to break them. Up till now, Derby creates database files and logs 
using the default operating system permission in effect. On Unix/linux 
this is controlled by the umask of the process starting the Derby 
engine, be in embedded or a standalone Derby network server. Similarly 
on Windows, NTFS has a security model can be configured to give various 
default permissions. 

Now, often the defaults will allow other (than the one starting the VM) 
OS users the permissions to read and possibly write to the database 
files. This can be intention to allow several users to boot the same 
(shared) database), or it can be accidental. In DERBY-5363 we have been 
discussion use cases and scenarios for when it would be desirable to let 
Derby be more secure than the default permissions. Other database also 
do this, e.g. PostGreSQL, MS SQL server. 

Typically, only the OS user creating the database would have access 
(default behavior) unless one told the database to be lax and not worry 
about tightening up the default OS permissions. 
Obviously, one can achieve the same restrictive permissions, by setting 
the umask to 0077 on Unix, or tweak the NTFS settings similarly 
(Windows), but this requires some care and presumes that the users 
remembers to do so (many people don't grok the NTFS security model..) 

To be clear, one would be able to enable/disable this extra security by 
providing Derby with a property setting, so the question is really what 
is the msot appropriate default: use lax permissions (rely on the user 
having tightened up be fore starting the VM), or use the new proactive 
secure settings proposed in DERBY-5363. 

Secure default pros: 
- users will get better security by default. If one needs to share the 
database files, one can use a property to get old, lax behavior. 
- no need to change startup scripts to get better security 

Secure default cons: 
- upwards compatibility: if an installation relies on sharing database 
files, on would have to start using a property after upgrading. 
- requires at least Java 6 (on Unix), Java 7 on Windows/NTFS to work (an 
incentive to upgrade, though :-) 

In the discussion it as been suggested that many deployments, especially 
of embedded Derby, rely on several OS users having permissions, so 
changing the default Derby behavior would cause upgrade issues. Probably 
for most client/server deployments, where the server is started from the 
command line, it would be the same OS user starting the Derby server 
every time. In mixed deployments (embedded, but the server is sometimes 
started via the API), the latter may not hold true. 

A possible trade-off between the concerns would thus be to start 
embedded with the exisiting, lax permissions by default, but start the 
server from the command line with a secure (restrictive) default. In 
both cases, one would get the opposite behavior by providing a system 
property on VM startup. 

Before we settle the discussion on this, it would be good to hear what 
you think! Thanks! 

Dag