J9 + Encrypted Embedded Database
Hi, So far I have an unencrypted embedded database which I can use within an OSGi bundle running inside the Knopflerfish framework. This database was created using the ij tool (using Java 1.4) CONNECT 'jdbc:derby:MYDB;create=true; This all works fine with even when I'm running with J9 (with JSR169.jar). Now I need to encrypt this database, therefore I had to change by database creation script CONNECT 'jdbc:derby:MYDB;create=true; dataEncryption=true; bootPassword=we1come;' The database will be created successfully, but when I try to 'open' the database (setting the appropriate parameters) I get the following errors java.sql.SQLException: Encryption algorithm 'DES/CBC/NoPadding' does not exist. Please check that the chosen provider 'default' supports this algorithm. Using Java 1.4 it will all be fine. Has anyone successfully created and opened an encrypted database using j9? There doesn't seem to be a definitive guide on how to do this. Thanks in advance. -- View this message in context: http://www.nabble.com/J9-%2B-Encrypted-Embedded-Database-tp24107762p24107762.html Sent from the Apache Derby Users mailing list archive at Nabble.com.
Re: J9 + Encrypted Embedded Database
Hi, I have no idea about J9 but I have created encrypted derby databases and I think your missing some parameters. dataEncryption=true; bootPassword=blabla; encryptionProvider=com.sun.crypto.provider.SunJCE; encryptionAlgorithm=DESede/CBC/NoPadding; The above will get you a somewhat decent encrypted DB.. if you consider DES as decent. You'd need to change the privider and algorithm to apply a strong crypt. Hope this helps. -- George H george@gmail.com On Fri, Jun 19, 2009 at 11:53 AM, Cuong cuong@kirona.com wrote: Hi, So far I have an unencrypted embedded database which I can use within an OSGi bundle running inside the Knopflerfish framework. This database was created using the ij tool (using Java 1.4) CONNECT 'jdbc:derby:MYDB;create=true; This all works fine with even when I'm running with J9 (with JSR169.jar). Now I need to encrypt this database, therefore I had to change by database creation script CONNECT 'jdbc:derby:MYDB;create=true; dataEncryption=true; bootPassword=we1come;' The database will be created successfully, but when I try to 'open' the database (setting the appropriate parameters) I get the following errors java.sql.SQLException: Encryption algorithm 'DES/CBC/NoPadding' does not exist. Please check that the chosen provider 'default' supports this algorithm. Using Java 1.4 it will all be fine. Has anyone successfully created and opened an encrypted database using j9? There doesn't seem to be a definitive guide on how to do this. Thanks in advance. -- View this message in context: http://www.nabble.com/J9-%2B-Encrypted-Embedded-Database-tp24107762p24107762.html Sent from the Apache Derby Users mailing list archive at Nabble.com.
Re: J9 + Encrypted Embedded Database
Thanks George, but I already tried something similar to this. In this instance I get the following error Caused by: java.sql.SQLException: Encryption algorithm 'DESede/CBC/NoPadding' does not exist But it will work if using Java1.4 I think the problem is because j9 does not include the classes to decrypt the database. J9 does have it's own enryption provider (com.ibm.j9.jce.provider.J9JCEProvider) but I'm not sure how to use it. I tried various things to try and create the database specifying this provider by to no avail. George H wrote: Hi, I have no idea about J9 but I have created encrypted derby databases and I think your missing some parameters. dataEncryption=true; bootPassword=blabla; encryptionProvider=com.sun.crypto.provider.SunJCE; encryptionAlgorithm=DESede/CBC/NoPadding; The above will get you a somewhat decent encrypted DB.. if you consider DES as decent. You'd need to change the privider and algorithm to apply a strong crypt. Hope this helps. -- George H george@gmail.com On Fri, Jun 19, 2009 at 11:53 AM, Cuong cuong@kirona.com wrote: Hi, So far I have an unencrypted embedded database which I can use within an OSGi bundle running inside the Knopflerfish framework. This database was created using the ij tool (using Java 1.4) CONNECT 'jdbc:derby:MYDB;create=true; This all works fine with even when I'm running with J9 (with JSR169.jar). Now I need to encrypt this database, therefore I had to change by database creation script CONNECT 'jdbc:derby:MYDB;create=true; dataEncryption=true; bootPassword=we1come;' The database will be created successfully, but when I try to 'open' the database (setting the appropriate parameters) I get the following errors java.sql.SQLException: Encryption algorithm 'DES/CBC/NoPadding' does not exist. Please check that the chosen provider 'default' supports this algorithm. Using Java 1.4 it will all be fine. Has anyone successfully created and opened an encrypted database using j9? There doesn't seem to be a definitive guide on how to do this. Thanks in advance. -- View this message in context: http://www.nabble.com/J9-%2B-Encrypted-Embedded-Database-tp24107762p24107762.html Sent from the Apache Derby Users mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/J9-%2B-Encrypted-Embedded-Database-tp24107762p24108092.html Sent from the Apache Derby Users mailing list archive at Nabble.com.
Re: J9 + Encrypted Embedded Database
hmm I see. I think the default algorithm is AES/CBC/NoPadding also I think this thread might help you. http://www.mail-archive.com/derby-user@db.apache.org/msg04564.html and perhaps this -- George H george@gmail.com On Fri, Jun 19, 2009 at 12:19 PM, Cuong cuong@kirona.com wrote: Thanks George, but I already tried something similar to this. In this instance I get the following error Caused by: java.sql.SQLException: Encryption algorithm 'DESede/CBC/NoPadding' does not exist But it will work if using Java1.4 I think the problem is because j9 does not include the classes to decrypt the database. J9 does have it's own enryption provider (com.ibm.j9.jce.provider.J9JCEProvider) but I'm not sure how to use it. I tried various things to try and create the database specifying this provider by to no avail. George H wrote: Hi, I have no idea about J9 but I have created encrypted derby databases and I think your missing some parameters. dataEncryption=true; bootPassword=blabla; encryptionProvider=com.sun.crypto.provider.SunJCE; encryptionAlgorithm=DESede/CBC/NoPadding; The above will get you a somewhat decent encrypted DB.. if you consider DES as decent. You'd need to change the privider and algorithm to apply a strong crypt. Hope this helps. -- George H george@gmail.com On Fri, Jun 19, 2009 at 11:53 AM, Cuong cuong@kirona.com wrote: Hi, So far I have an unencrypted embedded database which I can use within an OSGi bundle running inside the Knopflerfish framework. This database was created using the ij tool (using Java 1.4) CONNECT 'jdbc:derby:MYDB;create=true; This all works fine with even when I'm running with J9 (with JSR169.jar). Now I need to encrypt this database, therefore I had to change by database creation script CONNECT 'jdbc:derby:MYDB;create=true; dataEncryption=true; bootPassword=we1come;' The database will be created successfully, but when I try to 'open' the database (setting the appropriate parameters) I get the following errors java.sql.SQLException: Encryption algorithm 'DES/CBC/NoPadding' does not exist. Please check that the chosen provider 'default' supports this algorithm. Using Java 1.4 it will all be fine. Has anyone successfully created and opened an encrypted database using j9? There doesn't seem to be a definitive guide on how to do this. Thanks in advance. -- View this message in context: http://www.nabble.com/J9-%2B-Encrypted-Embedded-Database-tp24107762p24107762.html Sent from the Apache Derby Users mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/J9-%2B-Encrypted-Embedded-Database-tp24107762p24108092.html Sent from the Apache Derby Users mailing list archive at Nabble.com.
Re: J9 + Encrypted Embedded Database
Already seen this thread and this person seems to be having the same problem as me. But I can't seem to find an answer anywhere. It must be possible though -- View this message in context: http://www.nabble.com/J9-%2B-Encrypted-Embedded-Database-tp24107762p24108396.html Sent from the Apache Derby Users mailing list archive at Nabble.com.
Call for Participation: OpenSQL Camp at FrOSCon (St. Augustin, Germany, 22+23 August 2009)
Hi there, OpenSQL Camp is a free conference of, by, and for the open-source database community of users and developers. The OpenSQLCamp 2009, European Edition (http://opensqlcamp.org/) will take part in parallel to the Free and Open Source Conference 2009 (http://froscon.org/) on Saturday 22nd and Sunday 23rd August in St. Augustin, Germany, which is located close to Bonn and Cologne. The goal of this event is to spread the word about the vibrant communities and large ecosystems that exist around Open Source Databases and to educate the attendees about possible alternatives to commercial databases. It is a place where people come to learn, to participate and to contribute. We would like to invite your project to participate in this event. We've set up a call for participation (http://opensqlcamp.org/Events/2009/Call_for_Participation) - the deadline for submitting your proposal is *July 19th*. We are seeking talks related to Open Source Databases of all kind, not just relational databases! Submission about tools and technologies related to OSS databases (e.g. connectors/APIs) are also welcome. Some ideas and for submissions: * An introduction/overview about a certain database project/product or related tool * Providing best practices information for administrators * A deeply technical and developer-centric session about some project's internals or an API used to connect to a database We look forward to your contribution! Please don't hesitate to contact us via IRC (#opensqlcamp on FreeNode) or our Discussion Group (http://groups.google.com/group/opensqlcamp) Thanks! Bye, LenZ -- Lenz Grimmer l...@grimmer.com - http://www.lenzg.net/ signature.asc Description: OpenPGP digital signature
Re: Not able to create encrypted database
Hi, I know this thread is a while old now, but I seem to be having the same issues. Can you tell me if you ever resolved this problem and whether you have managed to get an encrypted derby db working for J9? -- View this message in context: http://www.nabble.com/Not-able-to-create-encrypted-database-tp5201768p24110750.html Sent from the Apache Derby Users mailing list archive at Nabble.com.
Re: Not able to create encrypted database
Cuong wrote: Hi, I know this thread is a while old now, but I seem to be having the same issues. Can you tell me if you ever resolved this problem and whether you have managed to get an encrypted derby db working for J9? There is a comment in our tests: // Encryption only supported for Derby in J2SE/J2EE environments. // J2ME (JSR169) does not support encryption. and then the encryption tests are skipped. My theory always is that if it is not tested, it doesn't exist. I don't immediately see this (or any of the other J2ME restrictions) documented in a comprehensive way. Perhaps I am just missing something. Maybe a description of the restrictions should be listed under http://db.apache.org/derby/docs/dev/devguide/cdevstart18978.html Do we need a new issue to list the restrictions or am I just missing something? If you think that J2ME/J9 has enough encryption support that we could get this to work with some development effort please file an improvement request in Jira. http://db.apache.org/derby/DerbyBugGuidelines.html Then if you would like to join the development community to implement it, I am sure you will find lots of support there in doing so, providing there is not some insurmountable technical obstacle. Kathey
Re: Not able to create encrypted database
Kathey Marsden wrote: Cuong wrote: Hi, I know this thread is a while old now, but I seem to be having the same issues. Can you tell me if you ever resolved this problem and whether you have managed to get an encrypted derby db working for J9? There is a comment in our tests: // Encryption only supported for Derby in J2SE/J2EE environments. // J2ME (JSR169) does not support encryption. and then the encryption tests are skipped. My theory always is that if it is not tested, it doesn't exist. I don't immediately see this (or any of the other J2ME restrictions) documented in a comprehensive way. Perhaps I am just missing something. Maybe a description of the restrictions should be listed under http://db.apache.org/derby/docs/dev/devguide/cdevstart18978.html Do we need a new issue to list the restrictions or am I just missing something? Kim recently added some useful documentation on the limitations of Derby's support for small devices: http://db.apache.org/derby/docs/10.5/ref/ref-single.html#rrefjdbcjsr169 As you can see, we do not claim to support encrypted databases on small devices. However, the limitation isn't in Derby. It's just that the encryption libraries aren't a standard part of the CDC/FP 1.1 platform. Most of the small device limitations listed in the Reference Guide can be lifted by loading more libraries onto your small device. From this thread it seems that, by wiring encryption libraries into the classpath, some people have gotten encryption to work on Derby databases on small devices. Of course, as you note, we don't test these combinations so there may be some gotchas out there. Regards, -Rick If you think that J2ME/J9 has enough encryption support that we could get this to work with some development effort please file an improvement request in Jira. http://db.apache.org/derby/DerbyBugGuidelines.html Then if you would like to join the development community to implement it, I am sure you will find lots of support there in doing so, providing there is not some insurmountable technical obstacle. Kathey
Re: Hibernate + Derby - Unable to determine stream header for hibernate type 'text'
kashyup wrote: Hello all, We have just started to look into Derby as our embedded DB for a desktop app version. Env: Hibernate 3.2 and Derby 10.5.1 I have defined a java string field of type 'text' in Hibernate Mapping (works great on Postgres 8.1 and Oracle 10g) I use Hibernate SessionFactory to generate Table schema, which is done without any error. But, when I try to save records in tables via Hibernate Session, I get the following exception: Note: If I change hibernate mapping to type 'string', then I dont get this error: java.lang.IllegalStateException: No context, unable to determine which stream header format to generate at org.apache.derby.iapi.types.ClobStreamHeaderGenerator.determineMode(Unknown Source) at org.apache.derby.iapi.types.ClobStreamHeaderGenerator.expectsCharCount(Unknown Source) at org.apache.derby.iapi.types.ReaderToUTF8Stream.fillBuffer(Unknown Source) at org.apache.derby.iapi.types.ReaderToUTF8Stream.read(Unknown Source) at java.io.DataInputStream.read(Unknown Source) at org.apache.derby.iapi.types.SQLClob.readExternal(Unknown Source) at org.apache.derby.iapi.types.SQLChar.getString(Unknown Source) at org.apache.derby.iapi.types.SQLClob.getClone(Unknown Source) at org.apache.derby.impl.sql.GenericParameter.getClone(Unknown Source) at org.apache.derby.impl.sql.GenericParameterValueSet.init(Unknown Source) at org.apache.derby.impl.sql.GenericParameterValueSet.getClone(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.addBatch(Unknown Source) at org.hibernate.jdbc.BatchingBatcher.addToBatch(BatchingBatcher.java:31) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2252) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2665) Also, I need to save it as type 'text' even though it gets saved as CLOB(255), as the CLOB column size can later be altered. What could be the cause? or changes needed? Hello, You have discovered / triggered a bug in the new Clob handling code introduced in 10.5.1. I logged DERBY-4278 [1], and have started working on a fix (the regression tests are running, patch needs review). Regarding workarounds, I'm afraid all I can think of is using the client driver instead of the embedded driver, avoid batching for inserts with Clobs, change the data type, or downgrade to 10.4. Another option is to build Derby with the patch and test it out :) I observe that you are using very small Clobs (255 characters). Clobs are in general less effective than using for instance VARCHAR. Is your concern about altering the column size related to Derby, Hibernate or something else? Regards, -- Kristian [1] https://issues.apache.org/jira/browse/DERBY-4278 thanks all
Re: Not able to create encrypted database
Rick Hillegas wrote: Kim recently added some useful documentation on the limitations of Derby's support for small devices: http://db.apache.org/derby/docs/10.5/ref/ref-single.html#rrefjdbcjsr169 As you can see, we do not claim to support encrypted databases on small devices. Thanks Rick for pointing this out and Kim for adding the information. It is good to see we do have this documented. From this thread it seems that, by wiring encryption libraries into the classpath, some people have gotten encryption to work on Derby databases on small devices. Of course, as you note, we don't test these combinations so there may be some gotchas out there. Mostly what I have seen on this thread is people that *haven't* gotten encryption to work on small devices, but if anyone does get it working it would be great to outline how it is done and create an issue to add a test so it keeps on working. Kathey
Re: Hibernate + Derby - Unable to determine stream header for hibernate type 'text'
Dear Kristian, Regarding your remark about effectiveness of very small CLOB vs VARCHAR should I just use VARCHAR to store 100 to 2000 character data instead of CLOB? I do not want to index this column and storage size and access speed (read only) are important. Thanks, Peter 2009/6/19, Kristian Waagan kristian.waa...@sun.com: kashyup wrote: Hello all, We have just started to look into Derby as our embedded DB for a desktop app version. Env: Hibernate 3.2 and Derby 10.5.1 I have defined a java string field of type 'text' in Hibernate Mapping (works great on Postgres 8.1 and Oracle 10g) I use Hibernate SessionFactory to generate Table schema, which is done without any error. But, when I try to save records in tables via Hibernate Session, I get the following exception: Note: If I change hibernate mapping to type 'string', then I dont get this error: java.lang.IllegalStateException: No context, unable to determine which stream header format to generate at org.apache.derby.iapi.types.ClobStreamHeaderGenerator.determineMode(Unknown Source) at org.apache.derby.iapi.types.ClobStreamHeaderGenerator.expectsCharCount(Unknown Source) at org.apache.derby.iapi.types.ReaderToUTF8Stream.fillBuffer(Unknown Source) at org.apache.derby.iapi.types.ReaderToUTF8Stream.read(Unknown Source) at java.io.DataInputStream.read(Unknown Source) at org.apache.derby.iapi.types.SQLClob.readExternal(Unknown Source) at org.apache.derby.iapi.types.SQLChar.getString(Unknown Source) at org.apache.derby.iapi.types.SQLClob.getClone(Unknown Source) at org.apache.derby.impl.sql.GenericParameter.getClone(Unknown Source) at org.apache.derby.impl.sql.GenericParameterValueSet.init(Unknown Source) at org.apache.derby.impl.sql.GenericParameterValueSet.getClone(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.addBatch(Unknown Source) at org.hibernate.jdbc.BatchingBatcher.addToBatch(BatchingBatcher.java:31) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2252) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2665) Also, I need to save it as type 'text' even though it gets saved as CLOB(255), as the CLOB column size can later be altered. What could be the cause? or changes needed? Hello, You have discovered / triggered a bug in the new Clob handling code introduced in 10.5.1. I logged DERBY-4278 [1], and have started working on a fix (the regression tests are running, patch needs review). Regarding workarounds, I'm afraid all I can think of is using the client driver instead of the embedded driver, avoid batching for inserts with Clobs, change the data type, or downgrade to 10.4. Another option is to build Derby with the patch and test it out :) I observe that you are using very small Clobs (255 characters). Clobs are in general less effective than using for instance VARCHAR. Is your concern about altering the column size related to Derby, Hibernate or something else? Regards, -- Kristian [1] https://issues.apache.org/jira/browse/DERBY-4278 thanks all
Re: Not able to create encrypted database
Kathey Marsden wrote: Rick Hillegas wrote: Kim recently added some useful documentation on the limitations of Derby's support for small devices: http://db.apache.org/derby/docs/10.5/ref/ref-single.html#rrefjdbcjsr169 As you can see, we do not claim to support encrypted databases on small devices. Thanks Rick for pointing this out and Kim for adding the information. It is good to see we do have this documented. From this thread it seems that, by wiring encryption libraries into the classpath, some people have gotten encryption to work on Derby databases on small devices. Of course, as you note, we don't test these combinations so there may be some gotchas out there. Mostly what I have seen on this thread is people that *haven't* gotten encryption to work on small devices, but if anyone does get it working it would be great to outline how it is done and create an issue to add a test so it keeps on working. You are, of course, right. As I go back through the thread, I don't see any success stories. It might be worth experimenting with a different encryption package, like this old package from the 10.3 days: http://java.sun.com/products/archive/jce/ Kathey
Re: Not able to create encrypted database
Rick Hillegas wrote: Kathey Marsden wrote: Rick Hillegas wrote: Kim recently added some useful documentation on the limitations of Derby's support for small devices: http://db.apache.org/derby/docs/10.5/ref/ref-single.html#rrefjdbcjsr169 As you can see, we do not claim to support encrypted databases on small devices. Thanks Rick for pointing this out and Kim for adding the information. It is good to see we do have this documented. From this thread it seems that, by wiring encryption libraries into the classpath, some people have gotten encryption to work on Derby databases on small devices. Of course, as you note, we don't test these combinations so there may be some gotchas out there. Mostly what I have seen on this thread is people that *haven't* gotten encryption to work on small devices, but if anyone does get it working it would be great to outline how it is done and create an issue to add a test so it keeps on working. You are, of course, right. As I go back through the thread, I don't see any success stories. It might be worth experimenting with a different encryption package, like this old package from the 10.3 days: http://java.sun.com/products/archive/jce/ Sigh. Make that the jdk1.3 days. Kathey
Re: Not able to create encrypted database
Rick Hillegas wrote: You are, of course, right. As I go back through the thread, I don't see any success stories. It might be worth experimenting with a different encryption package, like this old package from the JDK 1.3 days: http://java.sun.com/products/archive/jce/ That might be a hopeless task, standard Java methods and classes have been removed from CDC/Foundation, there is probably a significant chance that the JDK 1.3 security features depend on those classes. From a quick look at the JME web-site, it seems that Foundation 1.1 supports crypto, the best approach would be to make the changes in Derby to support that environment, it may be mainly build changes to allow Derby to load its crypto support in an CDC/Foundation environment. http://java.sun.com/javame/reference/apis.jsp http://java.sun.com/javame/reference/apis/jsr219/ Dan.
