[ANNOUNCE] Apache Derby 10.17.1.0 released

The Apache Derby project is pleased to announce feature release 10.17.1.0. Apache Derby is a sub-project of the Apache DB project. Derby is a pure Java relational database engine which conforms to the ISO/ANSI SQL and JDBC standards. Derby aims to be easy for developers and end-users to work

new hurdle for applications which programatically install a SecurityManager

Build 18-ea+23-1525 has introduced another hurdle for applications which use the SecurityManager. In order to install a SecurityManager, you now have to set -Djava.security.manager=allow on the boot command line. This property cannot be set programatically, unlike the other system properties

Re: Accessing transitional tables from trigger procedures

Trigger transition tables can only be used for row-scoped triggers (FOR EACH ROW triggers). They cannot be used for statement-scoped triggers (FOR EACH STATEMENT triggers). I suppose that you could create a row-scoped trigger which populates a scratch table from the values of the transition table.

Re: -jar option and the modulepath

Thanks, Alan. On 11/23/18 12:06 AM, Alan Bateman wrote: On 22/11/2018 19:27, Richard Hillegas wrote: Can I scribble something in a jar file manifest which will cause "java -jar" to boot with a modulepath rather than a classpath? I do not see any support for a modulepath attribute i

-jar option and the modulepath

Can I scribble something in a jar file manifest which will cause "java -jar" to boot with a modulepath rather than a classpath? I do not see any support for a modulepath attribute in the Java 9 jar file documentation at https://docs.oracle.com/javase/9/docs/specs/jar/jar.html. My sense is

Re: speed of class loading via a modulepath

Thanks, Rémi and Alan. On 11/19/18 12:53 AM, Alan Bateman wrote: On 18/11/2018 20:00, Richard Hillegas wrote: I am updating Apache Derby documentation to reflect the recent modularization of the codeline. While doing this, I have stumbled across an old piece of advice from the Derby Tuning

speed of class loading via a modulepath

I am updating Apache Derby documentation to reflect the recent modularization of the codeline. While doing this, I have stumbled across an old piece of advice from the Derby Tuning Guide: "The structure of your classpath can affect Derby startup time and the time required to load a particular

Re: generated code and jigsaw modules

Thanks, Alan. This is very helpful. On 10/10/18 9:53 AM, Alan Bateman wrote: On 10/10/2018 16:37, Richard Hillegas wrote: : java.lang.invoke.MethodHandles.lookup().defineClass(generatedClassBytes) This approach does indeed put the generated class where I want it: inside the Derby engine

Re: generated code and jigsaw modules

to get over these speed bumps. Thanks, -Rick On 10/4/18 9:10 AM, Richard Hillegas wrote: I am looking for advice about how to tighten up module encapsulation while generating byte code on the fly. I ask this question on behalf of Apache Derby, a pure-Java relational database whose original code

Re: generated code and jigsaw modules

On 10/4/18 9:45 AM, Alan Bateman wrote: On 04/10/2018 17:10, Richard Hillegas wrote: I am looking for advice about how to tighten up module encapsulation while generating byte code on the fly. I ask this question on behalf of Apache Derby, a pure-Java relational database whose original code

Re: generated code and jigsaw modules

On 10/4/18 9:26 AM, Remi Forax wrote: - Mail original - De: "Richard Hillegas" À: "core-libs-dev" Envoyé: Jeudi 4 Octobre 2018 18:10:13 Objet: generated code and jigsaw modules I am looking for advice about how to tighten up module encapsulation while generating

generated code and jigsaw modules

I am looking for advice about how to tighten up module encapsulation while generating byte code on the fly. I ask this question on behalf of Apache Derby, a pure-Java relational database whose original code dates back to Java 1.2. I want to reduce Derby's attack-surface when running with a

Re: handling the deprecations introduced by early access builds 116 and 118 of jdk 9

Thanks for that response, Stuart. One comment inline... On 5/31/16 5:34 PM, Stuart Marks wrote: On 5/30/16 11:48 AM, Richard Hillegas wrote: Dalibor Topic recommended that I post this feedback on core-libs-dev. This is my feedback after ameliorating the deprecation warnings which surfaced

handling the deprecations introduced by early access builds 116 and 118 of jdk 9

Dalibor Topic recommended that I post this feedback on core-libs-dev. This is my feedback after ameliorating the deprecation warnings which surfaced when I compiled and tested Apache Derby with early access builds 116 and 118 of JDK 9. Derby is a pure Java relational database whose original

Re: Spark sql jdbc fails for Oracle NUMBER type columns

:22 PM: > From: Madabhattula Rajesh Kumar <mrajaf...@gmail.com> > To: Richard Hillegas/San Francisco/IBM@IBMUS > Cc: "u...@spark.incubator.apache.org" > <u...@spark.incubator.apache.org>, "user@spark.apache.org" > <user@spark.apache.org> > Date: 11/0

Re: Spark sql jdbc fails for Oracle NUMBER type columns

Or you may be referring to https://issues.apache.org/jira/browse/SPARK-10648. That issue has a couple pull requests but I think that the limited bandwidth of the committers still applies. Thanks, Rick Richard Hillegas/San Francisco/IBM@IBMUS wrote on 11/05/2015 09:16:42 AM: > From: Rich

Re: Spark sql jdbc fails for Oracle NUMBER type columns

Hi Rajesh, I think that you may be referring to https://issues.apache.org/jira/browse/SPARK-10909. A pull request on that issue was submitted more than a month ago but it has not been committed. I think that the committers are busy working on issues which were targeted for 1.6 and I doubt that

Re: Spark scala REPL - Unable to create sqlContext

Note that embedded Derby supports multiple, simultaneous connections, that is, multiple simultaneous users. But a Derby database is owned by the process which boots it. Only one process can boot a Derby database at a given time. The creation of multiple SQL contexts must be spawning multiple

Re: Can not subscript to mailing list

Hi Jeff, Hard to say what's going on. I have had problems subscribing to the Apache lists in the past. My problems, which may be different than yours, were caused by replying to the confirmation request from a different email account than the account I was trying to subscribe from. It was easy

Re: Spark SQL: Preserving Dataframe Schema

As an academic aside, note that all datatypes are nullable according to the SQL Standard. NOT NULL is modelled in the Standard as a constraint on data values, not as a parallel universe of special data types. However, very few databases implement NOT NULL via integrity constraints. Instead,

Re: Network-related environemental problem when running JDBCSuite

-getlocalhost-throws-unknownhostexception Thanks, -Rick Richard Hillegas/San Francisco/IBM@IBMUS wrote on 10/15/2015 11:15:29 AM: > From: Richard Hillegas/San Francisco/IBM@IBMUS > To: Dev <dev@spark.apache.org> > Date: 10/15/2015 11:16 AM > Subject: Re: Network-related environem

Network-related environemental problem when running JDBCSuite

I am seeing what look like environmental errors when I try to run a test on a clean local branch which has been sync'd to the head of the development trunk. I would appreciate advice about how to debug or hack around this problem. For the record, the test ran cleanly last week. This is the

Re: SQL Context error in 1.5.1 - any work around ?

A crude workaround may be to run your spark shell with a sudo command. Hope this helps, Rick Hillegas Sourav Mazumder wrote on 10/15/2015 09:59:02 AM: > From: Sourav Mazumder > To: user > Date: 10/15/2015 09:59

Re: Network-related environemental problem when running JDBCSuite

-with-two-workers export SPARK_LOCAL_IP=127.0.0.1 Then I got errors related to booting the metastore_db. So I deleted that directory. After that I was able to run spark-shell again. Now let's see if this hack fixes the tests... Thanks, Rick Hillegas Richard Hillegas/San Francisco/IBM@IBMUS

Re: Network-related environemental problem when running JDBCSuite

.scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) :10: error: not found: value sqlContext import sqlContext.implicits._ ^ :10: error: not found: value sqlContext import sqlContext.sql Thanks, Rick Hillegas Richard Hillegas/San Francisco

Re: pagination spark sq

Hi Ravi, If you build Spark with Hive support, then your sqlContext variable will be an instance of HiveContext and you will enjoy the full capabilities of the Hive query language rather than the more limited capabilities of Spark SQL. However, even Hive QL does not support the OFFSET clause, at

draft release announcement for 10.12.1.1

Here is a first draft of a release announcement for 10.12.1.1. Please let me know how I can improve this. Thanks, -Rick --- The Apache Derby project is pleased to announce feature release 10.12.1.1. Apache Derby is a subproject of the Apache DB project. Derby is a pure Java

Re: This post has NOT been accepted by the mailing list yet.

Hi Akhandeshi, It may be that you are not seeing your own posts because you are sending from a gmail account. See for instance https://support.google.com/a/answer/1703601?hl=en Hope this helps, Rick Hillegas STSM, IBM Analytics, Platform - IBM USA akhandeshi wrote on

[RESULT] [VOTE] 10.12.1.1 release

Thanks for everyone's work on coding, documenting, and testing 10.12.1.1. The polls have closed. The community has approved 10.12.1.1 as an official Derby release: +1: Bryan Pendleton (pmc) Kim Haase (pmc) Rick Hillegas (pmc) Kathey Marsden (pmc) Myrna van Lunteren (pmc) No other votes were

Re: save DF to JDBC

Hi Ruslan, Here is some sample code which writes a DataFrame to a table in a Derby database: import org.apache.spark.sql._ import org.apache.spark.sql.types._ val binaryVal = Array[Byte] ( 1, 2, 3, 4 ) val timestampVal = java.sql.Timestamp.valueOf("1996-01-01 03:30:36") val dateVal =

Re: [VOTE] 10.12.1.1 release

On 9/20/15 10:14 AM, Richard Hillegas wrote: Please test-drive the 10.12.1.1 candidate, then vote on whether to accept it as a Derby release. The candidate lives at: http://people.apache.org/~rhillegas/10.12.1.1/ The polls close at 5:00 pm San Francisco time on Monday, October 5. 10.12.1.1

Re: unsubscribe

Hi Sukesh, To unsubscribe from the dev list, please send a message to dev-unsubscr...@spark.apache.org. To unsubscribe from the user list, please send a message user-unsubscr...@spark.apache.org. Please see: http://spark.apache.org/community.html#mailing-lists. Thanks, -Rick sukesh kumar

Re: unsubscribe

Hi Sukesh, To unsubscribe from the dev list, please send a message to dev-unsubscr...@spark.apache.org. To unsubscribe from the user list, please send a message user-unsubscr...@spark.apache.org. Please see: http://spark.apache.org/community.html#mailing-lists. Thanks, -Rick sukesh kumar

Re: Derby version used by Hive

tarting point for implementing a portable SQL layer. Thanks, -Rick > > On Mon, Sep 28, 2015 at 11:02 AM, Richard Hillegas <rhil...@us.ibm.com> > wrote: > > > > > > > I haven't received a response to the following message, which I posted last > > week. Maybe m

Re: [Discuss] NOTICE file for transitive "NOTICE"s

Thanks, Sean! Sean Owen <so...@cloudera.com> wrote on 09/25/2015 06:35:46 AM: > From: Sean Owen <so...@cloudera.com> > To: Reynold Xin <r...@databricks.com>, Richard Hillegas/San > Francisco/IBM@IBMUS > Cc: "dev@spark.apache.org" <dev@spark.apache.org

re: Derby version used by Hive

I haven't received a response to the following message, which I posted last week. Maybe my message rambled too much. Here is an attempt to pose my question more succinctly: Q: Does anyone know of any reason why we can't upgrade Hive's Derby version to 10.12.1.1, the new version being vetted by

Re: [Discuss] NOTICE file for transitive "NOTICE"s

NY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE." Thanks, -Rick Reynold Xin <r...@databricks.com> wrote on 09/24/2015 10:55:53 AM: > From: Reynold Xin <r...@databricks.com> > To: Sean Owen <so...@cloudera.com> > Cc: Rich

Re: [Discuss] NOTICE file for transitive "NOTICE"s

-howto.html#permissive-deps Thanks, -Rick Sean Owen <so...@cloudera.com> wrote on 09/24/2015 12:07:01 PM: > From: Sean Owen <so...@cloudera.com> > To: Richard Hillegas/San Francisco/IBM@IBMUS > Cc: "dev@spark.apache.org" <dev@spark.apache.org> > Date: 09/24/2015

Re: [VOTE] Release Apache Spark 1.5.1 (RC1)

ll <pwend...@gmail.com> > To: Sean Owen <so...@cloudera.com> > Cc: Richard Hillegas/San Francisco/IBM@IBMUS, "dev@spark.apache.org" > <dev@spark.apache.org> > Date: 09/24/2015 10:24 AM > Subject: Re: [VOTE] Release Apache Spark 1.5.1 (RC1) > > Hey Richard

Re: [VOTE] Release Apache Spark 1.5.1 (RC1)

-1 (non-binding) I was able to build Spark cleanly from the source distribution using the command in README.md: build/mvn -DskipTests clean package However, while I was waiting for the build to complete, I started going through the NOTICE file. I was confused about where to find licenses

Re: unsubscribe

Hi Ntale, To unsubscribe from the user list, please send a message to user-unsubscr...@spark.apache.org as described here: http://spark.apache.org/community.html#mailing-lists. Thanks, -Rick Ntale Lukama wrote on 09/23/2015 04:34:48 AM: > From: Ntale Lukama

Derby version used by Hive

I am following the Hive build instructions here: https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-InstallationandConfiguration . I noticed that Hive development seems to be using an old version of Derby: 10.10.2.0. Is there some defect in the most recent Derby

column identifiers in Spark SQL

I am puzzled by the behavior of column identifiers in Spark SQL. I don't find any guidance in the "Spark SQL and DataFrame Guide" at http://spark.apache.org/docs/latest/sql-programming-guide.html. I am seeing odd behavior related to case-sensitivity and to delimited (quoted) identifiers.

Derby version in Spark

I see that lib_managed/jars holds these old Derby versions: lib_managed/jars/derby-10.10.1.1.jar lib_managed/jars/derby-10.10.2.0.jar The Derby 10.10 release family supports some ancient JVMs: Java SE 5 and Java ME CDC/Foundation Profile 1.1. It's hard to imagine anyone running Spark on

Re: column identifiers in Spark SQL

Thanks, -Rick Michael Armbrust <mich...@databricks.com> wrote on 09/22/2015 10:58:36 AM: > From: Michael Armbrust <mich...@databricks.com> > To: Richard Hillegas/San Francisco/IBM@IBMUS > Cc: Dev <dev@spark.apache.org> > Date: 09/22/2015 10:59 AM > Subject: Re: c

Re: Derby version in Spark

Thanks, Ted. I'll follow up with the Hive folks. Cheers, -Rick Ted Yu <yuzhih...@gmail.com> wrote on 09/22/2015 03:41:12 PM: > From: Ted Yu <yuzhih...@gmail.com> > To: Richard Hillegas/San Francisco/IBM@IBMUS > Cc: Dev <dev@spark.apache.org> > Date: 09/22/2015

Re: column identifiers in Spark SQL

ysisException: cannot resolve 'c\"d' given input columns A, b, c"d; line 1 pos 7 sqlContext.sql("""select `c\"d` from test_data""").show Thanks, -Rick Michael Armbrust <mich...@databricks.com> wrote on 09/22/2015 01:16:12 PM: > From: Michael Armb

Re: Derby version in Spark

jersey-guice-1.9.jar parquet-encoding-1.7.0.jar Ted Yu <yuzhih...@gmail.com> wrote on 09/22/2015 01:32:39 PM: > From: Ted Yu <yuzhih...@gmail.com> > To: Richard Hillegas/San Francisco/IBM@IBMUS > Cc: Dev <dev@spark.apache.org> > Date: 09/22/2015 01:33 P

Re: Count for select not matching count for group by

For what it's worth, I get the expected result that "filter" behaves like "group by" when I run the same experiment against a DataFrame which was loaded from a relational store: import org.apache.spark.sql._ import org.apache.spark.sql.types._ val df = sqlContext.read.format("jdbc").options(

Re: Unsubscribe

To unsubscribe from the dev list, please send a message to dev-unsubscr...@spark.apache.org as described here: http://spark.apache.org/community.html#mailing-lists. Thanks, -Rick Dulaj Viduranga wrote on 09/21/2015 10:15:58 AM: > From: Dulaj Viduranga

[VOTE] 10.12.1.1 release

Please test-drive the 10.12.1.1 candidate, then vote on whether to accept it as a Derby release. The candidate lives at: http://people.apache.org/~rhillegas/10.12.1.1/ The polls close at 5:00 pm San Francisco time on Monday, October 5. 10.12.1.1 is a feature release, described in greater

Re: unsubscribe

To unsubscribe from the user list, please send a message to user-unsubscr...@spark.apache.org as described here: http://spark.apache.org/community.html#mailing-lists. Thanks, -Rick

[VOTE] 10.12.1.0 release

Please test-drive the 10.12.1.0 candidate, then vote on whether to accept it as a Derby release. The candidate lives at: http://people.apache.org/~rhillegas/10.12.1.0/ The polls close at 5:00 pm San Francisco time on Monday, October 5. 10.12.1.0 is a feature release, described in greater

Re: [VOTE] 10.12.1.0 release

://wiki.apache.org/db-derby/TenTwelveOnePlatformTesting I plan to fix the defects which we identify and then generate a new release candidate next weekend. Thanks, -Rick On 9/14/15 6:58 AM, Richard Hillegas wrote: Please test-drive the 10.12.1.0 candidate, then vote on whether to accept it as a Derby

[RESULT] [VOTE] Sunsetting support for Java 7

The polls have closed. The developer community has approved the proposal to sunset support for Java 7. +1 votes: Rick Hillegas (PMC) Bryan Pendleton (PMC) Myrna van Lunteren (PMC) Gary Gregory Mike Matrigali (PMC) Kim Haase (PMC) Kristian Waagan (PMC) Knut Anders Hatlen (PMC) No other votes

need a build job for the new 10.12 branch

I've just created the 10.12 branch. Could someone create a build job for the new branch? This is step 2 of the branch creation instructions: http://wiki.apache.org/db-derby/CreatingDerbyBranch Thanks, -Rick

Re: Is there any Spark SQL reference manual?

The latest Derby SQL Reference manual (version 10.11) can be found here: https://db.apache.org/derby/docs/10.11/ref/index.html. It is, indeed, very useful to have a comprehensive reference guide. The Derby build scripts can also produce a BNF description of the grammar--but that is not part of

[VOTE] Sunsetting support for Java 7

Please vote on the following proposed policy for supported platforms. The polls close at 5:00 pm San Francisco time on Saturday September 12. A) The 10.12 release notes will tell users that 10.12 is the last release which supports Java 7. Note that we have already agreed to sunset support for

Re: [VOTE] Sunsetting support for Java 7

+1 On 9/5/15 8:02 AM, Richard Hillegas wrote: Please vote on the following proposed policy for supported platforms. The polls close at 5:00 pm San Francisco time on Saturday September 12. A) The 10.12 release notes will tell users that 10.12 is the last release which supports Java 7. Note

derby wiki and build/test environments

Is there a Derby wiki for development issues? The derby mats tests are hanging for me and I was hoping to rummage for clues somewhere. This is what I'm seeing: the xaStateTran tests run to conclusion successfully but hang on exit--and this freezes the overall suite. I'm running on the following

please add me as a derby developer

Hello, Please add me as a derby developer. Thanks, -Rick

Re: please add me as a derby developer

... Do you have a Jira account? If so, what is it? You would need to create a Jira account first, if you don't have one. You would need to be part of 'derby-developers' if you have plans to work on bugs or enhancements to Derby. Satheesh Richard Hillegas wrote: Hello, Please add me

Re: please add me as a derby developer

or enhancements... Let me know if you have this intent. (and I hope you do :-) ) Here is some info on Jira and how to work with bugs: http://incubator.apache.org/derby/DerbyBugGuidelines.html Satheesh Richard Hillegas wrote: Hi Satheesh, Thanks. I have just created a Jira account. The userid