New SQL JIRA components (was Re: setting component's on jira items.)

[Andrew McIntyre]

Does anyone have any further comments or suggestions concerning  
breaking out the issues in the SQL component into the categories  
discussed by Satheesh below?


If not, I will add the five new components, leaving all current  
issues in the generic SQL component, and leave it to those interested  
in language issues to recategorize the current issues to the new  
categories.


andrew

On Jan 23, 2006, at 10:25 PM, Satheesh Bandaram wrote:

Mike Matrigali wrote:


I don't expect most users to be able to set the internal component
so I think it is fine to leave SQL.  The subcomponents would be a
way for someone who understands the issue a little more to classify
it.  SQL-parser, SQL-optimizer, SQL-compiler, SQL-engine seem like
a good start.


This sounds good, balancing what is good for users and developers
working with JIRA's limitations. So, I would suggest:

SQL
SQL-parser
SQL-optimizer
SQL-compiler
SQL-datatype
SQL-execute

Satheesh

Re: (DERBY-655) getImportedKeys returns duplicate rows in some cases

[Satheesh Bandaram]

Sorry, it didn't work. Like I mentioned in my email, I wasn't sure if
the change was right. I was only focused on returing one row :-) 

Good luck with that monster query...

Satheesh

Mamta Satoor wrote:

  Hi Satheesh,
   
  Thanks for all the time you spent on this. 
   
  I copied the suggested changes into my codeline and tried
running derbyall against it. The existing metadata.java and
odbc_metadata.java fail and don't return any row for getImportedKeys
test. So, looks like more tweaking is needed to fix the sql in
metadata.properties for getImportedKeys. If you/anyone else think of
any tips, please let me know. In the mean time, I will continue to work
on my end too.
   
  thanks,
  Mamta
  
 
  On 1/27/06, Satheesh Bandaram <[EMAIL PROTECTED]>
wrote:
  If
I add one following line, getImportedKeys returns only one row for T1.

@@ -544,6 +580,7 @@

   
CONGLOMS.DESCRIPTOR.getKeyColumnPosition(COLS.COLUMNNUMBER) ELSE 0 END)
<> 0  \
    AND K.CONGLOMERATEID = CONGLOMS.CONGLOMERATEID  \
    AND C.TABLEID = COLS.REFERENCEID  \
+    AND CONGLOMS.CONGLOMERATENAME = C.CONSTRAINTNAME \
    ORDER BY PKTABLE_CAT,  \
    PKTABLE_SCHEM, \
    PKTABLE_NAME, \

With the change it returns two rows for T2 and no rows for
T3. I am not sure if this output is correct nor if the change is OK.

Satheesh

PS: After changing metadata.properties, a new database needs to be
created to see changed behavior.


[bandaram:satheesh] java keys T1
*** Call getImportedKeys

Imported keys# 1

PKTABLE_CAT:
PKTABLE_SCHEM: APP

PKTABLE_NAME: T2
PKCOLUMN_NAME: C21_ID
FKTABLE_CAT:
FKTABLE_SCHEM: APP
FKTABLE_NAME: T1
FKCOULMN_NAME: C11_ID
KEY_SEQ: 1
UPDATE_RULE: 3
DELETE_RULE: 0
FK_NAME: F_12
PK_NAME: SQL060127103319020

DEFERRABILITY: 7


[bandaram:satheesh] java keys T2
*** Call getImportedKeys

Imported keys# 1


PKTABLE_CAT:
PKTABLE_SCHEM: APP
PKTABLE_NAME: T3
PKCOLUMN_NAME: C31_ID
FKTABLE_CAT:
FKTABLE_SCHEM: APP
FKTABLE_NAME: T2
FKCOULMN_NAME: C21_ID
KEY_SEQ: 1

UPDATE_RULE: 3
DELETE_RULE: 0
FK_NAME: F_443
PK_NAME: SQL060127103320650
DEFERRABILITY: 7



Imported keys# 2

PKTABLE_CAT:
PKTABLE_SCHEM: APP
PKTABLE_NAME: T3
PKCOLUMN_NAME: C31_ID
FKTABLE_CAT:
FKTABLE_SCHEM: APP
FKTABLE_NAME: T2

FKCOULMN_NAME: C21_ID
KEY_SEQ: 1
UPDATE_RULE: 3
DELETE_RULE: 0
FK_NAME: F_443
PK_NAME: SQL060127103320650
DEFERRABILITY: 7


[bandaram:satheesh] java keys T3

*** Call getImportedKeys
[bandaram:satheesh]


Satheesh Bandaram wrote:

  Daniel John Debrunner wrote:

  
  
Mamta Satoor wrote:
 

My only advice is to break the query down from its inner elements out.
Ensure each of those in isolation is returning the correct data. Then
work on the next level out. Maybe even creating a view for the working
inner elements so the next one to tackle is somewhat readable.

E.g. with something like

SELECT * FROM T, (SELECT * FROM A,B WHERE ...) AS X
WHERE ...

 


  
  I tried to break up the query and run... The inner SELECT is returning
just one row, which seems to be correct. So, I suspect we have a problem
with the outer query, which joins several system tables with the derived
table...  I suspect we are missing one join condition, either between
system catalogs or between one of the system catalog and the derived table.

I will try little bit more...

Satheesh

  
  
Start with

SELECT * FROM A,B WHERE ...

ensure that works, then
do

create view SUB_AB AS SELECT * FROM A,B WHERE ...

then work on

SELECT * FROM T, SUB_AB
WHERE ...

Hope this is clear, just an idea to make the SQL visually
understandable. Maybe remove all the optimizer overrides as well to
clear out the clutter.

Dan.





 


  



  
  
  

Re: (DERBY-655) getImportedKeys returns duplicate rows in some cases

[Satheesh Bandaram]

You have to change the code in two classes, in case you haven't found it
already to make the query run (MethodCallNode and
StaticClassFieldReferenceNode). You may also have to change the
sqlgrammar.jj. Search for INTERNAL_SQL.

Satheesh

Mamta Satoor wrote:

>  
> Thanks. The query is also using internally available only sql syntax
> so I need to hack the code to let me allow those syntaxes in my sql
> (which is running at user level).
>  
> Mamta
>
>  
>

[jira] Commented: (DERBY-855) Document optimizer overrides which were introduced in 10.2

[Mamta A. Satoor (JIRA)]

[ 
http://issues.apache.org/jira/browse/DERBY-855?page=comments#action_12364436 ] 

Mamta A. Satoor commented on DERBY-855:
---

Couple comments
1)ctunoptimzoverride.html does a good job of cautioning the users to be careful 
about the correct syntax when using the -- DERBY-PROPERTIES clause. It will be 
good to point the users at this time to ctundepthoptover.html where they can 
verfiy if their overrides were correctly identified by the parser or not.
2)There shouldn't be any space between nested and loop when given as a value to 
joinStrategy.
3)The Tuning Guide has sections "Working with RunTimeStatistics"->"Analyzing 
the infromation". Under "Analyzing the information" page, can you add optimizer 
overrides to the existing list of Statistics timing, statement execution plan 
and optimizer estimates?

> Document optimizer overrides which were introduced in 10.2
> --
>
>  Key: DERBY-855
>  URL: http://issues.apache.org/jira/browse/DERBY-855
>  Project: Derby
> Type: Sub-task
>   Components: Documentation
> Versions: 10.2.0.0
> Reporter: Mamta A. Satoor
> Assignee: Eric Radzinski
>  Fix For: 10.2.0.0
>  Attachments: ctundepthoptover.html, ctunoptimzoverride.html
>
> Optimizer overrides support in Derby was added as jira entry DERBY-573. Eric 
> Radzinski is working on the documentation part of the feature. This issue is 
> to keep track of documentation changes.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Re: (DERBY-655) getImportedKeys returns duplicate rows in some cases

[Mamta Satoor]

Hi Dan,
 
My answers inline. 
 
Thanks for your time on it,
Mamta 
On 1/27/06, Daniel John Debrunner <[EMAIL PROTECTED]> wrote:
Mamta Satoor wrote:> Hi,>> I have been looking at Derby-655 getImportedKeys returns duplicate rows
> in some cases. Deepa reported that one of the databases with just t> many tables was returning duplicate rows for> DatabaseMetaData.getImportedKeys on a particular table. I was able to> work on that database and bring it down to 3 tables which are involved
> in the getImportedKeys call. Following is the sql which will show the> relationship between the 3 tables.>> CREATE TABLE t1(c11_ID BIGINT NOT NULL);> CREATE TABLE t2 (c21_ID BIGINT NOT NULL primary key);
> ALTER TABLE t1 ADD CONSTRAINT F_12 Foreign Key (c11_ID)>REFERENCES t2 (c21_ID) ON DELETE CASCADE ON UPDATE NO ACTION;> CREATE TABLE t3(c31_ID BIGINT NOT NULL primary key);> ALTER TABLE t2 ADD CONSTRAINT F_443 Foreign Key (c21_ID)
>REFERENCES t3(c31_ID) ON DELETE CASCADE ON UPDATE NO ACTION;>> t1(c11_id) has foreign key reference to t2(c21_id) which in turn has> foreign key reference to t3(c31_id). Now if a jdbc program tries to
> invoke DatabaseMetaData.getImportedKeys on t1, it returns 2 rows, one> for each chained foreign key reference.Is there anything significant when you say "it returns 2 rows, one> for each chained foreign key reference"? Just that it returns the same
row twice, so I'm wondering why you say "each chained reference".
 
Actually, when I wrote the mail, I thought Derby returns a duplicate row for each chained foregin key reference. ie I thought t1->t2->t3->t4 will return 3 duplicate rows for the 3-level foreign key chain among t1->t2->t3->t4 but that is not true. For both t1->t2->t3 and t1->t2->t3->t4, Derby returns 2 rows which is basically the same row twice. Which is incorrect. 

My only advice is to break the query down from its inner elements out.
Ensure each of those in isolation is returning the correct data. Thenwork on the next level out. Maybe even creating a view for the workinginner elements so the next one to tackle is somewhat readable.
E.g. with something likeSELECT * FROM T, (SELECT * FROM A,B WHERE ...) AS XWHERE ...Start withSELECT * FROM A,B WHERE ...ensure that works, thendocreate view SUB_AB AS SELECT * FROM A,B WHERE ...
then work onSELECT * FROM T, SUB_ABWHERE ...Hope this is clear, just an idea to make the SQL visuallyunderstandable. Maybe remove all the optimizer overrides as well toclear out the clutter.
Dan.
 
Thanks. The query is also using internally available only sql syntax so I need to hack the code to let me allow those syntaxes in my sql (which is running at user level). 
 
Mamta 

Re: (DERBY-655) getImportedKeys returns duplicate rows in some cases

[Mamta Satoor]

Hi Satheesh,
 
Thanks for all the time you spent on this. 
 
I copied the suggested changes into my codeline and tried running derbyall against it. The existing metadata.java and odbc_metadata.java fail and don't return any row for getImportedKeys test. So, looks like more tweaking is needed to fix the sql in 
metadata.properties for getImportedKeys. If you/anyone else think of any tips, please let me know. In the mean time, I will continue to work on my end too.
 
thanks,
Mamta 
On 1/27/06, Satheesh Bandaram <[EMAIL PROTECTED]> wrote:
If I add one following line, getImportedKeys returns only one row for T1.@@ -544,6 +580,7 @@
    CONGLOMS.DESCRIPTOR.getKeyColumnPosition(COLS.COLUMNNUMBER) ELSE 0 END) <> 0  \    AND K.CONGLOMERATEID = CONGLOMS.CONGLOMERATEID  \    AND C.TABLEID = COLS.REFERENCEID  \
+    AND CONGLOMS.CONGLOMERATENAME = C.CONSTRAINTNAME \    ORDER BY PKTABLE_CAT,  \    PKTABLE_SCHEM, \    PKTABLE_NAME, \
With the change it returns two rows for T2 and no rows for T3. I am not sure if this output is correct nor if the change is OK.SatheeshPS: After changing metadata.properties, a new database needs to be created to see changed behavior.
[bandaram:satheesh] java keys T1*** Call getImportedKeysImported keys# 1PKTABLE_CAT:PKTABLE_SCHEM: APP
PKTABLE_NAME: T2PKCOLUMN_NAME: C21_IDFKTABLE_CAT:FKTABLE_SCHEM: APPFKTABLE_NAME: T1FKCOULMN_NAME: C11_IDKEY_SEQ: 1UPDATE_RULE: 3DELETE_RULE: 0FK_NAME: F_12PK_NAME: SQL060127103319020
DEFERRABILITY: 7[bandaram:satheesh] java keys T2*** Call getImportedKeysImported keys# 1
PKTABLE_CAT:PKTABLE_SCHEM: APPPKTABLE_NAME: T3PKCOLUMN_NAME: C31_IDFKTABLE_CAT:FKTABLE_SCHEM: APPFKTABLE_NAME: T2FKCOULMN_NAME: C21_IDKEY_SEQ: 1
UPDATE_RULE: 3DELETE_RULE: 0FK_NAME: F_443PK_NAME: SQL060127103320650DEFERRABILITY: 7
Imported keys# 2PKTABLE_CAT:PKTABLE_SCHEM: APPPKTABLE_NAME: T3PKCOLUMN_NAME: C31_IDFKTABLE_CAT:FKTABLE_SCHEM: APPFKTABLE_NAME: T2
FKCOULMN_NAME: C21_IDKEY_SEQ: 1UPDATE_RULE: 3DELETE_RULE: 0FK_NAME: F_443PK_NAME: SQL060127103320650DEFERRABILITY: 7[bandaram:satheesh] java keys T3
*** Call getImportedKeys[bandaram:satheesh] 
Satheesh Bandaram wrote:
Daniel John Debrunner wrote:

  
Mamta Satoor wrote:
 

My only advice is to break the query down from its inner elements out.
Ensure each of those in isolation is returning the correct data. Then
work on the next level out. Maybe even creating a view for the working
inner elements so the next one to tackle is somewhat readable.

E.g. with something like

SELECT * FROM T, (SELECT * FROM A,B WHERE ...) AS X
WHERE ...

 

I tried to break up the query and run... The inner SELECT is returning
just one row, which seems to be correct. So, I suspect we have a problem
with the outer query, which joins several system tables with the derived
table...  I suspect we are missing one join condition, either between
system catalogs or between one of the system catalog and the derived table.

I will try little bit more...

Satheesh

  
Start with

SELECT * FROM A,B WHERE ...

ensure that works, then
do

create view SUB_AB AS SELECT * FROM A,B WHERE ...

then work on

SELECT * FROM T, SUB_AB
WHERE ...

Hope this is clear, just an idea to make the SQL visually
understandable. Maybe remove all the optimizer overrides as well to
clear out the clutter.

Dan.





 


  

[jira] Commented: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Kathey Marsden (JIRA)]

[ 
http://issues.apache.org/jira/browse/DERBY-871?page=comments#action_12364424 ] 

Kathey Marsden commented on DERBY-871:
--


>
>1) How are the new properties csinfo.server and csinfo.client different to 
>cs.serverhost and cs.trustedhost?
>
Looking at the patch I am guessing they are the same thing, but it was a lack 
of proper documentation in the policy file that led to the addition of the new 
properties.  Because the tests were only set up to run locally before myrna did 
the remote host work, I think the harness was always passing localhost before.

>2) Why is csinfo.trustedhost granted 'connect' permission for derbynet.jar in 
>the testing policy file?
>The server isn't connectiing to the client is it?
>
I don't know.  It doesn't sound right.

>
>3) Would csinfo.clienthost be a better name for csinfo.trustedhost? 
>
I think so.  or better yet something with derby instead of cs. Not the world's 
best namer though.

>It's possible that the remote testing (even using useProcess=false) does not 
>need any changes to the >policy file or the harness.

That would be cool, but please take a look at our other thread on this issue 
about the database creation.




> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Re: testing using remote host

[Daniel John Debrunner]

Kathey Marsden wrote:

> Daniel John Debrunner wrote:

>>I don't see why database creation would fail, the policy file has
>>permissions to create databases under ${derby.system.home} and ${user.dir}.
>>

> I think the network server knows nothing about the test it is running or
> what directories to create.
> Consider this URL.
> jdbc:derby:net://my.remotehost.com:1527/wombat;user=howardR;password=takeItEasy
> 
> Network server just has the derby.system.home it was started with and
> gets a database name "wombat", so all the tests that use that database
> name share a database.  In local mode network server is restarted for
> each test in the appropriate directory.

I think all that you say is true, but I don't see how it relates to not
having the permission to create a database. As far as I understand when
running the tests in the normal fashion a test will create its database in:

  ${derby.system.home} when useProcess not set (process mode)
  ${user.dir} when useProcess=false

Now in the process mode the value of derby.system.home changes for each
test (e.g. it's equal to ${user.dir}/insert when running the single test
lang/insert.sql), but that doesn't affect the permissions granted,
because they are granted relative to ${derby.system.home}.
The as far as Derby is concerned the location of the test wombat
database is the same in remote and process mode, it's
${derby.system.home}/wombat.

Dan.

Re: testing using remote host

[Kathey Marsden]

Daniel John Debrunner wrote:

>The text in 4.14 scared me as it seems the setup for test when running
>in remote mode is very different to running normally, thus increasing
>the chance for some change to a test to break it in remote mode. Thus it
>may be a never ending battle to keep the remote tests running.
>
>  
>
It seems like all of the conditions are a function of either
restrictions on remote access or useprocess itself, e.g. the need for
database cleanup and the location of the extin files.   It seems like
ideally we would want those tests that run remotely to always run with
useprocess=false when running with network server (or embedded for that
manner). Then the needed test maintenance will be caught as a matter of 
normal development.
[snip]

>I don't see why database creation would fail, the policy file has
>permissions to create databases under ${derby.system.home} and ${user.dir}.
>
>  
>
I think the network server knows nothing about the test it is running or
what directories to create.
Consider this URL.
jdbc:derby:net://my.remotehost.com:1527/wombat;user=howardR;password=takeItEasy

Network server just has the derby.system.home it was started with and
gets a database name "wombat", so all the tests that use that database
name share a database.  In local mode network server is restarted for
each test in the appropriate directory.

Kathey

[jira] Reopened: (DERBY-475) Add a system function mechanism and table of functions, including a set of initial functions.

[Daniel John Debrunner (JIRA)]

 [ http://issues.apache.org/jira/browse/DERBY-475?page=all ]
 
Daniel John Debrunner reopened DERBY-475:
-


These new match functions should use the methods in the class 
java.lang.StrictMath instead of java.lang.Math. Provides consistent/portable 
behaviour across different JVMs.

> Add a system function mechanism and table of  functions, including a set of 
> initial functions.
> --
>
>  Key: DERBY-475
>  URL: http://issues.apache.org/jira/browse/DERBY-475
>  Project: Derby
> Type: Improvement
>   Components: SQL
> Reporter: Daniel John Debrunner
> Assignee: Daniel John Debrunner
>  Fix For: 10.2.0.0

>
> Add a mechanism for system functions to be easily added. Resolution of 
> functions will check SYSFUN. for a function call in SQL when the 
> function is not qualified by a schema. If the current schema does not have a 
> function matching the name, then an additional resolution is made using 
> SYSFUN..
> Add a table driven mechanism for simple single argument functions (could be 
> expanded in the future).
> Add these functions
> /*
> ** SYSFUN functions
> *[0] = FUNCTION name
> *[1] = RETURNS type
> *[2] = Java class
> *[3] = method name
> *[4] = parameter type (single parameter)
> *
> */
> private static final String[][] SYSFUN_FUNCTIONS = {
> {"ACOS", "DOUBLE", "java.lang.Math", "acos", "DOUBLE"},
> {"ASIN", "DOUBLE", "java.lang.Math", "asin", "DOUBLE"},
> {"ATAN", "DOUBLE", "java.lang.Math", "atan", "DOUBLE"},
> {"COS", "DOUBLE", "java.lang.Math", "cos", "DOUBLE"},
> {"SIN", "DOUBLE", "java.lang.Math", "sin", "DOUBLE"},
> {"TAN", "DOUBLE", "java.lang.Math", "tan", "DOUBLE"},
> {"DEGREES", "DOUBLE", "java.lang.Math", "toDegrees", "DOUBLE"},
> {"RADIANS", "DOUBLE", "java.lang.Math", "toRadians", "DOUBLE"},
> {"LN", "DOUBLE", "java.lang.Math", "log", "DOUBLE"},
> {"EXP", "DOUBLE", "java.lang.Math", "exp", "DOUBLE"},
> {"CEIL", "DOUBLE", "java.lang.Math", "ceil", "DOUBLE"},
> {"CEILING", "DOUBLE", "java.lang.Math", "ceil", "DOUBLE"},
> {"FLOOR", "DOUBLE", "java.lang.Math", "floor", "DOUBLE"},
> };

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

[jira] Commented: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Daniel John Debrunner (JIRA)]

[ 
http://issues.apache.org/jira/browse/DERBY-871?page=comments#action_12364419 ] 

Daniel John Debrunner commented on DERBY-871:
-

Thanks Kathey,  leads to three new questions:

1) How are the new properties csinfo.server and csinfo.client different to 
cs.serverhost and cs.trustedhost?

2) Why is csinfo.trustedhost granted 'connect' permission for derbynet.jar in 
the testing policy file?
The server isn't connectiing to the client is it?

3) Would csinfo.clienthost be a better name for csinfo.trustedhost? 


Not sure exactly what you are asking of me for property cleanup in DERBY-871, 
but I think a start is a clear definition of any properties used in the testing 
policy file. It's possible that the remote testing (even using 
useProcess=false) does not need any changes to the policy file or the harness. 

> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Re: testing using remote host

[Daniel John Debrunner]

The text in 4.14 scared me as it seems the setup for test when running
in remote mode is very different to running normally, thus increasing
the chance for some change to a test to break it in remote mode. Thus it
may be a never ending battle to keep the remote tests running.

Myrna van Lunteren wrote:

> The current test harness will try to create databases in subdirectories,
> with the name of the test. Also, the derby_tests.policy file gets copied
> to the correct test dir and accessed through there. I envisaged that
> without useprocess=false, creation of databases would have failed.

I don't see why database creation would fail, the policy file has
permissions to create databases under ${derby.system.home} and ${user.dir}.

> Even
> if we'd have directories created by hand beforehand, we'd still have a
> problem because the derby_tests.policy file normally gets copied to a
> proper place in the test directories and as this doesn't happen on the
> remote host, we'd have incorrect file permissions...

I don't understand what you are trying to say here. The location of the
policy file (I think) doesn't matter, as long as it is pointed to by
java.security.policy.


> Also, it seemed to me that as the actual network interaction was bigger,
> it made sense to speed up the process by having as much as possible
> reuse of databases; but that was more of a side-issue.

Right, that's a separate issue, if we could speed up tests by running
them with useProcess=false than that should be done independent of
running remote or not.

Thanks for the info.
Dan.

[jira] Updated: (DERBY-239) Need a online backup feature that does not block update operations when online backup is in progress.

[Mike Matrigali (JIRA)]

 [ http://issues.apache.org/jira/browse/DERBY-239?page=all ]

Mike Matrigali updated DERBY-239:
-

Description: 
Currently Derby allows users to perfoms  online backups using 
SYSCS_UTIL.SYSCS_BACKUP_DATABASE() procedure,  but while the backup is in 
progress, update operations are temporarily blocked, but read operations can 
still proceed.

Blocking update operations can be real issue specifically in client server 
environments, because user requests will be blocked for a long time if a 
backup is in the progress on the server.

  was:
Currently Derby allows users to perfoms  online backups using 
SYSCS_UTIL.SYSCS_BACKUP_DATABASE() procedure,  but while the backup is in 
progress, update operations are temporarily blocked, but read operations can 
still proceed.

Blocking update operations can be real issue specifically in client server 
environments, because user requests will be blocked for a long time if a 
backup is in the progress on the server.

Environment: 

I committed the onlinebackup_8.diff patch to trunk as svn  373380

> Need a online backup feature  that does not block update operations   when 
> online backup is in progress.
> 
>
>  Key: DERBY-239
>  URL: http://issues.apache.org/jira/browse/DERBY-239
>  Project: Derby
> Type: New Feature
>   Components: Store
> Versions: 10.1.1.0
> Reporter: Suresh Thalamati
> Assignee: Suresh Thalamati
>  Attachments: obtest_customer.jar, onlinebackup.html, onlinebackup1.html, 
> onlinebackup_1.diff, onlinebackup_2.diff, onlinebackup_3.diff, 
> onlinebackup_4.diff, onlinebackup_5.diff, onlinebackup_6.diff, 
> onlinebackup_7.diff, onlinebackup_8.diff
>
> Currently Derby allows users to perfoms  online backups using 
> SYSCS_UTIL.SYSCS_BACKUP_DATABASE() procedure,  but while the backup is in 
> progress, update operations are temporarily blocked, but read operations can 
> still proceed.
> Blocking update operations can be real issue specifically in client server 
> environments, because user requests will be blocked for a long time if a 
> backup is in the progress on the server.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

[jira] Commented: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Kathey Marsden (JIRA)]

[ 
http://issues.apache.org/jira/browse/DERBY-871?page=comments#action_12364390 ] 

Kathey Marsden commented on DERBY-871:
--

Sorry, indicated the wrong bug number, should have said ..

'Can you please offer your ideas on how to organize them for DERBY-871  to not 
aggravate that situation given the various dependencies and the fact that we 
will ultimately want to move away from the harness? 


> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

[jira] Commented: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Kathey Marsden (JIRA)]

[ 
http://issues.apache.org/jira/browse/DERBY-871?page=comments#action_12364389 ] 

Kathey Marsden commented on DERBY-871:
--

Hi Dan,

csinfo.serverhost   -  Host name or ip where network server is started. 
This is needed  for SocketPermission "accept"  for  NetworkServerControl andmin 
commands like  trace,  runtimeinfo  etc. These commands need to be executed  
from the  host where network server was started.

csinfo.trustedhost   - Host name or ip from which client connections are 
allowed.

With regard to $csinfo.serverhost,  now that I think about it we could 
potentially use an alternate Socket constructor for these commands to bind the 
Socket to localhost, so that permission is not needed for the ip of  the host 
that started network server.  That way we could get rid of it all together (I 
think).

Speaking of naming,  properties etc,  as you have pointed out in the past, the 
test harness has a lot of properties that are tangled up with each other and 
not always clear.  Can you please offer your ideas on how to organize them for 
DERBY-817  to not aggravate that situation given the various dependencies and 
the fact that we will ultimately  want to move  away from the harness?


Thanks

Kathey



> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Re: testing using remote host

[Myrna van Lunteren]

The current test harness will try to create databases in subdirectories, with the name of the test. Also, the derby_tests.policy file gets copied to the correct test dir and accessed through there. I envisaged that without useprocess=false, creation of databases would have failed. Even if we'd have directories created by hand beforehand, we'd still have a problem because the derby_tests.policy file normally gets copied to a proper place in the test directories and as this doesn't happen on the remote host, we'd have incorrect file permissions...

Also, it seemed to me that as the actual network interaction was bigger, it made sense to speed up the process by having as much as possible reuse of databases; but that was more of a side-issue.
 
I'll try to think of a way to explain that succinctly in the section 4.14 if you think all this makes sense.
 
Myrna 
On 1/29/06, Daniel John Debrunner <[EMAIL PROTECTED]> wrote:
Sorry that the changes to run useProcess under the security managerbroke the remote testing, I was not aware of the implications. Looking
at section 4.14 or the testing README, I couldn't understand why whenrunning in this mode useProcess=false is choosen. Why not continue torun the clients as they did before?Dan.

[jira] Commented: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Daniel John Debrunner (JIRA)]

[ 
http://issues.apache.org/jira/browse/DERBY-871?page=comments#action_12364386 ] 

Daniel John Debrunner commented on DERBY-871:
-

In RunTest.java, what was the reason for changing the point at which the 
security manager is installed? I thought I had put it after the 
System.setIn/setOut calls so that the setIO permission was not required to be 
granted.

This file seems to have no real changes
java/testing/org/apache/derbyTesting/functionTests/suites/jdbc20.runall

These files seem to be unrelated to the bug
java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/resultset.java
java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/prepStmt.java

> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

[jira] Commented: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Daniel John Debrunner (JIRA)]

[ 
http://issues.apache.org/jira/browse/DERBY-871?page=comments#action_12364384 ] 

Daniel John Debrunner commented on DERBY-871:
-

In the policy file you've added two new properties, csinfo.client and 
csinfo.server but there are no comments in the top section indicating what 
those properties represent.

This coincides with a request I was going to make for someone to add comments 
to the policy file for the properties csinfo.serverhost and csinfo.trustedhost. 
That's part of the file I copied from nwsvr.policy but never understood.

Also, my question about the reason to run the tests differently should be 
answered before this patch is submitted.

> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

testing using remote host

[Daniel John Debrunner]

Sorry that the changes to run useProcess under the security manager
broke the remote testing, I was not aware of the implications. Looking
at section 4.14 or the testing README, I couldn't understand why when
running in this mode useProcess=false is choosen. Why not continue to
run the clients as they did before?

Dan.

[jira] Updated: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Myrna van Lunteren (JIRA)]

 [ http://issues.apache.org/jira/browse/DERBY-871?page=all ]

Myrna van Lunteren updated DERBY-871:
-

Other Info: [Patch available]

> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

[jira] Updated: (DERBY-871) remote host support in test harness needs more work because of security manager changes

[Myrna van Lunteren (JIRA)]

 [ http://issues.apache.org/jira/browse/DERBY-871?page=all ]

Myrna van Lunteren updated DERBY-871:
-

Attachment: DERBY-874-012906.stat
DERBY-874-012906.diff

> remote host support in test harness needs more work because of security 
> manager changes
> ---
>
>  Key: DERBY-871
>  URL: http://issues.apache.org/jira/browse/DERBY-871
>  Project: Derby
> Type: Bug
>   Components: Test
> Versions: 10.2.0.0
> Reporter: Myrna van Lunteren
> Assignee: Myrna van Lunteren
> Priority: Minor
>  Attachments: DERBY-874-012906.diff, DERBY-874-012906.stat
>
> With the changes to have security manager run for tests with 
> useprocess=false, the remote host support in the test harness no l onger 
> works.
> permissions need to be passed in derby_test.policy for the client.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Re: [jira] Updated: (DERBY-877) zOS - with DerbyClient getDate(#) fails with IllegalArgumentException - unsupported date format - resultset.java

[Myrna van Lunteren]

On 1/29/06, Kathey Marsden (JIRA)  wrote:
[ http://issues.apache.org/jira/browse/DERBY-877?page=all
 ]Kathey Marsden updated DERBY-877:-   Attachment: derby-877.diffThe patch fixes issues with getString, getTimeStamp, getDate and getTime on TIMESTAMP, DATE and TIME columns when the client JVM encoding does not match the server encoding for the characters being evaluated in 
DateTime.java methods- Changes the following methods in DateTime.java to take encoding parameter and create string based on encoding.dateBytesToDate, timeBytesToTime, timeBytesToTimeStamp, dateBytesToTimeStamp, timestampBytesToDate, timestampBytesToTime
- Changes calling code to pass column encoding and throw SQLExceptions for UnsupportedEncoding  exceptions if thrown from the methods above.Tests: derbyall  passed as did  the repro attached to this issue on Windows  with Sun JDK 
1.5 (note repro won't run with jdk 1.4.2).I still do not have a solution for testing within the harness on systems  where the JVM encoding  does match the server encoding.Any ideas on testing  solutions are welcome.

 
 
How about using the remote host stuff? Then you can set up the network server on a machine with a different encoding?  I'm almost ready with a patch to DERBY-871 (the securityManager with useprocess stuff broke the remote host stuff).

Of course, I'd still need to fix the harness to correctly handle the different encoding 
(DERBY-658 I had at one point started on this but got side-tracked.)... 

[jira] Updated: (DERBY-877) zOS - with DerbyClient getDate(#) fails with IllegalArgumentException - unsupported date format - resultset.java

[Kathey Marsden (JIRA)]

 [ http://issues.apache.org/jira/browse/DERBY-877?page=all ]

Kathey Marsden updated DERBY-877:
-

Attachment: derby-877.diff

The patch fixes issues with getString, getTimeStamp, getDate and getTime on 
TIMESTAMP, DATE and TIME columns when the client JVM encoding does not match 
the server encoding for the characters being evaluated in DateTime.java methods

- Changes the following methods in DateTime.java to take encoding parameter and 
create string based on encoding.
dateBytesToDate, timeBytesToTime, timeBytesToTimeStamp, dateBytesToTimeStamp, 
timestampBytesToDate, timestampBytesToTime

- Changes calling code to pass column encoding and throw SQLExceptions for 
UnsupportedEncoding  exceptions if thrown from the methods above.

Tests: derbyall  passed as did  the repro attached to this issue on Windows  
with Sun JDK 1.5 (note repro won't run with jdk 1.4.2).
I still do not have a solution for testing within the harness on systems  where 
the JVM encoding  does match the server encoding.
 Any ideas on testing  solutions are welcome. 


> zOS - with DerbyClient getDate(#) fails with IllegalArgumentException - 
> unsupported date format - resultset.java
> 
>
>  Key: DERBY-877
>  URL: http://issues.apache.org/jira/browse/DERBY-877
>  Project: Derby
> Type: Bug
>   Components: Network Client
> Versions: 10.1.1.0
>  Environment: OS/390 with classic (IBM) jvm142
> Reporter: Myrna van Lunteren
> Assignee: Kathey Marsden
>  Attachments: TestEnc.java, derby-877.diff
>
> The test lang/resultset.java fails with DerbyNetClient on zOS 
> because ResultSet.getDate(#) fails with an 
> java.lang.IllegalArgumentException - unsupported date format.
> This is the stack trace with 10.2 debug version (but it fails 
> with 10.1 also):
> --
> 
> getBytes(dt) got exception
> Data Conversion SQLException
> FAIL -- unexpected exception: 
> java.lang.IllegalArgumentException: Unsupported date format!
> java.lang.IllegalArgumentException: Unsupported date format!
> at 
> org.apache.derby.client.am.DateTime.dateBytesToDate(DateTime.java:63)
> at 
> org.apache.derby.client.am.Cursor.getDATE(Cursor.java:400)
> at 
> org.apache.derby.client.am.Cursor.getDate(Cursor.java:712)
> at 
> org.apache.derby.client.am.ResultSet.getDate(ResultSet.java:687)
> at 
> org.apache.derbyTesting.functionTests.tests.jdbcapi.resultset.main(Unknown 
> Source)
> --
> Note: does not fail with jcc.
> Also, test lang/updatableResultSet.java failed with e.g.:
>   - instead of 'Got expected exception : Illegal Conversion' :
>'Got expected exception : Unsupported date format!' . 
>   - instead of 'Got expected exception : Illegal Conversion' :
>'Got expected exception : nanos > 999 or < 0' .

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira