Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-09-11 Thread Christian Schneider 

You can use the new PreHook of pax-jdbc.
It allows to run an OSGi service before the datasource is published.

This can be used in production and test for db migration as well as for 
setup of a temp db before each test.


This is an example for a migration hook:
https://github.com/cschneider/Karaf-Tutorial/blob/master/liquibase/service/src/main/java/net/lr/tutorial/db/service/Migrator.java

and this is how to use it in the config:
https://github.com/cschneider/Karaf-Tutorial/blob/master/liquibase/org.ops4j.datasource-person.cfg

Christian


On 24.08.2017 09:36, karsten.bl...@hermes-ottoint.com wrote:

Hello everybody,
i am running pax exam 4 with a karaf 3 container.
The bundle under test is a camel 2.16.3 bundle.
The bundle uses a time triggered camel jpa component and pax-jdbc with 
(XA and pooling) to instantiate the datasource.

For the test i need some table and some test data in a schema.
If i run the test with an existing postgres database everything is 
fine. If i try to instantiate and setup the database first (H2 or 
Derby) the bundle under test does not

find the database with the jndi name:
javax.naming.NameNotFoundException: 
osgi:service/javax.sql.DataSource/"(osgi.jndi.service.name=jdbc/scv_xa)"


How can i startup and setup the temporary db before the bundle under 
test is started?


Thanks and best regards
Karsten
--
--
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

---
You received this message because you are subscribed to the Google 
Groups "OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to ops4j+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.



--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

--
--
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups "OPS4J" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-09-01 Thread 'Christoph Läubrich' via OPS4J 

Hi,

I mean that you already have the datasource at hand so there is no need 
to look it up in JNDI. But it seems there is no way of connecting camel 
with the underlying persitence unit.


BTW: Your problem has nothing to do with pax-jdbc or pax-exam it is more 
related to aries-jpa that does not account the dynamic nature of OSGi 
when doing the JNDI lookup it obviously does not wait for a service to 
become available but fails instantly.


What you can try is to adjust the start order of your bundles so the 
pax-jdbc, config-admin and derby bundles start as soon as possible 
before aries-jpa kicks in. Also make yure your osgi-jndi bundle starts 
as soon as possible!




Am 31.08.2017 um 18:01 schrieb karsten.bl...@hermes-ottoint.com:

I have added to the blueprint description of my bundle under test:

 

with no result.
Getting still the unfriendly Exception and after that the
INFO  | FelixStartLevel
  | nfig.impl.DataSourceRegistration | Creating 
DataSource jdbc/scv


message.

what do you mean by :
-- "
Beside that you should check if camel can even use those fetched service 
as a datasource instead of JNDI because that might create issues if the 
service changes while blueprint keeps track of those changes."?


--
--
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

---
You received this message because you are subscribed to the Google 
Groups "OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to ops4j+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
--
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups "OPS4J" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-09-01 Thread karsten . blume 
here is some more information:
i have started the karaf service created by pax exam manually, getting on 
the command line:

service:list DataSource shows:

[javax.sql.DataSource]
--
 databaseName = scvdev
 dataSourceName = jdbc/scv_xa
 felix.fileinstall.filename = 
file:/D:/HYPER-V/data/git/AMX/hoi.itest.karaf/target/pax/fe9a14e5-ac9a-47a3-acb8-bec13851ab36/etc/org.ops4j.datasource-scv_xa.cfg
 osgi.jdbc.driver.name = derby-pool-xa
 osgi.jndi.service.name = jdbc/scv_xa
 password = xxx
 service.factoryPid = org.ops4j.datasource
 service.id = 409
 service.pid = org.ops4j.datasource.5d401396-0dc7-445e-8dad-0c1077a2a28f
 url = jdbc:derby:target/scvdev;create=true;user=xxx;password=xxx
 user = xxx
Provided by :
 OPS4J Pax JDBC Config (39)

also 
jdbc:datasources shows:
jdbc/scv_xa, 409   | Apache Derby | 10.11.1.1 - (1616546) | 
jdbc:derby:target/scvdev


| OK
and jdbc:tables shows all the Derby system tables.

So this seems to be a timing problem between those bundles.
Wondering does nobody use the pax-jdbc bundle and pax exam?

-- 
-- 
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups 
"OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-08-31 Thread karsten . blume 
I have added to the blueprint description of my bundle under test:



with no result.
Getting still the unfriendly Exception and after that the 
INFO  | FelixStartLevel   
 | nfig.impl.DataSourceRegistration | Creating 
DataSource jdbc/scv 

message.

what do you mean by :
-- "
Beside that you should check if camel can even use those fetched service as 
a datasource instead of JNDI because that might create issues if the 
service changes while blueprint keeps track of those changes."?

-- 
-- 
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups 
"OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-08-30 Thread 'Christoph Läubrich' via OPS4J 
From the log it seems your blueprint starts up before the service is 
registered. Since Camel does not know directly about the dependency you 
might try the following:
Let your blueprint import a javax.sql.DataSource service, then it won't 
start up before one is avaiable e.g. with


  

that will make your bundle wait for at least one DatasourceService (you 
might want to adjust the interface if pax-jdbc registers a different 
one, havent checked that...).


Beside that you should check if camel can even use those fetched service 
as a datasource instead of JNDI because that might create issues if the 
service changes while blueprint keeps track of those changes.


Am 30.08.2017 11:46, schrieb karsten.bl...@hermes-ottoint.com:

full Stack Trace for exception and log excerpt
datasources.osgi includes the pax-jdbc config files in live, for test 
the additional pax-jdbc bundles for derby and config files are added 
in the PaxExam configure() method
The only strange thing i see now is that the "Creating DataSource" log 
message is after the JNDI search by "org.apache.aries.jpa.container".

Might this be the cause? And how to change?

...
2017-08-30 11:26:42,770 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature datasources.osgi 
17.32.2-SNAPSHOT | bcid:  
exchange:  msgid:
2017-08-30 11:26:42,770 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature camel-jdbc 
2.16.3 | bcid:  exchange:  msgid:
2017-08-30 11:26:42,772 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-config 
0.9.0 | bcid:  exchange:  msgid:
2017-08-30 11:26:42,772 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-spec 
0.9.0 | bcid:  exchange:  msgid:
2017-08-30 11:26:42,776 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature 
pax-jdbc-postgresql 0.9.0 | bcid:  
exchange:  msgid:
2017-08-30 11:26:42,777 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature 
pax-jdbc-pool-dbcp2 0.9.0 | bcid:  
exchange:  msgid:




2017-08-30 11:26:44,968 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-derby 
0.9.0 | bcid:  exchange:  msgid:
2017-08-30 11:26:44,968 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-spec 
0.9.0 | bcid:  exchange:  msgid:


2017-08-30 11:27:04,677 | INFO  | Factory Update: 
factoryPid=[org.ops4j.datasource]) | nfig.impl.DataSourceRegistration 
| Detected config for DataSource jdbc/scv. Tracking DSF with filter 
(&(objectClass=org.osgi.service.jdbc.DataSourceFactory)(osgi.jdbc.driver.name=derby-pool)) 
| bcid:  exchange:  msgid:
2017-08-30 11:27:05,343 | INFO  | Factory Update: 
factoryPid=[org.ops4j.datasource]) | nfig.impl.DataSourceRegistration 
| Detected config for DataSource jdbc/scv_xa. Tracking DSF with filter 
(&(objectClass=org.osgi.service.jdbc.DataSourceFactory)(osgi.jdbc.driver.name=derby-pool-xa)) 
| bcid:  exchange:  msgid:

...

2017-08-30 11:27:27,421 | INFO  | 
FelixStartLevel| 
openjpa.Runtime  | Starting OpenJPA 2.2.2 | 
bcid:  exchange:  msgid:
2017-08-30 11:27:27,482 | ERROR | 
FelixStartLevel| 
org.apache.aries.jpa.container   | The DataSource 
osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/scv_xa) 
required by bundle UNDERTEST/17.32.2.SNAPSHOT could not be found. | 
bcid:  exchange:  msgid:
javax.naming.NameNotFoundException: 
osgi:service/javax.sql.DataSource/"(osgi.jndi.service.name=jdbc/scv_xa)"
at 
org.apache.aries.jndi.url.ServiceRegistryContext.lookup(ServiceRegistryContext.java:113)
at 
org.apache.aries.jndi.url.ServiceRegistryContext.lookup(ServiceRegistryContext.java:138)
at 
org.apache.aries.jndi.DelegateContext.lookup(DelegateContext.java:161)

at javax.naming.InitialContext.lookup(InitialContext.java:417)
at 
org.apache.aries.jpa.container.unit.impl.JndiDataSource.getDs(JndiDataSource.java:66)
at 
org.apache.aries.jpa.container.unit.impl.DelayedLookupDataSource.getConnection(DelayedLookupDataSource.java:36)
at

Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-08-30 Thread karsten . blume 
How is your test triggered? By timer? 

The test is started automatically by PaxExam.

>
> Can you create a simpler test-case you can publish on github that shows 
> your problem?
>

Not really, because there are several additional dependencies (db 
unrelated) involved.

-- 
-- 
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups 
"OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-08-30 Thread karsten . blume 
full Stack Trace for exception and log excerpt
datasources.osgi includes the pax-jdbc config files in live, for test the 
additional pax-jdbc bundles for derby and config files are added in the 
PaxExam configure() method
The only strange thing i see now is that the "Creating DataSource" log 
message is after the JNDI search by "org.apache.aries.jpa.container".
Might this be the cause? And how to change?

...
2017-08-30 11:26:42,770 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature datasources.osgi 
17.32.2-SNAPSHOT | bcid:  
exchange:  msgid: 
2017-08-30 11:26:42,770 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature camel-jdbc 2.16.3 | 
bcid:  exchange:  
msgid: 
2017-08-30 11:26:42,772 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-config 0.9.0 
| bcid:  exchange:  
msgid: 
2017-08-30 11:26:42,772 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-spec 0.9.0 | 
bcid:  exchange:  
msgid: 
2017-08-30 11:26:42,776 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-postgresql 
0.9.0 | bcid:  exchange:  
msgid: 
2017-08-30 11:26:42,777 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-pool-dbcp2 
0.9.0 | bcid:  exchange:  
msgid: 



2017-08-30 11:26:44,968 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-derby 0.9.0 
| bcid:  exchange:  
msgid: 
2017-08-30 11:26:44,968 | INFO  | 
FelixStartLevel| 
res.internal.FeaturesServiceImpl | Installing feature pax-jdbc-spec 0.9.0 | 
bcid:  exchange:  
msgid: 

2017-08-30 11:27:04,677 | INFO  | Factory Update: 
factoryPid=[org.ops4j.datasource]) | nfig.impl.DataSourceRegistration | 
Detected config for DataSource jdbc/scv. Tracking DSF with filter 
(&(objectClass=org.osgi.service.jdbc.DataSourceFactory)(osgi.jdbc.driver.name=derby-pool))
 
| bcid:  exchange:  
msgid: 
2017-08-30 11:27:05,343 | INFO  | Factory Update: 
factoryPid=[org.ops4j.datasource]) | nfig.impl.DataSourceRegistration | 
Detected config for DataSource jdbc/scv_xa. Tracking DSF with filter 
(&(objectClass=org.osgi.service.jdbc.DataSourceFactory)(osgi.jdbc.driver.name=derby-pool-xa))
 
| bcid:  exchange:  
msgid: 
...

2017-08-30 11:27:27,421 | INFO  | 
FelixStartLevel| 
openjpa.Runtime  | Starting OpenJPA 2.2.2 | 
bcid:  exchange:  
msgid: 
2017-08-30 11:27:27,482 | ERROR | 
FelixStartLevel| 
org.apache.aries.jpa.container   | The DataSource 
osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/scv_xa) 
required by bundle UNDERTEST/17.32.2.SNAPSHOT could not be found. | 
bcid:  exchange:  
msgid: 
javax.naming.NameNotFoundException: 
osgi:service/javax.sql.DataSource/"(osgi.jndi.service.name=jdbc/scv_xa)"
at 
org.apache.aries.jndi.url.ServiceRegistryContext.lookup(ServiceRegistryContext.java:113)
at 
org.apache.aries.jndi.url.ServiceRegistryContext.lookup(ServiceRegistryContext.java:138)
at 
org.apache.aries.jndi.DelegateContext.lookup(DelegateContext.java:161)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at 
org.apache.aries.jpa.container.unit.impl.JndiDataSource.getDs(JndiDataSource.java:66)
at 
org.apache.aries.jpa.container.unit.impl.DelayedLookupDataSource.getConnection(DelayedLookupDataSource.java:36)
at 
org.apache.openjpa.lib.jdbc.DelegatingDataSource.getConnection(DelegatingDataSource.java:110)
at 
org.apache.openjpa.lib.jdbc.DecoratingDataSource.getConnection(DecoratingDataSource.java:87)
at 
org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:91)
at 
org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
at

RE: Setup database structure before bundle under test is started with pax exam + karaf

2017-08-30 Thread Blume, Karsten 
Hello,
yes this might help.
Bundle under test is a blueprint bundle with a camel jpa endpoint/component
   
  
  
   

and a persistence.xml
   
  
osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/scv_xa)

a running live pax-jdbc configuration
this works also for the pax exam test
osgi.jdbc.driver.name = PostgreSQL JDBC Driver-pool-xa
databaseName=scv
dataSourceName=jdbc/scv_xa
user=xxx
password=xxx
url = jdbc:postgresql:// 155:155:155:155:5432/scv

What does not work:
A pax exam pax-jdbc configuration which I expect starts up the db before 
starting the test like:
osgi.jdbc.driver.name=derby-pool-xa
databaseName=scv
dataSourceName=jdbc/scv_xa
user=xxx
password=xxx
url=jdbc:derby:target/scvdev;create=true;user=xxx;password=xxx
or similar with H2

For the different test methods I need different test data inserted into the db. 
So a single startup script is not sufficient.

For the pax exam tests
How can a startup the db, modify it per test method and publish it so that the 
probe and the bundle under test can access it?

Regards
Karsten



From: laeubi.mobile via OPS4J [mailto:ops4j@googlegroups.com]
Sent: Dienstag, 29. August 2017 15:37
To: OPS4J
Subject: Re: Setup database structure before bundle under test is started with 
pax exam + karaf

You should provide a little more info about your setup, e.g. "working" vs "not 
working" configs, how you run the camel bundle (bluprint? spring? 
Routebuilders?)

I assume you want to test the route itself not the pax-jdbc setup, then you 
should configure your datasource that way that it can either run from JNDI or 
or a local jdbc-url to make testing easier.
You can the provide a config for test and one for deployment that contains 
either jndi or plain jdbc with e.g. inmemory db of h2 (which can be configured 
to init the db from a file).

Am Donnerstag, 24. August 2017 09:36:05 UTC+2 schrieb 
karste...@hermes-ottoint.com<mailto:karste...@hermes-ottoint.com>:
Hello everybody,
i am running pax exam 4 with a karaf 3 container.
The bundle under test is a camel 2.16.3 bundle.
The bundle uses a time triggered camel jpa component and pax-jdbc with (XA and 
pooling) to instantiate the datasource.
For the test i need some table and some test data in a schema.
If i run the test with an existing postgres database everything is fine. If i 
try to instantiate and setup the database first (H2 or Derby) the bundle under 
test does not
find the database with the jndi name:
javax.naming.NameNotFoundException: 
osgi:service/javax.sql.DataSource/"(osgi.jndi.service.name<http://osgi.jndi.service.name>=jdbc/scv_xa)"

How can i startup and setup the temporary db before the bundle under test is 
started?

Thanks and best regards
Karsten
--
--
--
OPS4J - http://www.ops4j.org - 
ops4j@googlegroups.com<mailto:ops4j@googlegroups.com>

---
You received this message because you are subscribed to a topic in the Google 
Groups "OPS4J" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/ops4j/F1YhX1Fvq_Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
ops4j+unsubscr...@googlegroups.com<mailto:ops4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

-- 
-- 
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups 
"OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Setup database structure before bundle under test is started with pax exam + karaf

2017-08-29 Thread laeubi.mobile via OPS4J 
You should provide a little more info about your setup, e.g. "working" vs 
"not working" configs, how you run the camel bundle (bluprint? spring? 
Routebuilders?)

I assume you want to test the route itself not the pax-jdbc setup, then you 
should configure your datasource that way that it can either run from JNDI 
or or a local jdbc-url to make testing easier.
You can the provide a config for test and one for deployment that contains 
either jndi or plain jdbc with e.g. inmemory db of h2 (which can be 
configured to init the db from a file).

Am Donnerstag, 24. August 2017 09:36:05 UTC+2 schrieb 
karste...@hermes-ottoint.com:
>
> Hello everybody,
> i am running pax exam 4 with a karaf 3 container.
> The bundle under test is a camel 2.16.3 bundle.
> The bundle uses a time triggered camel jpa component and pax-jdbc with (XA 
> and pooling) to instantiate the datasource.
> For the test i need some table and some test data in a schema.
> If i run the test with an existing postgres database everything is fine. 
> If i try to instantiate and setup the database first (H2 or Derby) the 
> bundle under test does not 
> find the database with the jndi name:
> javax.naming.NameNotFoundException: osgi:service/javax.sql.DataSource/"(
> osgi.jndi.service.name=jdbc/scv_xa)"
>
> How can i startup and setup the temporary db before the bundle under test 
> is started?
>
> Thanks and best regards
> Karsten
>

-- 
-- 
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups 
"OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.