Hi! Sorry about the late reply.
1. In my opinion it is not question about provisioning as as soon as the rhythym application starts up, it tries to select from khepri_beats (see logs below). I'm not using demo server at all. I use Postman scripts for provisioning. I wonder if I should run some step before starting rhythm micro service. I get the error on both with latest docker image of rhythm as well as when starting the service locally. I logged bug report here: https://issues.apache.org/jira/browse/FINCN-177 Here are the logs when running from localhost 13:56:54.948 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer - Jetty started on port(s) 2022 (http/1.1) 13:56:54.958 [main] INFO o.a.f.c.r.service.RhythmApplication - Started RhythmApplication in 13.207 seconds (JVM running for 14.079) [EL Finer]: transaction: 2019-09-18 13:56:54.998--UnitOfWork(641160635)--Thread(Thread[pool-2-thread-1,5,main])--begin unit of work flush [EL Finer]: transaction: 2019-09-18 13:56:54.999--UnitOfWork(641160635)--Thread(Thread[pool-2-thread-1,5,main])--end unit of work flush [EL Finest]: query: 2019-09-18 13:56:54.999--UnitOfWork(641160635)--Thread(Thread[pool-2-thread-1,5,main])--Execute query ReadAllQuery(referenceClass=BeatEntity ) [EL Finest]: connection: 2019-09-18 13:56:55.013--ServerSession(988179589)--Connection(1724597978)--Thread(Thread[pool-2-thread-1,5,main])--Connection acquired from connection pool [default]. [EL Finer]: transaction: 2019-09-18 13:56:55.013--ClientSession(571004194)--Connection(1724597978)--Thread(Thread[pool-2-thread-1,5,main])--begin transaction [EL Finest]: connection: 2019-09-18 13:56:55.013--ClientSession(571004194)--Thread(Thread[pool-2-thread-1,5,main])--reconnecting to external connection pool [EL Fine]: sql: 2019-09-18 13:56:55.014--ClientSession(571004194)--Connection(1203869313)--Thread(Thread[pool-2-thread-1,5,main])--SELECT id, alignment_hour, application_identifier, beat_identifier, next_beat, tenant_identifier FROM khepri_beats WHERE (next_beat < ?) FOR UPDATE bind => [2019-09-18 13:56:54.923] [EL Fine]: sql: 2019-09-18 13:56:55.053--ClientSession(571004194)--Thread(Thread[pool-2-thread-1,5,main])--SELECT 1 [EL Warning]: 2019-09-18 13:56:55.055--UnitOfWork(641160635)--Thread(Thread[pool-2-thread-1,5,main])--Local Exception Stack: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.4.v20160829-44060b6): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: org.postgresql.util.PSQLException: ERROR: relation "khepri_beats" does not exist Position: 103 Error Code: 0 Call: SELECT id, alignment_hour, application_identifier, beat_identifier, next_beat, tenant_identifier FROM khepri_beats WHERE (next_beat < ?) FOR UPDATE bind => [2019-09-18 13:56:54.923] Query: ReadAllQuery(referenceClass=BeatEntity sql="SELECT id, alignment_hour, application_identifier, beat_identifier, next_beat, tenant_identifier FROM khepri_beats WHERE (next_beat < ?) FOR UPDATE") at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1620) Regards Juhan Kontakt Ebenezer Graham (<egraha...@alustudent.com>) kirjutas kuupäeval R, 6. september 2019 kell 14:28: > Thanks for the update, > > 1. For the error below, I believe that is because you didn't provision the > microservices. Can you please confirm if you set > -Ddemoserver.provision=true on the first run. Also, you can use > https://github.com/apache/fineract-cn-demo-server/blob/develop/scripts/Unix/initial-setup.sh > to > test the integration. I have deleted a few of the branches in the > eclipselink-test-script.sh > > 1. Rhythm keeps giving me this error: >> rhythm-ms_1 | Internal Exception: >> org.postgresql.util.PSQLException: ERROR: relation "khepri_beats" does not >> exist >> rhythm-ms_1 | Position: 103 > > > 2. I will make the make the changes and submit the necessary PRs asap. > > 3. I have updated the Setup Guide to cater for PostgreSQL. However, I need > someone to test the MacOS instructions work correctly. > > Regards > > On Wed, 4 Sep 2019 at 15:54, Juhan Aasaru <aas...@gmail.com> wrote: > >> Hi Graham and Isaac! >> >> Great progress and excellent work regarding Postgresql and EclipseLink >> migration! >> I did some tests (using the official docker images I'm working on). This >> is what I found >> >> 1. Rhythm keeps giving me this error: >> rhythm-ms_1 | Internal Exception: >> org.postgresql.util.PSQLException: ERROR: relation "khepri_beats" does not >> exist >> rhythm-ms_1 | Position: 103 >> rhythm-ms_1 | Error Code: 0 >> rhythm-ms_1 | Call: SELECT id, alignment_hour, >> application_identifier, beat_identifier, next_beat, tenant_identifier FROM >> khepri_beats WHERE (next_beat < ?) FOR UPDATE >> rhythm-ms_1 | bind => [2019-09-04 11:34:43.778] >> rhythm-ms_1 | Query: ReadAllQuery(referenceClass=BeatEntity >> sql="SELECT id, alignment_hour, application_identifier, beat_identifier, >> next_beat, tenant_identifier FROM khepri_beats WHERE (next_beat < ?) FOR >> UPDATE") >> >> 2. many services (provisioner, office, customer, accounting, teller, >> reporting, payroll) now define data source properties twice >> in service/src/main/resources/application.yaml >> The problem is that if I want to overwrite some value I have to pass it >> in as two separate parameters. >> >> For example: >> https://github.com/apache/fineract-cn-provisioner/commit/4327ce032c97cc961ce15e39c22863654cefc230 >> has: >> >> postgresql: >> driverClass: org.postgresql.Driver >> database: seshat >> host: localhost >> port: 5432 >> user: postgres >> password: postgres >> >> spring: >> datasource: >> driver-class-name: org.postgresql.Driver >> url: jdbc:postgresql://localhost:5432/seshat >> username: postgres >> password: postgres >> >> Please change the last block to something like this: >> >> spring: >> datasource: >> driver-class-name: org.postgresql.Driver >> url: jdbc:postgresql://${postgresql.host}:${postgresql.port}/seshat >> username: ${postgresql.user} >> password: ${postgresql.password} >> >> Then there should be no need to pass in the variables twice if I want to >> change some value. >> >> 3. Instructions should be updated as well and MariaDB/Mysql replaced with >> Postgresql: >> >> https://cwiki.apache.org/confluence/display/FINERACT/How+To+Build+Apache+Fineract+CN >> >> Also Vishwas has some good instructions that need to be updated >> https://github.com/vishwasbabu/ProvisioningFineractCN >> >> Kind regards >> Juhan >> >> Kontakt Ebenezer Graham (<egraha...@alustudent.com>) kirjutas kuupäeval >> E, 2. september 2019 kell 15:03: >> >>> Thank you for the update. >>> >>> I will resolve the conflicts on the notification service and revert to >>> you. >>> >>> >>> On Mon, 2 Sep 2019, 03:52 Isaac Kamga, <u2is...@gmail.com> wrote: >>> >>>> Hello Graham, >>>> >>>> That's some very good work you've done so far. >>>> >>>> I have executed the script and everything works well on my end right >>>> through to the fims-web-app. >>>> >>>> EclipseLink configurations was merged into PostgreSQL library last week >>>> and the changes to fineract-cn-test library on FINCN-114 which migrate it >>>> from MariaDB to PostgreSQL have been merged into the develop branch. The >>>> latter causes the Travis checks on the other repositories to succeed. >>>> >>>> However, the Thymeleaf templating PR heading to Notifications develop >>>> branch needs your attention. >>>> >>>> More grease to your elbows. >>>> >>>> Cheers, >>>> Isaac Kamga. >>>> >>>> On Sun, Aug 25, 2019 at 2:43 PM Ebenezer Graham < >>>> egraha...@alustudent.com> wrote: >>>> >>>>> Hello Devs, >>>>> >>>>> I would like to revive the topic discussed in the previous mail. >>>>> >>>>> I have been running Fineract CN on EclipseLink and PostgreSQL for a >>>>> while now and I believe it's ready to be merged into the develop branches. >>>>> But before that, it is paramount that other experienced developers in the >>>>> community examine what has been done so far and QA it. >>>>> >>>>> Simply execute this script >>>>> https://raw.githubusercontent.com/ebenezergraham/fineract-cn-demo-server/FINCN-141/scripts/Unix/eclipselink-test-setup.sh >>>>> to >>>>> get all the necessary branches. >>>>> >>>>> *PS: *I have experienced better performance with PostgreSQL and >>>>> EclipseLink on my workstation. >>>>> >>>>> For instance, Application Startup + Provisioning took 14 mins >>>>> Application Startup alone took 12 mins >>>>> >>>>> With MariaDB and Hibernate out of the way, we will be closer to making >>>>> Fineract CN compliant. On top of that, if we are able to upgrade the >>>>> spring >>>>> versions, this should significantly improve Fineract CN's response time, >>>>> performance and memory usage. >>>>> >>>>> Looking forward to hearing from you. >>>>> >>>>> Regards. >>>>> >>>>> >>>>> On Tue, 16 Jul 2019 at 04:47, Ebenezer Graham < >>>>> egraha...@alustudent.com> wrote: >>>>> >>>>>> Hello Fineracters, >>>>>> >>>>>> I just finished integrating EclipseLink into Fineract CN. I would >>>>>> like to request your participation in the testing of the new ORM. Your >>>>>> constructive feedback will help me to fine-tune the integration and >>>>>> resolve >>>>>> all known issues with PostgreSQL. >>>>>> >>>>>> *Approach Used * >>>>>> Migrating from Hibernate to EclipseLink in Spring is actually very >>>>>> simple and seamless. >>>>>> >>>>>> 1. I simply removed hibernate core and entity-manager from >>>>>> spring-data-jpa [1] >>>>>> 2. Then, I added the recommended EclipseLink JPA dependency [2] to >>>>>> the MariaDB and PostgreSQL libraries. >>>>>> 4. I extended the JpaBaseConfiguration and configured it to use >>>>>> EclipseLink as the provider. As discussed earlier, Spring provides an >>>>>> adapter for EclipseLink so it was quite straightforward. >>>>>> 5. One critical decision I made, was choosing static weaving over the >>>>>> default dynamic weaving done at runtime. Static weaving improves >>>>>> performance and stability as it occurs during build-time. In addition, >>>>>> this >>>>>> approach is recommended for large applications with lots of entities. >>>>>> Read >>>>>> more at [3] and let me know what you think. >>>>>> >>>>>> Please conduct your tests and report any issues on this thread so >>>>>> that I can work on them. >>>>>> >>>>>> *Testing EclipseLink and PostgreSQL* >>>>>> You can test EclipseLink and PostgreSQL by using this script [4]. The >>>>>> eclipselink-test-setup script will clone and publish the relevant >>>>>> artefacts. >>>>>> >>>>>> NB: Remember to shut down your Postgres server when doing component >>>>>> tests so that the embedded db can start. >>>>>> >>>>>> NB: Use this for full integration tests >>>>>> java -Ddemoserver.persistent=true -Dcustom.cassandra.contactPoints= >>>>>> 127.0.0.1:9042 -Dcassandra.cluster.user=cassandra >>>>>> -Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost >>>>>> -Dcustom.postgresql.user=postgres -jar >>>>>> ~/<fineract-cn-dir>/integration-tests/fineract-cn-demo-server/build/libs/demo-server-0.1.0-BUILD-SNAPSHOT.jar >>>>>> >>>>>> NB: The logs are a bit excessive as I'm currently outputting all >>>>>> EclipseLink logs but once testing is over, I will tone it down. >>>>>> >>>>>> I have conducted component test for each microservice and a full >>>>>> integration test. A few tests failed during the component test however, >>>>>> the >>>>>> causes are not from the ORM but the test cases themselves. I will try and >>>>>> fix them once I have met all of my milestones for this project or raise >>>>>> issues for them in Jira. >>>>>> >>>>>> *Known Issues in Postgres (My current focus):* >>>>>> 1. ERROR: relation "khepri_beats" does not exist. click on the link >>>>>> at [5] to see the full log. This issue is not directly related to the >>>>>> ORM. >>>>>> I actually identified this error whiles testing Postgres and Hibernate. >>>>>> This is related to Postgres and I am currently looking into it. >>>>>> >>>>>> 2. Error: org.postgresql.util.PSQLException: FATAL: sorry, too many >>>>>> clients already. click on the link [6] to see the full log. >>>>>> >>>>>> *Solution*, Increase max_connections found in the postgresql.conf. >>>>>> *260* worked for me. >>>>>> Rational, each microservice can have up to 20 unique connections to >>>>>> the db per the BoneCP configuration. >>>>>> >>>>>> Also, you can test EclipseLink and MariaDB by pulling the FINCN-2 >>>>>> branch into fineract-cn-mariadb link [7] and rebuilding all of the >>>>>> microservices. >>>>>> >>>>>> *Identified Issue when using embedded dbs in full integration test * >>>>>> Tenant context not set, resulting in this error [8] >>>>>> >>>>>> Which brings me to my concern about MariaDB. Given that MariaDB is >>>>>> not compliant with Apache, can I safely assume that it's not relevant to >>>>>> migrate it's ORM as well? >>>>>> >>>>>> Looking forward to hearing from you all. >>>>>> >>>>>> *At your service,* >>>>>> Graham. >>>>>> >>>>>> [1] >>>>>> https://github.com/ebenezergraham/fineract-cn-postgresql/blob/FINCN-2/build.gradle#L76-L80 >>>>>> [2] >>>>>> https://github.com/ebenezergraham/fineract-cn-postgresql/blob/FINCN-2/build.gradle#L74 >>>>>> / https://www.baeldung.com/spring-eclipselink >>>>>> [3] >>>>>> https://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Advanced_JPA_Development/Performance/Weaving/Static_Weaving >>>>>> https://pastebin.com/FDAzd6bx >>>>>> [4] >>>>>> https://raw.githubusercontent.com/ebenezergraham/fineract-cn-demo-server/FINCN-2/scripts/Unix/eclipselink-test-setup.sh >>>>>> [5] https://pastebin.com/WFQ9satu >>>>>> [6] https://pastebin.com/M8GAHsit >>>>>> [7] >>>>>> https://github.com/ebenezergraham/fineract-cn-mariadb/tree/FINCN-2 >>>>>> [8] https://pastebin.com/FDAzd6bx >>>>>> >>>>>
