airajena opened a new pull request, #29:
URL: https://github.com/apache/fineract-business-intelligence/pull/29
## Summary
This PR introduces a two-job GitHub Actions CI workflow for the
fineract-business-intelligence project.
### Jobs
**compilation-checks** (no Docker, ~1 min)
- Python syntax check for extractor, bootstrap, and utility scripts
- `dbt parse` to validate all SQL models and ref() calls without a
database connection
**integration-test** (full stack, ~20 min)
- Spins up fineract-db, warehouse, extractor, dbt, and superset via
Docker Compose
- Seeds the Fineract source schema (schema_fineract_source.sql) since
Liquibase does not run in CI
- Creates bi_connector_source compatibility views matching the
bootstrap_source.sh definitions (epoch fallbacks for tables that
lack audit columns in real Fineract schema)
- Runs the full ELT backfill and all dbt models and tests
- Asserts 12 smoke tests:
1. PAR cumulative invariant: PAR 30 >= PAR 60 >= PAR 90 always
2. standard_par_band only contains expected classification values
3. Every office+product+date combo has an All Portfolio rollup row
4. No mart rows with an office_id absent from dim_office
5. No gaps in the daily snapshot series within each office+product range
6. NPA amount never exceeds total portfolio amount
7. All PAR ratios are in the range [0, 1]
8. Portfolio component amounts never exceed gross loan portfolio
9. At least one row with disbursed_amount_on_date > 0 (pipeline wrote data)
10. Latest snapshot contains >= 3 distinct offices
11. No snapshot dates beyond today
12. Superset /health endpoint returns OK
- Uploads Docker container logs as an artifact on failure
- Tears down the stack (without -v to preserve named volumes)
### Supporting changes
- `warehouse/seed/schema_fineract_source.sql`: schema-only DDL for the
12 Fineract source tables the extractor reads from. Required because
Liquibase migrations do not run in CI (no Fineract Java application).
Includes a head office seed row required by branch office FK constraints.
- `compose.yaml`: fix fineract_db_data volume mount path from
`/var/lib/postgresql/data` to `/var/lib/postgresql` as required by
postgres 18+ which stores data in a version-specific subdirectory.
- `dbt/profiles.yml`: add fallback defaults (`'ci'`) for PGUSER and
PGPASSWORD so `dbt parse` succeeds on runners where PG env vars are
not set.
### Notes
- The integration-test job requires `warehouse/seed/seed_fineract_source.sql`
to be present (procedural seed with 80 clients and 144 loans). This file
lives on the refactor/charts branch and must be merged before the
integration-test job is fully green end-to-end.
- Teardown uses `docker compose down` without `-v` intentionally to avoid
deleting named volumes on self-hosted runners.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
