[
https://issues.apache.org/jira/browse/GRIFFIN-333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160628#comment-17160628
]
Tushar commented on GRIFFIN-333:
--------------------------------
Hi [~obaid]
This is not regarding how much effort required to support this change. We
already have a Profiling dimension where you can model your use case so why you
want to add functionality that is already supported by Griffin.
Can you please also explain what additional steps you required to take to
support your usecase ?
> JDBC Connector: Ability to Use "group by" caluse
> ------------------------------------------------
>
> Key: GRIFFIN-333
> URL: https://issues.apache.org/jira/browse/GRIFFIN-333
> Project: Griffin
> Issue Type: Improvement
> Components: accuracy-batch
> Affects Versions: 0.6.0
> Reporter: Obaidul Karim
> Priority: Major
> Labels: column, groupby, jdbc
>
> *Background:*
> Refer to [https://issues.apache.org/jira/projects/GRIFFIN/issues/GRIFFIN-332].
> If we have the ability to select specific columns, it will open the door to
> use sql base aggregation, further reducing the volume of data from JDBC
> sources.
>
> *Proposed Improvement:*
> So, I propose the feature to allow JDBC connector to able to use sql based
> aggregations using clause `groupby`
> *Example:*
> Let's say we have source and target tables that have data like below.
> src:
> {code:java}
> ------------------------
> |employee_id |country|
> ------------------------
> |1 | NZ |
> |2 | DE |
> |3 | DE |
> |4 | NZ |
> |5 | DE |
> ....
> ....
> ------------------------
> {code}
> tgt:
> {code:java}
> ------------------------
> |total_employee|country|
> ------------------------
> |10 | NZ |
> |11 | DE |
> ------------------------
> {code}
> Then we can perform `accuracy` check [ `"rule":"src.total_employee =
> tgt.total_employee and src.country = tgt.country "` ] directly like below
> using `columns` and `groupby` clauses for source table:
> {code:java}
> {
> "name":"src",
> "connector":{
> "type":"jdbc",
> "config":{
> "database":"mydatabase",
> "tablename":"mytable",
> "columns":"count(*) total_employee, country",
> "groupby":"country",
> "url":"jdbc:sqlserver://myhost:1433;databaseName=mydatabase",
> "user":"user",
> "password":"password",
> "driver":"com.microsoft.sqlserver.jdbc.SQLServerDriver",
> "where":""
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
