turbaszek commented on pull request #8:
URL: https://github.com/apache/kibble/pull/8#issuecomment-851057538
@kaxil @sharanf @Humbedooh @michalslowikowski00 happy to get your opinion.
As suggested on the dev list I introduced concept of `DataSource` and
`DataType`. Those for now can be configured yaml configuration file:
```yaml
data_sources:
- name: github_kibble
class: kibble.data_sources.github.GithubDataSource
config:
repo_owner: apache
repo_name: kibble
enabled_data_types:
- pr_issues
```
This form allow users to specify any external data sources as long as the
class path points to importable object.
The role of `DataSource` is to provide authentication methods for the
external service represented by it. `DataType` represent single type of
information we can get from this source, in case of this PR those are Github
issues (which include also PRs). Role of `DataType` is to define :
- how to process the raw data from external source and how to persist them
into database (to be done)
- how to read the data from database including aggregation, filters etc.
In general this is rough idea I have in m mind:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
