turbaszek commented on pull request #8: URL: https://github.com/apache/kibble/pull/8#issuecomment-851616956
> how do you see this working for example for our project mailing lists? Would each list the be a datasource and the conversations the datatype? That's a very good question @sharanf! I would lean to what you've written. Datasource does not only represent an "external service" entity but "account/organization within an external service". So, in case of mailing list each Apache project would required configuring their own data source. For example: ```yaml data_sources: - name: asf_mails_kibble class: kibble.data_sources.pony_mail.PonyMailDataSource config: project_name: kibble enabled_data_types: - mails - name: asf_mails_kafka class: kibble.data_sources.pony_mail.PonyMailDataSource config: project_name: kafka enabled_data_types: - mails - name: asf_mails_pulsar class: kibble.data_sources.pony_mail.PonyMailDataSource config: project_name: pulsar enabled_data_types: - mails ``` While there's a bit of duplication in configuration it allow more granularity. In case of ASF the config will be big and repeatable but for smaller Kibble deployments it would be smaller and more configuration maybe an advantage. Additionally this additional granularity is useful in case of sources that need authorisation. In such cases we may want to store the credentials in different way or use different auth methods. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org