[ https://issues.apache.org/jira/browse/AIRFLOW-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Davydov resolved AIRFLOW-118. --------------------------------- Resolution: Fixed > use targetPartitionSize as the default partition spec for HiveToDruidTransfer > operator > --------------------------------------------------------------------------------------- > > Key: AIRFLOW-118 > URL: https://issues.apache.org/jira/browse/AIRFLOW-118 > Project: Apache Airflow > Issue Type: Improvement > Components: operators > Reporter: Hongbo Zeng > > The definition of the two partition spec can be found > http://druid.io/docs/latest/ingestion/batch-ingestion.html. > Originally, the HiveToDruidTransfer uses numShards. The disadvantage of that > is users need to tune the numbers repeatedly, and do that again when the data > size changes. This is not scalable as the number of data sources grows. > targetPartitionSize approach calculates the number of segments automatically > and is hassle free. -- This message was sent by Atlassian JIRA (v6.3.4#6332)