xihong08 commented on issue #1140: URL: https://github.com/apache/incubator-pegasus/issues/1140#issuecomment-1240268769
> > Thanks for your bug report~ > > > `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` ‘s order are not guaranteed. > > > > > > Function `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` are execute in thread pool `THREAD_POOL_META_STATE`, which is single thread in current pegasus design. As a result, the order can be guaranteed, `update_partition_metadata_on_remote_storage` will always execute before `update_partition_info_on_remote_storage`. > > However, in current bulk load design, bulk load will be failed if there are some partitions don't have data. For example, if you have a table will 16 partitions, and partition[0] doesn't have data to be ingested, bulk load will be failed. > > I suggest: > > > > 1. check your configuration whether threadpool THREAD_POOL_META_STATE is single thread > > 2. check if all partitions have data to be ingested. THREAD_POOL_META_STATE has 2 threads all partions have data need to ingested update partition's metadata failed is my issue。like this, my partition's data has 30M, but when start ingest prompte 'partition doesn't have bulk load metadata, set bulk load failed'. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
