Yingyi Bu created ASTERIXDB-1341:
------------------------------------
Summary: Defer the file path decision into NC
Key: ASTERIXDB-1341
URL: https://issues.apache.org/jira/browse/ASTERIXDB-1341
Project: Apache AsterixDB
Issue Type: Improvement
Reporter: Yingyi Bu
Currently, the decision of storage file paths is made within the compiler,
e.g., in StoragePathUtil.java. It would be nice to defer the decision to NCs at
runtime. In this way, the index search, bulkload, insert/delete will have a
fixed degree-of-parallelism but in each NC they take file paths from
Dataset/Index lifecycle manager.
This will bring in the following benefits:
1. the degree of parallelism can be different from the number of file paths
that storage-related operators work with;
2. it avoids shipping all-file-paths (within the JobSpecification) to every
node for a query.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
