Xikui, If the number of localfs files is too large, a solution could be to put your files on HDFS and then load it. Loading from HDFS always has a fixed degree of parallelism regardless of the number of files.
>> I am wondering is there a way to append adm file to existed dataset? You can create an external dataset and then write an insert statement where the body is a scan query. AsterixDB doesn't load any data into its own storage for an external dataset but just keeps file paths. Here is a manual for external datasets: https://ci.apache.org/projects/asterixdb/aql/externaldata.html Best, Yingyi On Fri, Mar 4, 2016 at 11:47 AM, Xikui Wang <[email protected]> wrote: > Hi, > > I want to import data from multiple adm files into a same dataset. Merging > them together and then loading from localfs can be a viable solution, but > this may become a problem when the number become too large. I am wondering > is there a way to append adm file to existed dataset? > > Thank you. > > Best, > Xikui >
