[
https://issues.apache.org/jira/browse/FALCON-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sandeep samudrala reassigned FALCON-1368:
-----------------------------------------
Assignee: sandeep samudrala
> Improve Falcon server restart time
> -----------------------------------
>
> Key: FALCON-1368
> URL: https://issues.apache.org/jira/browse/FALCON-1368
> Project: Falcon
> Issue Type: Improvement
> Reporter: Ajay Yadava
> Assignee: sandeep samudrala
>
> Currently on restart, Falcon server loads all the entities from HDFS one by
> one. In a large set up like the one at Inmobi, where we have several
> thousands of feeds and processes, this takes several minutes to increase the
> start up time.
> Since this is an IO intensive task(reading a file from HDFS into memory),
> having multiple threads to load entities in parallel will improve the start
> up time of the server.
> Two points need to be taken care of:
> 1. Only a single type of entities should be load in parallel to preserve the
> order of load of entities.
> 2. Currently Falcon server fails to start if there is an error in loading any
> entity, it will be slightly tricky but we should preserve the same behaviour
> in case of parallel uploads via threads.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
