[
https://issues.apache.org/jira/browse/SLIDER-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shanyu zhao updated SLIDER-1090:
--------------------------------
Attachment: SLIDER-1090.1.patch
[[email protected]] Thanks for reviewing, I have updated the patch to address
your comments.
Here is a few thoughts:
1) The JSON summary file is of "higher priority" over XML summary file. Just
like how AgentUtils read the metainfo file from the package, if a JSON exist,
the XML file will be ignored.
2) I also modified actionPackageInstall() so that at package installation, the
JSON/XML would be extracted and uploaded to HDFS, alongside with the package
file. I think we don't need to remove the metainfo file from the package, the
installation simply copy the JSON/XML file out to speed up cluster creation.
3) Not sure if you need to modify the documentation. The way how users write
the app package is unchanged. And for old installed application, Slider
continue to work because it falls back to the old way of reading metainfo files
from inside the package.
> Speed up SliderClient and AM by reading metainfo from accompany file
> --------------------------------------------------------------------
>
> Key: SLIDER-1090
> URL: https://issues.apache.org/jira/browse/SLIDER-1090
> Project: Slider
> Issue Type: Bug
> Affects Versions: Slider 0.81, Slider 0.90.2
> Reporter: shanyu zhao
> Attachments: SLIDER-1090.1.patch, SLIDER-1090.patch
>
>
> Today both SliderClient and SliderAM read package metainfo from HDFS. This
> requires downloading the whole package from HDFS. If package size is big
> (~200MB), this operation wastes tens of seconds in starting up the
> application.
> To speed up the operation of reading metainfo, we can read from an accompany
> xml/json file if it exists in the same HDFS folder as the zip package. If the
> accompany file does not exist, it falls back to the old way of downloading
> the zip package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
