Hi, For a project where creation of Hadoop clusters needs to be fully automated, I utilized Ambari's blueprints and auto-discovery features, in addition to bootstraping hosts via REST instead of GUI (so the manual host registration is not necessary anymore).
I run into an issue where versions of services defined in stack used are ignored during cluster provisioning. For example, I specify "stack_name" as "HDP" and "stack_version" as "2.3" in blueprint I use, and "Spark" version associated with that stack is "1.4.1.2.3". However, once cluster is provisioned, even though http://[AmbariServerNode]:8080/#/main/admin/stack/services also shows "Spark" version "1.4.1.2.3" installed, when submitting a sample Spark job via Spark shell on client node, "Spark" version "1.5.2" is shown; `/usr/hdp/2.3.4.0-3485/spark/RELEASE` file on that node also shows "Spark" version "1.5.2.2.3.4.0". I got around this issue by changing the base URL for repository used to 'http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.2.0' (instead of "http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0";) prior to provisioning cluster from blueprint. I'm wondering whether this is something expected or if it's a bug, i.e. that service version defined in stack is ignored unless such change is made. Shouldn't services' versions defined in stack be respected and applied regardless of base URL? Perhaps I'm missing some other settings that should be applied in blueprint or cluster creation template, but AFAIK it's all as per docs/examples I read. Thank you for your input. Regards, Michal Siemaszko
