There is a way to get the same end result by using plugins. It will be
good to understand why plugin can't be used and can they be extended to
provide the required functionality.
Thank you,
Vlad
On 1/29/18 15:14, Sergey Golovko wrote:
Hello All,
In Apex there are two ways to deploy non-Hadoop jars to the deployed
cluster.
The first approach is static (hardcoded) and it is used by Apex platform
developers only. There are several final static arrays of Java classes
in StramClient.java
that define which of the available jars should be included into deployment
for every Apex application.
The second approach is to add paths of all dependent jar-files to the value
of the attribute LIB_JARS. The end-user can set/update the value of the
attribute LIB_JARS via dt-site.xml files, command line parameters,
application properties and plugins. The usage of the
attribute LIB_JARS is the official documented way for all Apex users to
manage by the deployment jars.
But some of the dependent jars (not from the Apex core) can be common for
all customer's applications for a specific installation and/or execution
environment. Unfortunately the Apex implementation does not contain the
middle solution that would allow the Apex developers and customer support to
define and add new dependent jar-files (jars that should not be
configurable/managed by the end-user) without the updates/recompilation of
the Apex Java code during the Apex building process and/or
installation/configuration.
Also the having of such kind of flexibility would allow the Apex core
developers to use Java interfaces during the development to define an
abstraction layer in Apex implementation and configurate Apex core to add
some specific jars to all Apex applications without recompilation of the
Apex source code.
For instance, now the usage of HDFS is hardcoded in Apex platform code but
it can be replaced with any other distributed or cloud base file system.
The Apex core code can use an interface for all I/O operations but the
supporting of a real specific file system implementation can be added as an
independent jar-file. Or if the implementation of some of Apex operators
depend on a specific service, and it is necessary to add some of the
service jars to every Apex application implicitly.
The proposal:
- add a predefined configuration text file (we can make any choice for the
file syntax: XML, JSON or Properties) to Apex engine resources with
predefined values of some of the Apex attributes (now we can include LIB_JARS
attribute only);
- allow to have a configuration text file with the same functionality in
the Apex installation folder "conf";
- read the content of the predefined configuration text files by the stram
client in runtime and add the jars to the list of the dependent jars;
- allow to use paths to jars and Java classes to refer to the dependent
jars (the references can have the extensions: .class and .jar).
Thanks,
Sergey
