I just feel we don't have developers resource to add something big then
maintain the packaging, manifests and smoke-test in the long term.
Actually current active committers do not have enough cycle even for
maintaining existing products in the stack.
Contributors are necessarily interested in only the products and
platforms they are using.
We just need someone continuously taking care of all of iceberg-flink,
iceberg-hive and iceberg-spark
for both RPM and DEB.
On 2023/09/05 17:56, Battula, Brahma Reddy wrote:
Thanks murali and Masatake.
Do we foresee any issue's if we maintain the iceberg in bigtop stack like
upgrades..?
Any idea, how others are adopted this..?
On 10/08/23, 7:16 AM, "Masatake Iwasaki" <iwasak...@oss.nttdata.com
<mailto:iwasak...@oss.nttdata.com>> wrote:
Looking at the packaging perspective the efforts should be minimal. At
present, Hive and Spark are the only Bigtop components that can support
iceberg, and only a single jar is required to be packaged into the
respective component lib directories.
I guess the versions of Flink, Spark and Hive matters.
https://github.com/apache/iceberg/blob/apache-iceberg-1.3.1/settings.gradle
<https://github.com/apache/iceberg/blob/apache-iceberg-1.3.1/settings.gradle>
If we package the Iceberg, the jars should be installed under /usr/lib/iceburg.
The jar should be added to the classpaths by settings of depending products.
We need Puppet manifests for that to deploy testing environment.
Smoke tests should make it certain that the combination works and
the jar in the classpath does not bring conflicting dependencies.
On 2023/08/09 16:28, Muralikrishna Dmmkr wrote:
Hi All,
I noticed that there have been various discussions within Bigtop and Ambari
communities regarding the addition of Iceberg support, Unfortunately these
have not reached a conclusion yet.
So, I am initiating this thread to discuss on the support for iceberg.
Looking at the packaging perspective the efforts should be minimal. At
present, Hive and Spark are the only Bigtop components that can support
iceberg, and only a single jar is required to be packaged into the
respective component lib directories.
Please share your thoughts on this.
Thanks
Murali Krishna
