dnskr opened a new pull request, #6046:
URL: https://github.com/apache/kyuubi/pull/6046
# :mag: Description
## Issue References ๐
The PR adds basic support for Hadoop configuration files to be used by
Apache Kyuubi, Apache Spark etc.
## Describe Your Solution ๐ง
The PR adds:
- Apache Hadoop related `ConfigMap` that has to be mounted to Kyuubi pods
as files.
- `HADOOP_CONF_DIR` environment variable to Kyuubi pods.
Notes:
- The `ConfigMap` change restarts Kyuubi pods!
- Only `core-site.xml` and `hdfs-site.xml` files are added. Not sure
`yarn-site.xml`, `mapred-site.xml` or other files should be added. Please, let
me know if so.
## Types of changes :bookmark:
<!--- What types of changes does your code introduce? Put an `x` in all the
boxes that apply: -->
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
Install the chart with configured Spark and the following values:
```yaml
# Hadoop configuration files
hadoopConf:
# The value (templated string) is used for core-site.xml file
# See
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
and Hadoop documentation for more details
coreSite: |
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>test.core.site</name>
<value>test-core-site</value>
</property>
</configuration>
# The value (templated string) is used for hdfs-site.xml file
# See
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
and Hadoop documentation for more details
hdfsSite: |
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>test.hdfs.site</name>
<value>test-hdfs-site</value>
</property>
</configuration>
```
Run `beeline` from Kyuubi pod:
```shell
./bin/beeline -u 'jdbc:hive2://kyuubi-thrift-binary:10009' -n test-user
```
Check specified property values:
```shell
0: jdbc:hive2://kyuubi-thrift-binary:10009> set test.core.site;
+-----------------+-----------------+
| key | value |
+-----------------+-----------------+
| test.core.site | test-core-site |
+-----------------+-----------------+
0: jdbc:hive2://kyuubi-thrift-binary:10009> set test.hdfs.site;
+-----------------+-----------------+
| key | value |
+-----------------+-----------------+
| test.hdfs.site | test-hdfs-site |
+-----------------+-----------------+
```
---
# Checklist ๐
<!--- Go over all the following points, and put an `x` in all the boxes that
apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
