kevinw66 commented on code in PR #40: URL: https://github.com/apache/bigtop-manager/pull/40#discussion_r1714569381
########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/metainfo.xml: ########## @@ -0,0 +1,63 @@ +<?xml version="1.0"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one + ~ or more contributor license agreements. See the NOTICE file + ~ distributed with this work for additional information + ~ regarding copyright ownership. The ASF licenses this file + ~ to you under the Apache License, Version 2.0 (the + ~ "License"); you may not use this file except in compliance + ~ with the License. You may obtain a copy of the License at + ~ + ~ https://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, + ~ software distributed under the License is distributed on an + ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + ~ KIND, either express or implied. See the License for the + ~ specific language governing permissions and limitations + ~ under the License. +--> +<metainfo> + <schema-version>2.0</schema-version> + <service> + <name>solr</name> + <display-name>Solr</display-name> + <desc> + Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene. + </desc> + <version>8.11.2</version> + <user>solr</user> + <group>solr</group> + <components> + <component> + <name>SOLR</name> Review Comment: Make sure component name is lowercase and similar to display name, it should be `solr_instance` here ########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/metainfo.xml: ########## @@ -0,0 +1,63 @@ +<?xml version="1.0"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one + ~ or more contributor license agreements. See the NOTICE file + ~ distributed with this work for additional information + ~ regarding copyright ownership. The ASF licenses this file + ~ to you under the Apache License, Version 2.0 (the + ~ "License"); you may not use this file except in compliance + ~ with the License. You may obtain a copy of the License at + ~ + ~ https://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, + ~ software distributed under the License is distributed on an + ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + ~ KIND, either express or implied. See the License for the + ~ specific language governing permissions and limitations + ~ under the License. +--> +<metainfo> + <schema-version>2.0</schema-version> + <service> + <name>solr</name> + <display-name>Solr</display-name> + <desc> + Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene. + </desc> + <version>8.11.2</version> + <user>solr</user> + <group>solr</group> + <components> + <component> + <name>SOLR</name> + <display-name>Solr Instance</display-name> + <category>MASTER</category> + <cardinality>1+</cardinality> + <command-script> + <script-id>org.apache.bigtop.manager.stack.bigtop.v3_3_0.solr.SolrServerScript</script-id> + <script-type>java</script-type> + <timeout>1200</timeout> + </command-script> Review Comment: Could you add quicklinks here? you can refers to [hdfs](https://github.com/apache/bigtop-manager/blob/main/bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/hdfs/metainfo.xml#L58) config ########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/configuration/solr-log4j.xml: ########## @@ -0,0 +1,100 @@ +<?xml version="1.0"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one + ~ or more contributor license agreements. See the NOTICE file + ~ distributed with this work for additional information + ~ regarding copyright ownership. The ASF licenses this file + ~ to you under the Apache License, Version 2.0 (the + ~ "License"); you may not use this file except in compliance + ~ with the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, + ~ software distributed under the License is distributed on an + ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + ~ KIND, either express or implied. See the License for the + ~ specific language governing permissions and limitations + ~ under the License. +--> +<configuration> + <property> + <name>log_maxfilesize</name> + <value>10</value> + <description>The maximum size of backup file before the log is rotated</description> + <display-name>Solr Log: backup file size</display-name> + </property> + + <property> + <name>log_maxbackupindex</name> + <value>9</value> + <description>The number of backup files</description> + <display-name>Solr Log: # of backup files</display-name> + </property> + <property> + <name>content</name> + <display-name>solr-log4j template</display-name> + <description>This is the jinja template for log4j2.xml</description> Review Comment: Also freemarker here ########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/metainfo.xml: ########## @@ -0,0 +1,63 @@ +<?xml version="1.0"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one + ~ or more contributor license agreements. See the NOTICE file + ~ distributed with this work for additional information + ~ regarding copyright ownership. The ASF licenses this file + ~ to you under the Apache License, Version 2.0 (the + ~ "License"); you may not use this file except in compliance + ~ with the License. You may obtain a copy of the License at + ~ + ~ https://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, + ~ software distributed under the License is distributed on an + ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + ~ KIND, either express or implied. See the License for the + ~ specific language governing permissions and limitations + ~ under the License. +--> +<metainfo> + <schema-version>2.0</schema-version> + <service> + <name>solr</name> + <display-name>Solr</display-name> + <desc> + Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene. + </desc> + <version>8.11.2</version> Review Comment: We should align to bigtop's version, which contains the release version, it should be 8.11.2-2 here ########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/configuration/solr-env.xml: ########## @@ -0,0 +1,295 @@ +<?xml version="1.0"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one + ~ or more contributor license agreements. See the NOTICE file + ~ distributed with this work for additional information + ~ regarding copyright ownership. The ASF licenses this file + ~ to you under the Apache License, Version 2.0 (the + ~ "License"); you may not use this file except in compliance + ~ with the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, + ~ software distributed under the License is distributed on an + ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + ~ KIND, either express or implied. See the License for the + ~ specific language governing permissions and limitations + ~ under the License. +--> +<configuration> + <property> + <name>solr_port</name> + <value>8983</value> + <description>Solr port</description> + </property> + + <property> + <name>solr_jmx_port</name> + <value>18983</value> + <description>Solr JMX port</description> + </property> + + <property> + <name>solr_pid_dir</name> + <value>/var/run/solr</value> + <description>Solr Process ID Directory</description> + </property> + + <property> + <name>solr_log_dir</name> + <value>/var/log/solr</value> + <description>Directory for Solr logs</description> + </property> + + <property> + <name>solr_user</name> + <value>solr</value> + <description>Solr user</description> + </property> + + <property> + <name>solr_datadir</name> + <value>/var/lib/solr/data</value> + <description>Directory for storting Solr index. Make sure you have enough disk space</description> + </property> + + <property> + <name>solr_ssl_enabled</name> + <value>false</value> + <description>Enable ssl to Solr</description> + </property> + + <property> + <name>solr_truststore_location</name> + <value>/etc/security/serverKeys/infra.solr.trustStore.jks</value> + <description>Location of the trust store file. (default value is not generated)</description> + </property> + + <property> + <name>solr_truststore_type</name> + <value>jks</value> + <description>Type of the trust store file.</description> + </property> + + <property> + <name>solr_truststore_password</name> + <value>bigdata</value> + <description>Password to open the trust store file.</description> + </property> + + <property> + <name>solr_keystore_location</name> + <value>/etc/security/serverKeys/infra.solr.keyStore.jks</value> + </property> + + <property> + <name>solr_keystore_type</name> + <value>jks</value> + </property> + + <property> + <name>solr_keystore_password</name> + <value>bigdata</value> + <description>Password to open the key store file.</description> + </property> + + <property> + <name>solr_znode</name> + <value>/solr</value> + <description>Zookeeper znode, e.g: /solr</description> + </property> + + <property> + <name>solr_minmem</name> + <value>1024</value> + <description>Solr minimum heap size e.g. 512m</description> + </property> + + <property> + <name>solr_maxmem</name> + <value>2048</value> + <description>Solr maximum heap size e.g. 512m</description> + </property> + + <property> + <name>solr_java_stack_size</name> + <value>1</value> + <description>Java Stack Size of Solr (-Xss) in MB.</description> + </property> + + <property> + <name>solr_jmx_enabled</name> + <value>false</value> + <description>Set to true to activate the JMX RMI connector to allow remote JMX client applications to monitor the JVM hosting Solr + </description> + </property> + + <property> + <name>solr_kerberos_keytab</name> + <value>/etc/security/keytabs/solr.service.keytab</value> + <description>The path to the Kerberos Keytab file containing service principal of the Solr.</description> + </property> + + <property> + <name>solr_kerberos_principal</name> + <value>solr</value> + <description>The service principal for Solr.</description> + </property> + + <property> + <name>solr_web_kerberos_keytab</name> + <value>/etc/security/keytabs/spnego.service.keytab</value> + <description>The path to the Kerberos Keytab file containing service principal of the Solr.</description> + </property> + + <property> + <name>solr_web_kerberos_principal</name> + <value>HTTP/[email protected]</value> + <description>The service principal for the Solr.</description> + </property> + + <property> + <name>solr_zookeeper_quorum</name> + <value><![CDATA[<#if zookeeper_quorum?? ><#list zookeeper_quorum as host>${host}:2181<#sep>,</#sep></#list><#else>localhost:2181</#if>]]></value> + <description>Placeholder for Solr Zookeeper connection string. (Use the cluster one by default, you can override this with a custom one if ZK needs to be external)</description> + </property> + <property> + <name>solr_host</name> + <value>${host}</value> + <description>Placeholder for Solr Zookeeper connection string. (Use the cluster one by default, you can override this with a custom one if ZK needs to be external)</description> + </property> + + <property> + <name>solr_zookeeper_external_principal</name> + <value>zookeeper/[email protected]</value> + <description>The kerberos service principal name for external ZooKeeper.</description> + </property> + + <property> + <name>solr_zookeeper_external_enabled</name> + <value>false</value> + <description>Enable external ZooKeeper. If the Solr is secure, the external ZK should be secure as well.</description> + </property> + + <property> + <name>solr_kerberos_name_rules</name> + <value>DEFAULT</value> + <description>Kerberos name rules for Spnego</description> + </property> + + <property> + <name>solr_user_nofile_limit</name> + <value>128000</value> + <description>Max open files limit setting for solr user.</description> + </property> + + <property> + <name>solr_user_nproc_limit</name> + <value>65536</value> + <description>Max number of processes limit setting for solr user.</description> + </property> + + <property> + <name>solr_extra_java_opts</name> + <value></value> + <description>Extra Solr java options (e.g.: -Dproperty=value), that will be added to SOLR_OPTS environment variable</description> + </property> + + <property> + <name>solr_gc_log_opts</name> + <value></value> + </property> + <property> + <name>solr_gc_tune</name> + <value></value> + </property> + <property> + <name>solr_min_mem</name> + <value>1024</value> + </property> + <property> + <name>solr_max_mem</name> + <value>2048</value> + </property> + <property> + <name>security_enabled</name> + <value>false</value> + </property> + + <!-- solr-env.sh --> + <property> + <name>content</name> + <display-name>solr-env template</display-name> + <description>This is the jinja template for solr-env.sh file</description> Review Comment: We are using freemarker template, not jinja ########## bigtop-manager-stack/bigtop-manager-stack-bigtop/src/main/java/org/apache/bigtop/manager/stack/bigtop/v3_3_0/solr/SolrParams.java: ########## @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.bigtop.manager.stack.bigtop.v3_3_0.solr; + +import lombok.Getter; +import org.apache.bigtop.manager.common.message.entity.payload.CommandPayload; +import org.apache.bigtop.manager.stack.common.annotations.GlobalParams; +import org.apache.bigtop.manager.stack.common.utils.BaseParams; +import org.apache.bigtop.manager.stack.common.utils.LocalSettings; +import lombok.extern.slf4j.Slf4j; + + +import java.text.MessageFormat; +import java.util.List; +import java.util.Map; + +@Getter +@Slf4j +public class SolrParams extends BaseParams { + + private String SOLR_PORT = "8983"; + private String solrLogDir = "/var/log/solr"; + private String solrPidDir = "/var/run/solr"; + private String solrHomeDir = "/var/lib/solr"; + private String solrDataDir = "/var/lib/solr/data"; + private String solrPidFile = solrPidDir + "/solr-" + SOLR_PORT+ ".pid"; + + public SolrParams(CommandPayload commandPayload) { + super(commandPayload); + globalParamsMap.put("java_home", "/usr/local/java"); + globalParamsMap.put("solr_home", solrHomeDir); + globalParamsMap.put("security_enabled", false); + globalParamsMap.put("solr_pid_file", solrPidFile); + globalParamsMap.put("yarn_user", user()); + globalParamsMap.put("yarn_group", group()); + } + + @GlobalParams + public Map<String, Object> solrxml() { + return LocalSettings.configurations(serviceName(), "solr-xml"); + } + @GlobalParams + public Map<String, Object> solrLog4j() { + return LocalSettings.configurations(serviceName(), "solr-log4j"); + } + public String get_znode(){ + Map<String, Object> solrEnv = LocalSettings.configurations(serviceName(), "solr-env"); + String znode = (String) solrEnv.get("solr_znode"); + return znode; + } + public String get_zkstring(){ + List<String> zookeeperServerHosts = LocalSettings.hosts("zookeeper_server"); + Map<String, Object> zkport = LocalSettings.configurations("zookeeper", "zoo.cfg"); + String clientPort = (String) zkport.get("clientPort"); + String solrzkstring = MessageFormat.format("{0}:{1}", zookeeperServerHosts.get(0),clientPort); Review Comment: Both method name and variable name should be camelcase ########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/metainfo.xml: ########## @@ -0,0 +1,63 @@ +<?xml version="1.0"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one + ~ or more contributor license agreements. See the NOTICE file + ~ distributed with this work for additional information + ~ regarding copyright ownership. The ASF licenses this file + ~ to you under the Apache License, Version 2.0 (the + ~ "License"); you may not use this file except in compliance + ~ with the License. You may obtain a copy of the License at + ~ + ~ https://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, + ~ software distributed under the License is distributed on an + ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + ~ KIND, either express or implied. See the License for the + ~ specific language governing permissions and limitations + ~ under the License. +--> +<metainfo> + <schema-version>2.0</schema-version> + <service> + <name>solr</name> + <display-name>Solr</display-name> + <desc> + Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene. + </desc> + <version>8.11.2</version> + <user>solr</user> + <group>solr</group> + <components> + <component> + <name>SOLR</name> + <display-name>Solr Instance</display-name> + <category>MASTER</category> Review Comment: Also lowercase here ########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/configuration/solr-env.xml: ########## Review Comment: Xml file should also be indent with 4 spaces ########## bigtop-manager-server/src/main/resources/stacks/bigtop/3.3.0/services/solr/order.json: ########## @@ -0,0 +1,3 @@ +{ + "SOLR-START" : ["ZOOKEEPER_SERVER-START"] Review Comment: Also need to change to SOLR_INSTANCE-START -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
