[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771040#comment-15771040 ] John Zhuge edited comment on HADOOP-13597 at 12/22/16 8:42 PM: --- Patch 006 - Xiao's comments TESTING DONE - Automated regression tests for KMS https://github.com/jzhuge/hadoop-regression-tests in unsecure and ssl mode - View docs was (Author: jzhuge): Patch 006 - Xiao's comments TESTING DONE - Automated regression tests for KMS and HttpFS https://github.com/jzhuge/hadoop-regression-tests in unsecure and ssl mode - View docs > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, > HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, > HADOOP-13597.006.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742931#comment-15742931 ] Allen Wittenauer edited comment on HADOOP-13597 at 12/12/16 7:48 PM: - {code} + hadoop_deprecate_envvar CATALINA_OUT + hadoop_deprecate_envvar CATALINA_PID + hadoop_deprecate_envvar KMS_ADMIN_PORT + hadoop_deprecate_envvar KMS_CATALINA_HOME + hadoop_deprecate_envvar KMS_SSL_TRUSTSTORE_PASS {code} We don't do this anywhere in the scripts. Instead, this is documented in the release notes. It's just extra console noise otherwise. {code} + hadoop_using_envvar KMS_HOME {code} This doesn't appear to have actually been configurable by users. I don't see a reason to add it now. {code} + hadoop_using_envvar KMS_HTTP_PORT + hadoop_using_envvar KMS_LOG + hadoop_using_envvar KMS_MAX_HTTP_HEADER_SIZE + hadoop_using_envvar KMS_MAX_THREADS + hadoop_using_envvar KMS_SSL_ENABLED + hadoop_using_envvar KMS_SSL_KEYSTORE_FILE + hadoop_using_envvar KMS_SSL_KEYSTORE_PASS + hadoop_using_envvar KMS_TEMP {code} I know that branch-2 spit out a bunch of stuff, but it always felt wrong. Is this actually valuable to anyone that aren't developers? Would \-\-debug be a better usage here? It seems like a lot of noise on the console that's probably more appropriate for a log file {code} + hadoop_using_envvar KMS_SSL_KEYSTORE_PASS {code} (!) Is this actually printing a password to the screen?!? Is there any chance we can switch this to being read from a file? env vars are exposed in /proc on some OSes... {code} hadoop_mkdir {code} We have a bunch of places where this same construct is being used. We should probably replace all of them if we're going to add a function to do it. FWIW, I definitely prefer the single function for handling kms. So Much Better. (and I'm really ecstatic of dropping kms-config.sh , etc, etc.) was (Author: aw): {code} + hadoop_deprecate_envvar CATALINA_OUT + hadoop_deprecate_envvar CATALINA_PID + hadoop_deprecate_envvar KMS_ADMIN_PORT + hadoop_deprecate_envvar KMS_CATALINA_HOME + hadoop_deprecate_envvar KMS_SSL_TRUSTSTORE_PASS {code} We don't do this anywhere in the scripts. Instead, this is documented in the release notes. It's just extra console noise otherwise. {code} + hadoop_using_envvar KMS_HOME {code} This doesn't appear to have actually be configurable by users. I don't see a reason to add it now. {code} + hadoop_using_envvar KMS_HTTP_PORT + hadoop_using_envvar KMS_LOG + hadoop_using_envvar KMS_MAX_HTTP_HEADER_SIZE + hadoop_using_envvar KMS_MAX_THREADS + hadoop_using_envvar KMS_SSL_ENABLED + hadoop_using_envvar KMS_SSL_KEYSTORE_FILE + hadoop_using_envvar KMS_SSL_KEYSTORE_PASS + hadoop_using_envvar KMS_TEMP {code} I know that branch-2 spit out a bunch of stuff, but it always felt wrong. Is this actually valuable to anyone that aren't developers? Would \-\-debug be a better usage here? It seems like a lot of noise on the console that's probably more appropriate for a log file {code} + hadoop_using_envvar KMS_SSL_KEYSTORE_PASS {code} (!) > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, > HADOOP-13597.003.patch, HADOOP-13597.004.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733584#comment-15733584 ] John Zhuge edited comment on HADOOP-13597 at 12/8/16 10:27 PM: --- However, do realize that the last {{elif}} condition can be omitted and changed to {{else}}: {code} if [[ -z "${oldval}" ]]; then return elif [[ -z "${newvar}" ]]; then hadoop_error "WARNING: ${oldvar} has been deprecated." elif [[ -n "${oldval}" && -n "${newvar}" ]]; then {code} was (Author: jzhuge): However, do realize that the following {{elif}} condition can be omitted and changed to {{else}}: {code} elif [[ -n "${oldval}" && -n "${newvar}" ]]; then {code} > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, > HADOOP-13597.003.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717586#comment-15717586 ] John Zhuge edited comment on HADOOP-13597 at 12/3/16 7:03 AM: -- Could have used [Decorator Pattern|https://en.wikipedia.org/wiki/Decorator_pattern] to design a wrapper to log configuration access: {code} interface ConfigurationAccess { String get(String name); String getInt(String name, int defaultValue); } class Configuration implements ConfigurationAccess { ... } class LoggedConfigurationAccess implements ConfigurationAccess { LoggedConfigurationAccess(conf, log) { this.conf = conf; this.log = log; } String get(String name) { String value = conf.get(name); log.info(..., name, value); return value; } String getInt(String name, int defaultValue) { String value = conf.getInt(name, defaultValue); log.info(..., name, value, defaultValue); return value; } } {code} A little downsize though: {{LoggedConfigurationAccess#getInt}} will log 2 messages because {{Configuration#getInt}} calls {{LoggedConfigurationAccess#get}}. was (Author: jzhuge): Could have used [Decorator Pattern|https://en.wikipedia.org/wiki/Decorator_pattern] to design a wrapper to log configuration access: {code} interface ConfigurationAccess { String get(String name); String getInt(String name, int defaultValue); } class Configuration implements ConfigurationAccess { ... } class LoggedConfigurationAccess { LoggedConfigurationAccess(conf, log) { this.conf = conf; this.log = log; } String get(String name) { String value = conf.get(name); log.info(..., name, value); return value; } String getInt(String name, int defaultValue) { String value = conf.getInt(name, defaultValue); log.info(..., name, value, defaultValue); return value; } } {code} A little downsize though: {{LoggedConfigurationAccess#getInt}} will log 2 messages because {{Configuration#getInt}} calls {{LoggedConfigurationAccess#get}}. > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715569#comment-15715569 ] John Zhuge edited comment on HADOOP-13597 at 12/2/16 5:31 PM: -- [~aw] Found mistake in {{hadoop-kms.sh}} for Patch 002 where everything is in {{hadoop_subcommand_kms}}. Should be: {code} if [[ "${HADOOP_SHELL_EXECNAME}" = hadoop ]]; then hadoop_add_profile kms hadoop_add_subcommand "kms" "run KMS, the Key Management Server" fi function _kms_hadoop_init { # init variables } function hadoop_subcommand_kms { # Called by bin/hadoop to provide subcommand case statement if any HADOOP_SUBCMD_SUPPORTDAEMONIZATION=true HADOOP_CLASSNAME=org.apache.hadoop.crypto.key.kms.server.KMSHttpServer } function _kms_hadoop_finalize { # Called in finalize phase, all env vars are settled hadoop_add_param HADOOP_OPTS "-Dkms.config.dir=" \ "-Dkms.config.dir=${HADOOP_CONF_DIR}" hadoop_add_param HADOOP_OPTS "-Dkms.log.dir=" \ "-Dkms.log.dir=${HADOOP_LOG_DIR}" } {code} The 3 functions are called in this order: # _kms_hadoop_init # hadoop_subcommand_kms # _kms_hadoop_finalize was (Author: jzhuge): [~aw] Found mistake in {{hadoop-kms.sh}} for Patch 002 where everything is in {{hadoop_subcommand_kms}}. Should be: {code} if [[ "${HADOOP_SHELL_EXECNAME}" = hadoop ]]; then hadoop_add_profile kms hadoop_add_subcommand "kms" "run KMS, the Key Management Server" fi function _kms_hadoop_init { # init variables } function hadoop_subcommand_kms { # Called by bin/hadoop to provide subcommand case statement if any HADOOP_SUBCMD_SUPPORTDAEMONIZATION=true HADOOP_CLASSNAME=org.apache.hadoop.crypto.key.kms.server.KMSHttpServer } function _kms_hadoop_finalize { # Called in finalize phase, all env vars are settled hadoop_add_param HADOOP_OPTS "-Dkms.config.dir=" \ "-Dkms.config.dir=${HADOOP_CONF_DIR}" hadoop_add_param HADOOP_OPTS "-Dkms.log.dir=" \ "-Dkms.log.dir=${HADOOP_LOG_DIR}" } {code} The 3 functions are called in this order: # _kms_hadoop_init # hadoop_subcommand_kms # _kmd_hadoop_finalize > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715747#comment-15715747 ] John Zhuge edited comment on HADOOP-13597 at 12/2/16 5:30 PM: -- {{hadoop_add_param HADOOP_OPTS "-Dkms.config.dir=" ...}} may be called too early. Since {{hadoop_subcommand_opts}} is called between {{hadoo_subcommand_kms}} and {{_kms_hadoop_finalize}}, in all-in-one approach, {{HADOOP_KMS_OPTS=-Dkms.config.dir=..}} would not have taken effect. was (Author: jzhuge): {{hadoop_add_param HADOOP_OPTS "-Dkms.config.dir=" "-Dkms.config.dir=${HADOOP_CONF_DIR}"}} may be called too early. Since {{hadoop_subcommand_opts}} is called between {{hadoo_subcommand_kms}} and {{_kms_hadoop_finalize}}, in all-in-one approach, {{HADOOP_KMS_OPTS=-Dkms.config.dir=..}} would not have taken effect. > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704129#comment-15704129 ] John Zhuge edited comment on HADOOP-13597 at 12/1/16 4:37 PM: -- Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer thus all KMS unit tests exercise KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete KMSJMXServlet. HttpServer2 uses compatible org.apache.hadoop.jmx.JMXJsonServlet with more features. - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway - Obsolete {{kms.sh version}} that prints Tomcat version TESTING DONE - All hadoop-kms unit tests. MiniKMS equals full KMS. - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status - JMX works - /logs works TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 was (Author: jzhuge): Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer thus all KMS unit tests exercise KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway - Obsolete {{kms.sh version}} that prints Tomcat version TESTING DONE - All hadoop-kms unit tests. MiniKMS equals full KMS. - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status - JMX works - /logs works TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704129#comment-15704129 ] John Zhuge edited comment on HADOOP-13597 at 11/30/16 8:20 AM: --- Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer thus all KMS unit tests exercise KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway - Obsolete {{kms.sh version}} that prints Tomcat version TESTING DONE - All hadoop-kms unit tests. MiniKMS equals full KMS. - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status - JMX works - /logs works TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 was (Author: jzhuge): Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway - Obsolete {{kms.sh version}} that prints Tomcat version TESTING DONE - All hadoop-kms unit tests. MiniKMS equals full KMS. - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status - JMX works - /logs works TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704286#comment-15704286 ] John Zhuge edited comment on HADOOP-13597 at 11/30/16 2:58 AM: --- Map {{sbin/kms.sh}} to {{bin/hadoop kms}}: | kms.sh run | hadoop kms | | kms.sh start | hadoop --daemon start kms | | kms.sh stop | hadoop --daemon stop kms | | kms.sh status | hadoop --daemon status kms | was (Author: jzhuge): Map {{sbin/kms.sh}} to {{bin/hadoop kms}}: | kms.sh run | hadoop kms | | kms.sh start | hadoop kms --daemon start | | kms.sh stop | hadoop kms --daemon stop | | kms.sh status | hadoop kms --daemon status | > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704129#comment-15704129 ] John Zhuge edited comment on HADOOP-13597 at 11/29/16 6:08 PM: --- Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway - Obsolete {{kms.sh version}} that prints Tomcat version TESTING DONE - All hadoop-kms unit tests. MiniKMS equals full KMS. - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status - JMX works - /logs works TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 was (Author: jzhuge): Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway - Obsolete {{kms.sh version}} that prints Tomcat version TESTING DONE - All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - JMX not working, existing issue - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704286#comment-15704286 ] John Zhuge edited comment on HADOOP-13597 at 11/29/16 6:12 AM: --- Map {{sbin/kms.sh}} to {{bin/hadoop kms}}: | kms.sh run | hadoop kms | | kms.sh start | hadoop kms --daemon start | | kms.sh stop | hadoop kms --daemon stop | | kms.sh status | hadoop kms --daemon status | was (Author: jzhuge): Map {{sbin/kms.sh}} to {{bin/hadoop kms}}: | kms.sh run | hadoop kms | | kms.sh start | hadoop kms --daemon start | | kms.sh stop | hadoop kms --daemon stop | | | hadoop kms --daemon status | > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704129#comment-15704129 ] John Zhuge edited comment on HADOOP-13597 at 11/29/16 6:11 AM: --- Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway - Obsolete {{kms.sh version}} that prints Tomcat version TESTING DONE - All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - JMX not working, existing issue - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 was (Author: jzhuge): Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway TESTING DONE - All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - JMX not working, existing issue - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704129#comment-15704129 ] John Zhuge edited comment on HADOOP-13597 at 11/29/16 4:33 AM: --- Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework - Obsolete HTTP admin port for the Tomcat Manager GUI which does not seem to work anyway TESTING DONE - All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Add static web content /index.html - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - JMX not working, existing issue - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 was (Author: jzhuge): Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework TESTING DONE - All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Still need admin port? - Still need /index.html? - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - JMX not working, existing issue - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13597) Switch KMS from Tomcat to Jetty
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704129#comment-15704129 ] John Zhuge edited comment on HADOOP-13597 at 11/29/16 4:26 AM: --- Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework TESTING DONE - All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status TODO - Set HTTP request header size by env KMS_MAX_HTTP_HEADER_SIZE - Still need admin port? - Still need /index.html? - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - JMX not working, existing issue - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 was (Author: jzhuge): Patch 001 - KMSHttpServer based on HttpServer2 - Redirect MiniKMS to KMSHttpServer - Add kms-default.xml - Add Jetty properties including SSL properties - Convert hadoop-kms from war to jar - Rewrite kms.sh to use Hadoop shell script framework TESTING DONE - All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS - Non-secure REST APIs - Non-secure “hadoop key” commands - SSL REST APIs - kms.sh run/start/stop/status TODO - Still need admin port? - Still need /index.html? - More ad-hoc testing - Integration testing - Update docs: index.md.vm TODO in new JIRAs: - Integrate with Hadoop SSL server configuration - Full SSL server configuration: includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc. - Design common Http server configuration. Common properties in “-site.xml” with config prefix, e.g., “hadoop.kms.”. - Design HttpServer2 configuration-based builder - JMX not working, existing issue - Share web apps code in Common, HDFS, and YARN My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001 > Switch KMS from Tomcat to Jetty > --- > > Key: HADOOP-13597 > URL: https://issues.apache.org/jira/browse/HADOOP-13597 > Project: Hadoop Common > Issue Type: New Feature > Components: kms >Affects Versions: 2.6.0 >Reporter: John Zhuge >Assignee: John Zhuge > Attachments: HADOOP-13597.001.patch > > > The Tomcat 6 we are using will reach EOL at the end of 2017. While there are > other good options, I would propose switching to {{Jetty 9}} for the > following reasons: > * Easier migration. Both Tomcat and Jetty are based on {{Servlet > Containers}}, so we don't have change client code that much. It would require > more work to switch to {{JAX-RS}}. > * Well established. > * Good performance and scalability. > Other alternatives: > * Jersey + Grizzly > * Tomcat 8 > Your opinions will be greatly appreciated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org