[jira] [Updated] (AMBARI-24928) Log Feeder: package remove - Non-fatal POSTUN scriptlet failure
[ https://issues.apache.org/jira/browse/AMBARI-24928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24928: Labels: pull-request-available (was: ) > Log Feeder: package remove - Non-fatal POSTUN scriptlet failure > --- > > Key: AMBARI-24928 > URL: https://issues.apache.org/jira/browse/AMBARI-24928 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24842) BE: Performance Tune Hosts Pages
[
https://issues.apache.org/jira/browse/AMBARI-24842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24842:
Labels: pull-request-available (was: )
> BE: Performance Tune Hosts Pages
>
>
> Key: AMBARI-24842
> URL: https://issues.apache.org/jira/browse/AMBARI-24842
> Project: Ambari
> Issue Type: Task
> Components: ambari-server
>Affects Versions: 2.7.0
>Reporter: Myroslav Papirkovskyi
>Assignee: Myroslav Papirkovskyi
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> Hosts List - The initial host page and filter operations should all take less
> than 2 seconds. Today on the 5k node cluster all operations are taking 4
> seconds.
> Request URL for filtering:
> {noformat}
> /api/v1/clusters/c1/hosts?fields=Hosts/rack_info,Hosts/host_name,Hosts/maintenance_state,Hosts/public_host_name,Hosts/cpu_count,Hosts/ph_cpu_count,alerts_summary,Hosts/host_status,Hosts/host_state,Hosts/last_heartbeat_time,Hosts/ip,host_components/HostRoles/state,host_components/HostRoles/maintenance_state,host_components/HostRoles/stale_configs,host_components/HostRoles/service_name,host_components/HostRoles/display_name,host_components/HostRoles/desired_admin_state,host_components/metrics/dfs/namenode/ClusterId,host_components/metrics/dfs/FSNamesystem/HAState,Hosts/total_mem,stack_versions/HostStackVersions,stack_versions/repository_versions/RepositoryVersions/repository_version,stack_versions/repository_versions/RepositoryVersions/id,stack_versions/repository_versions/RepositoryVersions/display_name_response=true,host_components/logging_size=10=0=Hosts/host_name.asc
> {noformat}
> with various parameters:
> {noformat}
> {"RequestInfo":{"query":"page_size:100
> from:0
> Hosts/host_status:HEALTHY"}}{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24916) Ambari Server setup for non embedded DB miss out on default configuration
[
https://issues.apache.org/jira/browse/AMBARI-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24916:
Labels: pull-request-available (was: )
> Ambari Server setup for non embedded DB miss out on default configuration
> -
>
> Key: AMBARI-24916
> URL: https://issues.apache.org/jira/browse/AMBARI-24916
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.7.1, 2.7.2
>Reporter: Viraj Jasani
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.0.0, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.7.1, 2.7.2
>
>
> If we setup Ambari Server with it's own embedded postgresql DB, we get all
> other configurations correctly e.g. setup command updates home.java,
> server.os_type, server.os_family etc in ambari.properties file that can be
> used during starting Ambari Server.
>
> However, if we provide server setup command with arguments to use
> non-embedded standalone DB, server setup does not update ambari.properties
> and does not perform any further setup utility that can be leveraged during
> starting Ambari Server.
> Example:
> 1) *ambari-server setup -j \{JDK_PATH} -s*
> This command updates ambari.properties for home.java, server.os_type,
> server.os_family etc.
>
> 2) *ambari-server setup --java-home=\{JDK_PATH} --jdbc-db=postgres
> --jdbc-driver=\{JDBC_DRIVER_PATH} --databasehost=\{IP_ADDR}
> --databaseport=5432 --databasename=ambari --postgresschema=ambari
> --databaseusername=\{DB_USER} --databasepassword=\{DB_PASSWD}
> --database=postgres -s*
> This command does not update ambari.properties and we need to update it
> manually to bring up Ambari Server. Hence, automation of server setup with
> server startup is blocked due to this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24925) DFS Directory creation fails for Spark History
[ https://issues.apache.org/jira/browse/AMBARI-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24925: Labels: pull-request-available (was: ) > DFS Directory creation fails for Spark History > -- > > Key: AMBARI-24925 > URL: https://issues.apache.org/jira/browse/AMBARI-24925 > Project: Ambari > Issue Type: Bug >Reporter: Andrew Onischuk >Assignee: Andrew Onischuk >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > Attachments: AMBARI-24925.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24924) hbase fails to start if fs.defaultFS is set to file:///
[ https://issues.apache.org/jira/browse/AMBARI-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24924: Labels: pull-request-available (was: ) > hbase fails to start if fs.defaultFS is set to file:/// > --- > > Key: AMBARI-24924 > URL: https://issues.apache.org/jira/browse/AMBARI-24924 > Project: Ambari > Issue Type: Bug >Reporter: Andrew Onischuk >Assignee: Andrew Onischuk >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > Attachments: AMBARI-24924.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692251#comment-16692251
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere closed pull request #29: AMBARI-24833. Use clustername and hostname in
cloud log archive dir
URL: https://github.com/apache/ambari-logsearch/pull/29
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
index 0cfdbcc1e2..ea422b9c31 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
@@ -21,6 +21,7 @@
import org.apache.ambari.logfeeder.common.LogFeederConstants;
import org.apache.ambari.logfeeder.conf.LogFeederProps;
import org.apache.ambari.logfeeder.plugin.input.Input;
+import org.apache.ambari.logfeeder.util.LogFeederUtil;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.core.LoggerContext;
@@ -56,10 +57,11 @@ public static Logger createLogger(Input input,
LoggerContext loggerContext, LogF
String type = input.getLogType().replace(LogFeederConstants.CLOUD_PREFIX,
"");
String uniqueThreadName = input.getThread().getName();
Configuration config = loggerContext.getConfiguration();
-String destination = logFeederProps.getCloudStorageDestination().getText();
String baseDir =
logFeederProps.getRolloverConfig().getRolloverArchiveBaseDir();
-String activeLogDir = Paths.get(baseDir, destination, ACTIVE_FOLDER,
type).toFile().getAbsolutePath();
-String archiveLogDir = Paths.get(baseDir, destination, ARCHIVED_FOLDER,
type).toFile().getAbsolutePath();
+String destination = logFeederProps.getCloudStorageDestination().getText();
+String clusterHostnameBaseDir = Paths.get(baseDir, destination,
logFeederProps.getClusterName(),
LogFeederUtil.hostName).toFile().getAbsolutePath();
+String activeLogDir = Paths.get(clusterHostnameBaseDir, ACTIVE_FOLDER,
type).toFile().getAbsolutePath();
+String archiveLogDir = Paths.get(clusterHostnameBaseDir, ARCHIVED_FOLDER,
type).toFile().getAbsolutePath();
boolean useGzip = logFeederProps.getRolloverConfig().isUseGzip();
final String archiveFilePattern;
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageUploader.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageUploader.java
index af9326aed4..22c7fc1c61 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageUploader.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageUploader.java
@@ -26,6 +26,7 @@
import org.apache.logging.log4j.Logger;
import java.io.File;
+import java.nio.file.Paths;
import java.util.Collection;
/**
@@ -74,10 +75,11 @@ public void run() {
*/
void doUpload() {
try {
- final String archiveLogDir = String.join(File.separator,
logFeederProps.getRolloverConfig().getRolloverArchiveBaseDir(), uploaderType,
"archived");
- if (new File(archiveLogDir).exists()) {
+ final File archiveLogDir =
Paths.get(logFeederProps.getRolloverConfig().getRolloverArchiveBaseDir(),
+uploaderType, clusterName, hostName, "archived").toFile();
+ if (archiveLogDir.exists()) {
String[] extensions = {"log", "json", "gz"};
-Collection filesToUpload = FileUtils.listFiles(new
File(archiveLogDir), extensions, true);
+Collection filesToUpload = FileUtils.listFiles(archiveLogDir,
extensions, true);
if (filesToUpload.isEmpty()) {
logger.debug("Not found any files to upload.");
} else {
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components:
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692252#comment-16692252
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere closed pull request #30: AMBARI-24833. Let logfeeder.properties
override core-site configs.
URL: https://github.com/apache/ambari-logsearch/pull/30
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSS3UploadClient.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSS3UploadClient.java
index 5405a1051c..df1b8a55b2 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSS3UploadClient.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSS3UploadClient.java
@@ -57,6 +57,7 @@ public void init(LogFeederProps logFeederProps) {
conf.set("fs.s3a.endpoint", s3OutputConfig.getEndpoint());
conf.set("fs.s3a.path.style.access",
String.valueOf(s3OutputConfig.isPathStyleAccess()));
conf.set("fs.s3a.multiobjectdelete.enable",
String.valueOf(s3OutputConfig.isMultiobjectDeleteEnable()));
+LogFeederHDFSUtil.overrideFileSystemConfigs(logFeederProps, conf);
this.fs = LogFeederHDFSUtil.buildFileSystem(conf);
}
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSUploadClient.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSUploadClient.java
index 421c4c5cb7..7e1b471de0 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSUploadClient.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/upload/HDFSUploadClient.java
@@ -84,6 +84,7 @@ public void init(LogFeederProps logFeederProps) {
}
}
logger.info("HDFS client - will use '{}' permission for uploaded files",
hdfsOutputConfig.getHdfsFilePermissions());
+LogFeederHDFSUtil.overrideFileSystemConfigs(logFeederProps, configuration);
this.fs = LogFeederHDFSUtil.buildFileSystem(configuration);
}
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederHDFSUtil.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederHDFSUtil.java
index 3549e04af6..b741229a13 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederHDFSUtil.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederHDFSUtil.java
@@ -19,7 +19,10 @@
package org.apache.ambari.logfeeder.util;
import java.io.IOException;
+import java.util.Map;
+import java.util.Properties;
+import org.apache.ambari.logfeeder.conf.LogFeederProps;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
@@ -95,4 +98,20 @@ public static void closeFileSystem(FileSystem fileSystem) {
}
}
}
+
+ /**
+ * Override Hadoop configuration object based on logfeeder.properties
configurations (with keys that starts with "fs." or "hadoop.*")
+ * @param logFeederProps global property holder
+ * @param configuration hadoop configuration holder
+ */
+ public static void overrideFileSystemConfigs(LogFeederProps logFeederProps,
Configuration configuration) {
+Properties properties = logFeederProps.getProperties();
+for (Map.Entry prop : properties.entrySet()) {
+ String propertyName = prop.getKey().toString();
+ if (propertyName.startsWith("fs.")) {
+logger.info("Override {} configuration (by logfeeder.properties)",
propertyName);
+configuration.set(propertyName, prop.getValue().toString());
+ }
+}
+ }
}
\ No newline at end of file
diff --git a/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
b/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
index 45c05f39cf..2537913bbe 100644
--- a/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
+++ b/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
@@ -68,3 +68,5 @@ logfeeder.s3.access.key=MyAccessKey
logfeeder.s3.object.acl=public-read
logfeeder.s3.path.style.access=true
logfeeder.s3.multiobject.delete.enable=false
+
+fs.s3a.acl.default=PublicReadWrite
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692249#comment-16692249 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #30: AMBARI-24833. Let logfeeder.properties override core-site configs. URL: https://github.com/apache/ambari-logsearch/pull/30#issuecomment-440036535 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/61/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692235#comment-16692235 ] ASF GitHub Bot commented on AMBARI-24833: - oleewere opened a new pull request #30: AMBARI-24833. Let logfeeder.properties override core-site configs. URL: https://github.com/apache/ambari-logsearch/pull/30 # What changes were proposed in this pull request? core site configs can be override by logfeeder.properties. It can be useful if no core-site.xml available on a host ## How was this patch tested? FT: manually This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691894#comment-16691894 ] ASF GitHub Bot commented on AMBARI-24833: - oleewere opened a new pull request #29: AMBARI-24833. Use clustername and hostname in cloud log archive dir URL: https://github.com/apache/ambari-logsearch/pull/29 # What changes were proposed in this pull request? Use hostname + cluster name as well in archive dir (to make it work properly with mounts) ## How was this patch tested? FT manually with docker env please review @g-boros This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691899#comment-16691899 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #29: AMBARI-24833. Use clustername and hostname in cloud log archive dir URL: https://github.com/apache/ambari-logsearch/pull/29#issuecomment-439942525 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/60/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691756#comment-16691756
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere closed pull request #28: AMBARI-24833. Use seconds instead of minutes
for cloud log threshold
URL: https://github.com/apache/ambari-logsearch/pull/28
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
index a15ac7468a..1b0aa534d6 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
@@ -117,7 +117,7 @@
public static final String CLOUD_STORAGE_BASE_PATH =
"logfeeder.cloud.storage.base.path";
public static final String CLOUD_ROLLOVER_ARCHIVE_LOCATION =
"logfeeder.cloud.rollover.archive.base.dir";
- public static final String CLOUD_ROLLOVER_THRESHOLD_TIME_MIN =
"logfeeder.cloud.rollover.threshold.min";
+ public static final String CLOUD_ROLLOVER_THRESHOLD_TIME_SECONDS =
"logfeeder.cloud.rollover.threshold.min";
public static final String CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE =
"logfeeder.cloud.rollover.threshold.size";
public static final String CLOUD_ROLLOVER_MAX_BACKUP_FILES =
"logfeeder.cloud.rollover.max.files";
public static final String CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE_UNIT =
"logfeeder.cloud.rollover.threshold.size.unit";
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/output/RolloverConfig.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/output/RolloverConfig.java
index 282792ae86..bdc908bd5e 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/output/RolloverConfig.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/output/RolloverConfig.java
@@ -37,14 +37,14 @@
private String rolloverArchiveBaseDir;
@LogSearchPropertyDescription(
-name = LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_MIN,
-description = "Rollover cloud log files after an interval (minutes)",
-examples = {"1"},
-defaultValue = "60",
+name = LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_SECONDS,
+description = "Rollover cloud log files after an interval (seconds)",
+examples = {"60"},
+defaultValue = "3600",
sources = {LogFeederConstants.LOGFEEDER_PROPERTIES_FILE}
)
- @Value("${"+ LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_MIN + ":60}")
- private int rolloverThresholdTimeMins;
+ @Value("${"+ LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_SECONDS +
":3600}")
+ private int rolloverThresholdTimeSeconds;
@LogSearchPropertyDescription(
name = LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE,
@@ -116,12 +116,12 @@
@Value("${"+ LogFeederConstants.CLOUD_ROLLOVER_ON_STARTUP + ":false}")
private boolean rolloverOnStartup;
- public int getRolloverThresholdTimeMins() {
-return rolloverThresholdTimeMins;
+ public int getRolloverThresholdTimeSeconds() {
+return rolloverThresholdTimeSeconds;
}
- public void setRolloverThresholdTimeMins(int rolloverThresholdTimeMins) {
-this.rolloverThresholdTimeMins = rolloverThresholdTimeMins;
+ public void setRolloverThresholdTimeSeconds(int
rolloverThresholdTimeSeconds) {
+this.rolloverThresholdTimeSeconds = rolloverThresholdTimeSeconds;
}
public Integer getRolloverMaxBackupFiles() {
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
index 0cfdbcc1e2..d39349f4d1 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/cloud/CloudStorageLoggerFactory.java
@@ -78,7 +78,7 @@ public static Logger createLogger(Input input, LoggerContext
loggerContext, LogF
String rolloverSize =
logFeederProps.getRolloverConfig().getRolloverSize().toString() +
logFeederProps.getRolloverConfig().getRolloverSizeFormat();
SizeBasedTriggeringPolicy sizeBasedTriggeringPolicy =
SizeBasedTriggeringPolicy.createPolicy(rolloverSize);
CustomTimeBasedTriggeringPolicy customTimeBasedTriggeringPolicy =
CustomTimeBasedTriggeringPolicy
-
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691755#comment-16691755 ] ASF GitHub Bot commented on AMBARI-24833: - oleewere commented on issue #28: AMBARI-24833. Use seconds instead of minutes for cloud log threshold URL: https://github.com/apache/ambari-logsearch/pull/28#issuecomment-439908578 i will close this, let's keep as minutes This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691711#comment-16691711 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #28: AMBARI-24833. Use seconds instead of minutes for cloud log threshold URL: https://github.com/apache/ambari-logsearch/pull/28#issuecomment-439897680 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/59/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691700#comment-16691700 ] ASF GitHub Bot commented on AMBARI-24833: - oleewere opened a new pull request #28: AMBARI-24833. Use seconds instead of minutes for cloud log threshold URL: https://github.com/apache/ambari-logsearch/pull/28 # What changes were proposed in this pull request? Use seconds instead of minutes for cloud log threshold ## How was this patch tested? waiting for UTs This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691685#comment-16691685 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #27: AMBARI-24833. HDFS client kerberos support + small fixes URL: https://github.com/apache/ambari-logsearch/pull/27#issuecomment-43945 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/58/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24917) Implement complex Add Service request using default configs
[ https://issues.apache.org/jira/browse/AMBARI-24917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24917: Labels: pull-request-available (was: ) > Implement complex Add Service request using default configs > --- > > Key: AMBARI-24917 > URL: https://issues.apache.org/jira/browse/AMBARI-24917 > Project: Ambari > Issue Type: Task > Components: ambari-server >Reporter: Doroszlai, Attila >Assignee: Doroszlai, Attila >Priority: Major > Labels: pull-request-available > Fix For: trunk > > > Continuing AMBARI-24901, create components and host components for the Add > Service request. Also create stack default configs, necessary for install > and start tasks. > User-defined and advisor-recommended configs to be implemented separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691686#comment-16691686
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere closed pull request #27: AMBARI-24833. HDFS client kerberos support +
small fixes
URL: https://github.com/apache/ambari-logsearch/pull/27
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
index f9ef32d688..a15ac7468a 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
@@ -132,6 +132,8 @@
public static final String HDFS_PORT = "logfeeder.hdfs.port";
public static final String HDFS_FILE_PERMISSIONS =
"logfeeder.hdfs.file.permissions";
public static final String HDFS_KERBEROS = "logfeeder.hdfs.kerberos";
+ public static final String HDFS_KERBEROS_KEYTAB = "logfeeder.hdfs.keytab";
+ public static final String HDFS_KERBEROS_PRINCIPAL =
"logfeeder.hdfs.principal";
public static final String S3_ENDPOINT = "logfeeder.s3.endpoint";
public static final String S3_ENDPOINT_DEFAULT = "https://s3.amazonaws.com;;
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
index f2eb6c741e..b6ab4c7342 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
@@ -19,7 +19,7 @@
package org.apache.ambari.logfeeder.conf;
import org.apache.ambari.logfeeder.common.LogFeederConstants;
-import org.apache.ambari.logfeeder.conf.output.ExternalHdfsOutputConfig;
+import org.apache.ambari.logfeeder.conf.output.HdfsOutputConfig;
import org.apache.ambari.logfeeder.conf.output.RolloverConfig;
import org.apache.ambari.logfeeder.conf.output.S3OutputConfig;
import org.apache.ambari.logfeeder.plugin.common.LogFeederProperties;
@@ -53,7 +53,7 @@
private S3OutputConfig s3OutputConfig;
@Inject
- private ExternalHdfsOutputConfig hdfsOutputConfig;
+ private HdfsOutputConfig hdfsOutputConfig;
private Properties properties;
@@ -258,7 +258,7 @@
defaultValue = "false",
sources = {LogFeederConstants.LOGFEEDER_PROPERTIES_FILE}
)
- @Value("${" + LogFeederConstants.CLOUD_STORAGE_USE_HDFS_CLIENT + ":false}")
+ @Value("${" + LogFeederConstants.CLOUD_STORAGE_USE_HDFS_CLIENT + ":true}")
private boolean useCloudHdfsClient;
@LogSearchPropertyDescription(
@@ -280,15 +280,6 @@
@Value("${" + LogFeederConstants.CLOUD_STORAGE_BASE_PATH + ":}")
private String cloudBasePath;
- @LogSearchPropertyDescription(
-name = LogFeederConstants.HDFS_USER,
-description = "Overrides HADOOP_USER_NAME variable at runtime",
-examples = {"hdfs"},
-sources = {LogFeederConstants.LOGFEEDER_PROPERTIES_FILE}
- )
- @Value("${"+ LogFeederConstants.HDFS_USER + ":}")
- private String logfeederHdfsUser;
-
@LogSearchPropertyDescription(
name = LogFeederConstants.CLOUD_STORAGE_USE_FILTERS,
description = "Use filters for inputs (with filters the output format will
be JSON)",
@@ -460,7 +451,7 @@ public void setCloudStorageMode(LogFeederMode
cloudStorageMode) {
this.cloudStorageMode = cloudStorageMode;
}
- public ExternalHdfsOutputConfig getHdfsOutputConfig() {
+ public HdfsOutputConfig getHdfsOutputConfig() {
return hdfsOutputConfig;
}
@@ -480,7 +471,7 @@ public void setRolloverConfig(RolloverConfig
rolloverConfig) {
this.rolloverConfig = rolloverConfig;
}
- public void setHdfsOutputConfig(ExternalHdfsOutputConfig hdfsOutputConfig) {
+ public void setHdfsOutputConfig(HdfsOutputConfig hdfsOutputConfig) {
this.hdfsOutputConfig = hdfsOutputConfig;
}
@@ -512,14 +503,6 @@ public boolean isUseCloudHdfsClient() {
return useCloudHdfsClient;
}
- public String getLogfeederHdfsUser() {
-return logfeederHdfsUser;
- }
-
- public void setLogfeederHdfsUser(String logfeederHdfsUser) {
-this.logfeederHdfsUser = logfeederHdfsUser;
- }
-
public void setUseCloudHdfsClient(boolean useCloudHdfsClient) {
this.useCloudHdfsClient = useCloudHdfsClient;
}
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/output/ExternalHdfsOutputConfig.java
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691678#comment-16691678
]
ASF GitHub Bot commented on AMBARI-24833:
-
kasakrisz commented on a change in pull request #27: AMBARI-24833. HDFS client
kerberos support + small fixes
URL: https://github.com/apache/ambari-logsearch/pull/27#discussion_r234610512
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/output/HdfsOutputConfig.java
##
@@ -62,7 +71,27 @@
sources = {LogFeederConstants.LOGFEEDER_PROPERTIES_FILE}
)
@Value("${"+ LogFeederConstants.HDFS_KERBEROS + ":false}")
- private boolean secure;
+ private boolean hdfsKerberos;
+
+ @LogSearchPropertyDescription(
+name = LogFeederConstants.HDFS_KERBEROS_KEYTAB,
+description = "Kerberos keytab location for Log Feeder for communicating
with secure HDFS. ",
+examples = {"/etc/security/keytabs/mykeytab.keytab"},
+defaultValue = "/etc/security/keytabs/logfeeder.service.keytab",
+sources = {LogFeederConstants.LOGFEEDER_PROPERTIES_FILE}
+ )
+ @Value("${"+ LogFeederConstants.HDFS_KERBEROS_KEYTAB +
":/etc/security/keytabs/logfeeder.service.keytab}")
+ private String keytab;
+
+ @LogSearchPropertyDescription(
+name = LogFeederConstants.HDFS_KERBEROS_PRINCIPAL,
+description = "Kerberos keytab location for Log Feeder for communicating
with secure HDFS. ",
Review comment:
This is the same as in line 78
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24922) No need to create test jar if tests are skipped
[
https://issues.apache.org/jira/browse/AMBARI-24922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24922:
Labels: pull-request-available (was: )
> No need to create test jar if tests are skipped
> ---
>
> Key: AMBARI-24922
> URL: https://issues.apache.org/jira/browse/AMBARI-24922
> Project: Ambari
> Issue Type: Improvement
> Components: ambari-server
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Minor
> Labels: pull-request-available
> Fix For: trunk
>
>
> Some test-related tasks ({{create-sample-upgrade-check-jar}},
> {{generate-test-oozie2-checks-dir}},
> {{generate-test-oozie2-server-actions-dir}}) can be skipped during build if
> tests are skipped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24912) Infra Manager: scheduled job fails with dateparse exception
[
https://issues.apache.org/jira/browse/AMBARI-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691652#comment-16691652
]
ASF GitHub Bot commented on AMBARI-24912:
-
kasakrisz closed pull request #16: AMBARI-24912 - Infra Manager: scheduled job
fails with dateparse exception
URL: https://github.com/apache/ambari-infra/pull/16
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrQueryBuilder.java
b/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrQueryBuilder.java
index f0563f95..40fa19b4 100644
---
a/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrQueryBuilder.java
+++
b/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrQueryBuilder.java
@@ -25,6 +25,7 @@
import java.time.Duration;
import java.time.OffsetDateTime;
+import java.time.ZoneOffset;
import java.util.HashMap;
import java.util.Map;
@@ -33,7 +34,7 @@
public class SolrQueryBuilder {
public static String computeEnd(String end, Duration ttl) {
-return computeEnd(end, OffsetDateTime.now(), ttl);
+return computeEnd(end, OffsetDateTime.now(ZoneOffset.UTC), ttl);
}
public static String computeEnd(String end, OffsetDateTime now, Duration
ttl) {
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: scheduled job fails with dateparse exception
> ---
>
> Key: AMBARI-24912
> URL: https://issues.apache.org/jira/browse/AMBARI-24912
> Project: Ambari
> Issue Type: Bug
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24913) Update ldap-configuration category in Ambari Configurations data to contain properties to aid in automated LDAP configuration
[
https://issues.apache.org/jira/browse/AMBARI-24913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24913:
Labels: pull-request-available (was: )
> Update ldap-configuration category in Ambari Configurations data to contain
> properties to aid in automated LDAP configuration
> -
>
> Key: AMBARI-24913
> URL: https://issues.apache.org/jira/browse/AMBARI-24913
> Project: Ambari
> Issue Type: Task
> Components: ambari-server, security
>Affects Versions: 2.7.0
>Reporter: Sandor Molnar
>Assignee: Sandor Molnar
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> Update {{ldap-configuration}} category in Ambari Configurations data to
> contain properties to aid in automated LDAP configuration:
> * {{ambari.ldap.manage_services}}
> ** This property is used to indicate whether Ambari is to manage relevant
> services' LDAP configurations or not ("true" | "false")
> * {{ambari.ldap.enabled_services}}
> ** This property is used to declare what services are expected to be
> configured for LDAP and is expected to be a comma-delimited list of services
> or "{{*}}" to indicate all services.
> Examples:
> {code:java|title=All services}
> "ambari.ldap.manage_services":"true",
> "ambari.ldap.enabled_services":"*"
> {code}
> {code:java|title=Only Ranger}
> "ambari.ldap.manage_services":"true",
> "ambari.ldap.enabled_services":"Ranger"
> {code}
> {code:java|title=Ranger, and Atlas}
> "ambari.ldap.manage_services":"true",
> "ambari.ldap.enabled_services":"Ranger, Atlas"
> {code}
> {code:java|title=Do not manage services}
> "ambari.ldap.manage_services":"false",
> "ambari.ldap.enabled_services":""
> {code}
> Each service in the set of services should have indicated it supports LDAP
> (see BUG-114409) else it will silently be ignored.
> This value should be set via Ambari's REST API or a Blueprint.
> Upon setting this value via the Ambari REST API, it is expected that internal
> logic will be triggered to ensure the relevant services in the list are
> configured for LDAP or not configured for LDAP as the case may be.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24921) If service does not have client service component created then "Run Service Check" option should be made hidden
[ https://issues.apache.org/jira/browse/AMBARI-24921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24921: Labels: pull-request-available (was: ) > If service does not have client service component created then "Run Service > Check" option should be made hidden > --- > > Key: AMBARI-24921 > URL: https://issues.apache.org/jira/browse/AMBARI-24921 > Project: Ambari > Issue Type: Task > Components: ambari-web >Affects Versions: 2.7.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > If service does not have client service component created then "Run Service > Check" option should be made hidden -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24920) LocalFS (file:///) directory creation fails
[ https://issues.apache.org/jira/browse/AMBARI-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24920: Labels: pull-request-available (was: ) > LocalFS (file:///) directory creation fails > --- > > Key: AMBARI-24920 > URL: https://issues.apache.org/jira/browse/AMBARI-24920 > Project: Ambari > Issue Type: Bug >Reporter: Andrew Onischuk >Assignee: Andrew Onischuk >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > Attachments: AMBARI-24920.patch, AMBARI-24920.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691475#comment-16691475 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #27: AMBARI-24833. HDFS client kerberos support + small fixes URL: https://github.com/apache/ambari-logsearch/pull/27#issuecomment-439831367 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/57/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24919) Allow Namenode HA setup with external namenodes in BlueprintConfigurationProcessor
[ https://issues.apache.org/jira/browse/AMBARI-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24919: Labels: pull-request-available (was: ) > Allow Namenode HA setup with external namenodes in > BlueprintConfigurationProcessor > -- > > Key: AMBARI-24919 > URL: https://issues.apache.org/jira/browse/AMBARI-24919 > Project: Ambari > Issue Type: Task >Affects Versions: 2.7.2 >Reporter: Balázs Bence Sári >Assignee: Balázs Bence Sári >Priority: Critical > Labels: pull-request-available > Fix For: trunk > > > There is an increasing demand for clusters with some of the components being > managed externally to Ambari. > This ticket addresses the problem where the cluster contains a HDFS_CLIENT > but server HDFS server components are external and NAMENODE is HA. > In this cases, cluster template validation failed as there were 0 name nodes > which is less than the expected minimum of two. > Going forward the following setup should pass validation: > - Namenode HA is enabled > - There are 0 namenodes > - all namenode dfs rpc addresses point to an external fqdn. > This will give sufficent protection against accidentally omitted namenodes, > however will enable clusters with external namenodes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691466#comment-16691466 ] ASF GitHub Bot commented on AMBARI-24833: - oleewere opened a new pull request #27: AMBARI-24833. HDFS client kerberos support + small fixes URL: https://github.com/apache/ambari-logsearch/pull/27 # What changes were proposed in this pull request? - add kerberos support for hdfs client - remove external hdfs client -> merge it with hdfs client - small fixes ## How was this patch tested? FT: manually - docker/vagrant/gce env This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691460#comment-16691460
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere closed pull request #26: AMBARI-24833. Support for cloud logs to using
filters + JSON output
URL: https://github.com/apache/ambari-logsearch/pull/26
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-logsearch-config-local/src/main/java/org/apache/ambari/logsearch/config/local/LogSearchConfigLogFeederLocal.java
b/ambari-logsearch-config-local/src/main/java/org/apache/ambari/logsearch/config/local/LogSearchConfigLogFeederLocal.java
index f6cb519b95..12af637d8c 100644
---
a/ambari-logsearch-config-local/src/main/java/org/apache/ambari/logsearch/config/local/LogSearchConfigLogFeederLocal.java
+++
b/ambari-logsearch-config-local/src/main/java/org/apache/ambari/logsearch/config/local/LogSearchConfigLogFeederLocal.java
@@ -84,12 +84,7 @@ public void monitorInputConfigChanges(final
InputConfigMonitor inputConfigMonito
File[] inputConfigFiles = new
File(configDir).listFiles(inputConfigFileFilter);
if (inputConfigFiles != null) {
for (File inputConfigFile : inputConfigFiles) {
-String inputConfig = new
String(Files.readAllBytes(inputConfigFile.toPath()));
-Matcher m = serviceNamePattern.matcher(inputConfigFile.getName());
-m.find();
-String serviceName = m.group(1);
-JsonElement inputConfigJson =
JsonHelper.mergeGlobalConfigWithInputConfig(parser, inputConfig,
globalConfigNode);
-inputConfigMonitor.loadInputConfigs(serviceName,
InputConfigGson.gson.fromJson(inputConfigJson, InputConfigImpl.class));
+tryLoadingInputConfig(inputConfigMonitor, parser, globalConfigNode,
inputConfigFile);
}
}
final FileSystem fs = FileSystems.getDefault();
@@ -100,6 +95,41 @@ public void monitorInputConfigChanges(final
InputConfigMonitor inputConfigMonito
executorService.submit(updater);
}
+ private void tryLoadingInputConfig(InputConfigMonitor inputConfigMonitor,
JsonParser parser, JsonArray globalConfigNode, File inputConfigFile) throws
Exception {
+// note: that will try to solve a intermittent issue when the input config
json is a null string (during file generation), that process will re-try to
process the files a few times
+int tries = 0;
+while(true) {
+ tries++;
+ Matcher m = serviceNamePattern.matcher(inputConfigFile.getName());
+ m.find();
+ String inputConfig = new
String(Files.readAllBytes(inputConfigFile.toPath()));
+ String serviceName = m.group(1);
+ JsonElement inputConfigJson = null;
+ logger.info("Trying to load '{}' service input config from input file
'{}'", serviceName, inputConfigFile.getAbsolutePath());
+ try {
+inputConfigJson = JsonHelper.mergeGlobalConfigWithInputConfig(parser,
inputConfig, globalConfigNode);
+ } catch (Exception e) {
+final String errorMessage;
+if (tries < 3) {
+ errorMessage = String.format("Cannot parse input config: %s, will
retry in a few seconds again (tries: %s)", inputConfig, String.valueOf(tries));
+ logger.error(errorMessage, e);
+ try {
+Thread.sleep(2000);
+ } catch (Exception ex) {
+// skip
+ }
+ continue;
+} else {
+ errorMessage = String.format("Cannot parse input config: %s, after
%s tries. Will skip to processing it", inputConfig, String.valueOf(tries));
+ logger.error(errorMessage, e);
+ break;
+}
+ }
+ inputConfigMonitor.loadInputConfigs(serviceName,
InputConfigGson.gson.fromJson(inputConfigJson, InputConfigImpl.class));
+ break;
+}
+ }
+
@Override
public void close() throws IOException {
}
diff --git
a/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogLevelFilterManagerZK.java
b/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogLevelFilterManagerZK.java
index fd08e07fed..0975c395fa 100644
---
a/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogLevelFilterManagerZK.java
+++
b/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogLevelFilterManagerZK.java
@@ -48,6 +48,7 @@
public LogLevelFilterManagerZK(Map properties) throws
Exception {
this.client = LogSearchConfigZKHelper.createZKClient(properties);
+this.client.start();
this.serverCache = new TreeCache(client, "/");
this.aclList = LogSearchConfigZKHelper.getAcls(properties);
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691454#comment-16691454
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere commented on a change in pull request #26: AMBARI-24833. Support for
cloud logs to using filters + JSON output
URL: https://github.com/apache/ambari-logsearch/pull/26#discussion_r234544396
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputLineEnricher.java
##
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.logfeeder.output;
+
+import com.google.common.hash.Hashing;
+import org.apache.ambari.logfeeder.plugin.common.MetricData;
+import org.apache.ambari.logfeeder.plugin.input.Input;
+import org.apache.ambari.logfeeder.plugin.input.InputMarker;
+import org.apache.ambari.logfeeder.util.LogFeederUtil;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Utility class for fill output with other fields
+ */
+public class OutputLineEnricher {
+
+ private static final Logger logger =
LogManager.getLogger(OutputLineEnricher.class);
+
+ private static final int MAX_OUTPUT_SIZE = 32765; // 32766-1
+
+ public void enrichFields(final Map jsonObj, final
InputMarker inputMarker, final MetricData messageTruncateMetric) {
+Input input = inputMarker.getInput();
+// Update the block with the context fields
+for (Map.Entry entry :
input.getInputDescriptor().getAddFields().entrySet()) {
+ if (jsonObj.get(entry.getKey()) == null ||
entry.getKey().equals("cluster") && "null".equals(jsonObj.get(entry.getKey(
{
Review comment:
i will solve that in a next commit
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691455#comment-16691455
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere commented on a change in pull request #26: AMBARI-24833. Support for
cloud logs to using filters + JSON output
URL: https://github.com/apache/ambari-logsearch/pull/26#discussion_r23455
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/manager/operations/impl/AbstractInputConfigHandler.java
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.logfeeder.manager.operations.impl;
+
+import org.apache.ambari.logfeeder.manager.InputConfigHolder;
+import org.apache.ambari.logfeeder.manager.operations.InputConfigHandler;
+import org.apache.ambari.logfeeder.plugin.common.AliasUtil;
+import org.apache.ambari.logfeeder.plugin.filter.Filter;
+import org.apache.ambari.logfeeder.plugin.input.Input;
+import
org.apache.ambari.logsearch.config.api.model.inputconfig.FilterDescriptor;
+import org.apache.commons.lang.BooleanUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Holds common operations for input config handlers
+ */
+public abstract class AbstractInputConfigHandler implements InputConfigHandler
{
+
+ private static final Logger logger =
LogManager.getLogger(AbstractInputConfigHandler.class);
+
+ protected void loadFilters(String serviceName, InputConfigHolder
inputConfigHolder) {
+sortFilters(inputConfigHolder);
+
+List toRemoveInputList = new ArrayList<>();
+for (Input input :
inputConfigHolder.getInputManager().getInputList(serviceName)) {
+ for (FilterDescriptor filterDescriptor :
inputConfigHolder.getFilterConfigList()) {
+if (filterDescriptor == null) {
+ logger.warn("Filter descriptor is smpty. Skipping...");
Review comment:
i will solve that in a next commit
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691379#comment-16691379
]
ASF GitHub Bot commented on AMBARI-24833:
-
kasakrisz commented on a change in pull request #26: AMBARI-24833. Support for
cloud logs to using filters + JSON output
URL: https://github.com/apache/ambari-logsearch/pull/26#discussion_r234520580
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/manager/operations/impl/AbstractInputConfigHandler.java
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.logfeeder.manager.operations.impl;
+
+import org.apache.ambari.logfeeder.manager.InputConfigHolder;
+import org.apache.ambari.logfeeder.manager.operations.InputConfigHandler;
+import org.apache.ambari.logfeeder.plugin.common.AliasUtil;
+import org.apache.ambari.logfeeder.plugin.filter.Filter;
+import org.apache.ambari.logfeeder.plugin.input.Input;
+import
org.apache.ambari.logsearch.config.api.model.inputconfig.FilterDescriptor;
+import org.apache.commons.lang.BooleanUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Holds common operations for input config handlers
+ */
+public abstract class AbstractInputConfigHandler implements InputConfigHandler
{
+
+ private static final Logger logger =
LogManager.getLogger(AbstractInputConfigHandler.class);
+
+ protected void loadFilters(String serviceName, InputConfigHolder
inputConfigHolder) {
+sortFilters(inputConfigHolder);
+
+List toRemoveInputList = new ArrayList<>();
+for (Input input :
inputConfigHolder.getInputManager().getInputList(serviceName)) {
+ for (FilterDescriptor filterDescriptor :
inputConfigHolder.getFilterConfigList()) {
+if (filterDescriptor == null) {
+ logger.warn("Filter descriptor is smpty. Skipping...");
Review comment:
typo: smpty -> empty
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691380#comment-16691380
]
ASF GitHub Bot commented on AMBARI-24833:
-
kasakrisz commented on a change in pull request #26: AMBARI-24833. Support for
cloud logs to using filters + JSON output
URL: https://github.com/apache/ambari-logsearch/pull/26#discussion_r234521392
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputLineEnricher.java
##
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.logfeeder.output;
+
+import com.google.common.hash.Hashing;
+import org.apache.ambari.logfeeder.plugin.common.MetricData;
+import org.apache.ambari.logfeeder.plugin.input.Input;
+import org.apache.ambari.logfeeder.plugin.input.InputMarker;
+import org.apache.ambari.logfeeder.util.LogFeederUtil;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Utility class for fill output with other fields
+ */
+public class OutputLineEnricher {
+
+ private static final Logger logger =
LogManager.getLogger(OutputLineEnricher.class);
+
+ private static final int MAX_OUTPUT_SIZE = 32765; // 32766-1
+
+ public void enrichFields(final Map jsonObj, final
InputMarker inputMarker, final MetricData messageTruncateMetric) {
+Input input = inputMarker.getInput();
+// Update the block with the context fields
+for (Map.Entry entry :
input.getInputDescriptor().getAddFields().entrySet()) {
+ if (jsonObj.get(entry.getKey()) == null ||
entry.getKey().equals("cluster") && "null".equals(jsonObj.get(entry.getKey(
{
Review comment:
```"cluster".equals(entry.getKey())```
would be safer
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690680#comment-16690680 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #26: AMBARI-24833. Support for cloud logs to using filters + JSON output URL: https://github.com/apache/ambari-logsearch/pull/26#issuecomment-439642020 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/56/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690678#comment-16690678 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #26: AMBARI-24833. Support for cloud logs to using filters + JSON output URL: https://github.com/apache/ambari-logsearch/pull/26#issuecomment-439641461 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/55/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24908) OneFS mpack should include webhdfs enable setting - Default Enabled
[ https://issues.apache.org/jira/browse/AMBARI-24908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24908: Labels: pull-request-available (was: ) > OneFS mpack should include webhdfs enable setting - Default Enabled > --- > > Key: AMBARI-24908 > URL: https://issues.apache.org/jira/browse/AMBARI-24908 > Project: Ambari > Issue Type: Bug > Components: ambari-server, contrib >Affects Versions: 3.0.0 > Environment: HDP 3.0 and Isilon 8.2.0 >Reporter: Andrew Huff >Priority: Major > Labels: pull-request-available > Original Estimate: 2h > Remaining Estimate: 2h > > The Isilon management pack should include the 'dfs.webhdfs.enabled' > configuration in the hdfs_site.xml file as it is needed by the Ambari Agent. > This previous Jira incorrectly removed the setting which was originally set > to 'false', however, Isilon has since increased its support for WebHDFS. The > previous Jira assumed that by removing the setting the Ambari Agent would > default to 'true' for this setting. This does not appear to be the case. > https://issues.apache.org/jira/browse/AMBARI-24539 > https://github.com/apache/ambari/pull/2170 > This setting needs to be added back to the Isilon management pack with a > default value of 'true'. This value should also be user configurable > 'overrideable = true'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24910) Grafana: Fix for Storm Components and Storm Kafka Offset dashboards
[ https://issues.apache.org/jira/browse/AMBARI-24910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24910: Labels: pull-request-available (was: ) > Grafana: Fix for Storm Components and Storm Kafka Offset dashboards > --- > > Key: AMBARI-24910 > URL: https://issues.apache.org/jira/browse/AMBARI-24910 > Project: Ambari > Issue Type: Bug > Components: ambari-metrics >Reporter: Ishan Bhatt >Assignee: Ishan Bhatt >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > These two dashboards error out if no data point is provided. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24910) Grafana: Fix for Storm Components and Storm Kafka Offset dashboards
[
https://issues.apache.org/jira/browse/AMBARI-24910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689852#comment-16689852
]
ASF GitHub Bot commented on AMBARI-24910:
-
ishanbha closed pull request #11: [AMBARI-24910] Grafana: Fix for Storm
Components and Storm Kafka Offse…
URL: https://github.com/apache/ambari-metrics/pull/11
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/ambari-metrics-grafana/ambari-metrics/datasource.js
b/ambari-metrics-grafana/ambari-metrics/datasource.js
index c6885da..dd64566 100644
--- a/ambari-metrics-grafana/ambari-metrics/datasource.js
+++ b/ambari-metrics-grafana/ambari-metrics/datasource.js
@@ -642,8 +642,8 @@ define([
//Templatized Dashboards for Storm Components
if (templateSrv.variables[0].query === "topologies" &&
templateSrv.variables[1] &&
templateSrv.variables[1].name === "component") {
-var selectedTopology = templateSrv._values.topologies;
-var selectedComponent = templateSrv._values.component;
+var selectedTopology =
templateSrv.index.topologies.options[0].value || "";
+var selectedComponent = templateSrv.index.component.options[0] ?
templateSrv.index.component.options[0].value : "";
metricsPromises.push(_.map(options.targets, function (target) {
target.sTopology = selectedTopology;
target.sComponent = selectedComponent;
@@ -655,8 +655,8 @@ define([
//Templatized Dashboard for Storm Kafka Offset
if (templateSrv.variables[0].query === "topologies" &&
templateSrv.variables[1] &&
templateSrv.variables[1].name === "topic") {
-var selectedTopology = templateSrv._values.topologies;
-var selectedTopic = templateSrv._values.topic;
+var selectedTopology =
templateSrv.index.topologies.options[0].value || "";
+var selectedTopic = templateSrv.index.topic.options[0] ?
templateSrv.index.topic.options[0].value : "";
metricsPromises.push(_.map(options.targets, function (target) {
target.sTopology = selectedTopology;
target.sTopic = selectedTopic;
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Grafana: Fix for Storm Components and Storm Kafka Offset dashboards
> ---
>
> Key: AMBARI-24910
> URL: https://issues.apache.org/jira/browse/AMBARI-24910
> Project: Ambari
> Issue Type: Bug
> Components: ambari-metrics
>Reporter: Ishan Bhatt
>Assignee: Ishan Bhatt
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> These two dashboards error out if no data point is provided.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689697#comment-16689697 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #26: AMBARI-24833. Support for cloud logs to using filters + JSON output URL: https://github.com/apache/ambari-logsearch/pull/26#issuecomment-439465913 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/54/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689628#comment-16689628 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #26: AMBARI-24833. Support for cloud logs to using filters + JSON output URL: https://github.com/apache/ambari-logsearch/pull/26#issuecomment-439447527 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/53/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689622#comment-16689622 ] ASF GitHub Bot commented on AMBARI-24833: - oleewere opened a new pull request #26: AMBARI-24833. Support for cloud logs to using filters + JSON output URL: https://github.com/apache/ambari-logsearch/pull/26 # What changes were proposed in this pull request? - optionally use filters (+ json output) for logs that will be archived to cloud storage (default: false) - create output enricher class - only extend common operations there (there is not any refactor there) ## How was this patch tested? FT: manually This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689276#comment-16689276
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere closed pull request #25: AMBARI-24833. Extend cloud log rolling options
URL: https://github.com/apache/ambari-logsearch/pull/25
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/ambari-logsearch-logfeeder/pom.xml
b/ambari-logsearch-logfeeder/pom.xml
index e71b3cc555..003b23357f 100644
--- a/ambari-logsearch-logfeeder/pom.xml
+++ b/ambari-logsearch-logfeeder/pom.xml
@@ -150,6 +150,12 @@
org.apache.hadoop
hadoop-aws
${hadoop.version}
+
+
+ com.amazonaws
+ aws-java-sdk-bundle
+
+
com.amazonaws
@@ -161,6 +167,11 @@
aws-java-sdk-s3
${aws-sdk.version}
+
+ com.amazonaws
+ aws-java-sdk-dynamodb
+ ${aws-sdk.version}
+
org.apache.hadoop
hadoop-azure
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
index 11d351f23d..b5fffa829b 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
@@ -34,10 +34,6 @@
// Event History Constants History
public static final String VALUES = "jsons";
public static final String ROW_TYPE = "rowtype";
-
- // S3 Constants
- public static final String S3_PATH_START_WITH = "s3://";
- public static final String S3_PATH_SEPARATOR = "/";
public static final String IN_MEMORY_TIMESTAMP = "in_memory_timestamp";
@@ -122,14 +118,17 @@
public static final String CLOUD_ROLLOVER_ARCHIVE_LOCATION =
"logfeeder.cloud.rollover.archive.base.dir";
public static final String CLOUD_ROLLOVER_THRESHOLD_TIME_MIN =
"logfeeder.cloud.rollover.threshold.min";
public static final String CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE =
"logfeeder.cloud.rollover.threshold.size";
+ public static final String CLOUD_ROLLOVER_MAX_BACKUP_FILES =
"logfeeder.cloud.rollover.max.files";
+ public static final String CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE_UNIT =
"logfeeder.cloud.rollover.threshold.size.unit";
public static final String CLOUD_ROLLOVER_USE_GZIP =
"logfeeder.cloud.rollover.use.gzip";
public static final String CLOUD_ROLLOVER_IMMEDIATE_FLUSH =
"logfeeder.cloud.rollover.immediate.flush";
public static final String CLOUD_ROLLOVER_ON_SHUTDOWN =
"logfeeder.cloud.rollover.on.shutdown";
public static final String CLOUD_ROLLOVER_ON_STARTUP =
"logfeeder.cloud.rollover.on.startup";
+ public static final String HDFS_USER = "logfeeder.hdfs.user";
+
public static final String HDFS_HOST = "logfeeder.hdfs.host";
public static final String HDFS_PORT = "logfeeder.hdfs.port";
- public static final String HDFS_USER = "logfeeder.hdfs.user";
public static final String HDFS_FILE_PERMISSIONS =
"logfeeder.hdfs.file.permissions";
public static final String HDFS_KERBEROS = "logfeeder.hdfs.kerberos";
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/CloudStorageDestination.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/CloudStorageDestination.java
index 1a7eafa509..0246b98306 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/CloudStorageDestination.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/CloudStorageDestination.java
@@ -19,7 +19,7 @@
package org.apache.ambari.logfeeder.conf;
public enum CloudStorageDestination {
- HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), NONE("none");
+ HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), WASB("wasb"),
DEFAULT_FS("default-fs"), NONE("none");
private String text;
diff --git
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
index d32e1df9db..83f10e497c 100644
---
a/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
+++
b/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
@@ -224,7 +224,8 @@
@LogSearchPropertyDescription(
name = LogFeederConstants.CLOUD_STORAGE_DESTINATION,
description = "Type of storage that is the destination for cloud output
logs.",
-examples = {"hdfs", "s3",
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689274#comment-16689274 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #25: AMBARI-24833. Extend cloud log rolling options URL: https://github.com/apache/ambari-logsearch/pull/25#issuecomment-439354665 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/52/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24912) Infra Manager: scheduled job fails with dateparse exception
[ https://issues.apache.org/jira/browse/AMBARI-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689242#comment-16689242 ] ASF GitHub Bot commented on AMBARI-24912: - asfgit commented on issue #16: AMBARI-24912 - Infra Manager: scheduled job fails with dateparse exception URL: https://github.com/apache/ambari-infra/pull/16#issuecomment-439343977 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-Infra-Github-PR-Builder/24/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Infra Manager: scheduled job fails with dateparse exception > --- > > Key: AMBARI-24912 > URL: https://issues.apache.org/jira/browse/AMBARI-24912 > Project: Ambari > Issue Type: Bug > Components: ambari-infra >Affects Versions: 2.8.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24912) Infra Manager: scheduled job fails with dateparse exception
[ https://issues.apache.org/jira/browse/AMBARI-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24912: Labels: pull-request-available (was: ) > Infra Manager: scheduled job fails with dateparse exception > --- > > Key: AMBARI-24912 > URL: https://issues.apache.org/jira/browse/AMBARI-24912 > Project: Ambari > Issue Type: Bug > Components: ambari-infra >Affects Versions: 2.8.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689253#comment-16689253
]
ASF GitHub Bot commented on AMBARI-24833:
-
kasakrisz commented on a change in pull request #25: AMBARI-24833. Extend cloud
log rolling options
URL: https://github.com/apache/ambari-logsearch/pull/25#discussion_r234152011
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/output/RolloverConfig.java
##
@@ -49,12 +49,32 @@
@LogSearchPropertyDescription(
name = LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE,
description = "Rollover cloud log files after the log file size reach this
limit",
-examples = {"1024KB"},
-defaultValue = "80MB",
+examples = {"1024"},
+defaultValue = "80",
sources = {LogFeederConstants.LOGFEEDER_PROPERTIES_FILE}
)
- @Value("${"+ LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE +
":80MB}")
- private String rolloverSize;
+ @Value("${"+ LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE + ":80}")
+ private Integer rolloverSize;
+
+ @LogSearchPropertyDescription(
+name = LogFeederConstants.CLOUD_ROLLOVER_MAX_BACKUP_FILES,
+description = "The number of max backup log files for rolled over logs.",
+examples = {"50"},
+defaultValue = "10",
+sources = {LogFeederConstants.LOGFEEDER_PROPERTIES_FILE}
+ )
+ @Value("${"+ LogFeederConstants.CLOUD_ROLLOVER_MAX_BACKUP_FILES + ":10}")
+ private Integer rolloverMaxBackupFiles;
+
+ @LogSearchPropertyDescription(
+name = LogFeederConstants.CLOUD_ROLLOVER_THRESHOLD_TIME_SIZE_FORMAT,
+description = "Rollover cloud log file size format (e.g: KB, MB etc.)",
Review comment:
Unit or format?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689254#comment-16689254
]
ASF GitHub Bot commented on AMBARI-24833:
-
kasakrisz commented on a change in pull request #25: AMBARI-24833. Extend cloud
log rolling options
URL: https://github.com/apache/ambari-logsearch/pull/25#discussion_r234151152
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java
##
@@ -224,7 +224,8 @@
@LogSearchPropertyDescription(
name = LogFeederConstants.CLOUD_STORAGE_DESTINATION,
description = "Type of storage that is the destination for cloud output
logs.",
-examples = {"hdfs", "s3", "gcs", "adls", "none"},
+examples = {"hdfs", "s3", "gcs", "adls", "wasb", "none"},
Review comment:
Can `default-fs` be set here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689236#comment-16689236 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #25: AMBARI-24833. Extend cloud log rolling options URL: https://github.com/apache/ambari-logsearch/pull/25#issuecomment-439342879 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/51/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24912) Infra Manager: scheduled job fails with dateparse exception
[ https://issues.apache.org/jira/browse/AMBARI-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689241#comment-16689241 ] ASF GitHub Bot commented on AMBARI-24912: - kasakrisz opened a new pull request #16: AMBARI-24912 - Infra Manager: scheduled job fails with dateparse exception URL: https://github.com/apache/ambari-infra/pull/16 ## What changes were proposed in this pull request? Use UTC time zone when a scheduled archiving job launched to calculate the end of interval of archivable document from system time ## How was this patch tested? UTs ITs passed Manually: 1. Deploy ambari and a cluster: zookeeper, infra-solr, infra-manager, logsearch 2. Schedule an archiving job like archive_service_logs 3. wait until its first run 4. check for archived documents This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Infra Manager: scheduled job fails with dateparse exception > --- > > Key: AMBARI-24912 > URL: https://issues.apache.org/jira/browse/AMBARI-24912 > Project: Ambari > Issue Type: Bug > Components: ambari-infra >Affects Versions: 2.8.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24899) [Log Search UI] Change capture countdown text
[
https://issues.apache.org/jira/browse/AMBARI-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689207#comment-16689207
]
ASF GitHub Bot commented on AMBARI-24899:
-
tobias-istvan closed pull request #24: [AMBARI-24899] [Log Search UI] Change
capture countdown text
URL: https://github.com/apache/ambari-logsearch/pull/24
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/ambari-logsearch-web/src/assets/i18n/en.json
b/ambari-logsearch-web/src/assets/i18n/en.json
index 770f5e9dfe..0e60563e15 100644
--- a/ambari-logsearch-web/src/assets/i18n/en.json
+++ b/ambari-logsearch-web/src/assets/i18n/en.json
@@ -55,7 +55,7 @@
"filter.capture": "Capture",
"filter.captureSnapshot": "Snapshot",
- "filter.refreshingLogListIn": "Refreshing log list in...",
+ "filter.refreshingLogListIn": "Please wait while the logs for your capture
period are indexed...",
"filter.capture.min": "Min",
"filter.capture.sec": "Sec",
"filter.capture.triggeringRefresh": "Triggering auto-refresh in
{{remainingSeconds}} sec",
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [Log Search UI] Change capture countdown text
> -
>
> Key: AMBARI-24899
> URL: https://issues.apache.org/jira/browse/AMBARI-24899
> Project: Ambari
> Issue Type: Task
> Components: ambari-logsearch, logsearch
>Affects Versions: 2.7.3
>Reporter: Istvan Tobias
>Assignee: Istvan Tobias
>Priority: Trivial
> Labels: pull-request-available
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The text should be: "Please wait while the logs for your capture period are
> indexed..."
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24891) [Log Serach UI] The Log Index Filter panel does not work
[
https://issues.apache.org/jira/browse/AMBARI-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689205#comment-16689205
]
ASF GitHub Bot commented on AMBARI-24891:
-
tobias-istvan closed pull request #23: [AMBARI-24891] [Log Serach UI] The Log
Index Filter panel does not work
URL: https://github.com/apache/ambari-logsearch/pull/23
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-logsearch-web/src/app/components/action-menu/action-menu.component.ts
b/ambari-logsearch-web/src/app/components/action-menu/action-menu.component.ts
index 721ae93d74..a5d9ae2f32 100644
---
a/ambari-logsearch-web/src/app/components/action-menu/action-menu.component.ts
+++
b/ambari-logsearch-web/src/app/components/action-menu/action-menu.component.ts
@@ -75,6 +75,9 @@ export class ActionMenuComponent implements OnInit,
OnDestroy {
this.selectedClusterName$.takeUntil(this.destroyed$).subscribe(
(clusterName: string) => this.setModalSubmitDisabled(!clusterName)
);
+this.clustersListItems$.filter((items: ListItem[]) => items.some((item:
ListItem) => item.isChecked)).first()
+ .map((items: ListItem[]) => items.find((item: ListItem) =>
item.isChecked))
+ .subscribe((item) => this.selectedClusterName$.next(item.value));
}
ngOnDestroy() {
diff --git
a/ambari-logsearch-web/src/app/components/filter-button/filter-button.component.ts
b/ambari-logsearch-web/src/app/components/filter-button/filter-button.component.ts
index d6f24e5291..21cc1510f2 100644
---
a/ambari-logsearch-web/src/app/components/filter-button/filter-button.component.ts
+++
b/ambari-logsearch-web/src/app/components/filter-button/filter-button.component.ts
@@ -51,7 +51,7 @@ export class FilterButtonComponent extends
MenuButtonComponent implements Contro
}
writeValue(items: ListItem[]) {
-this.selection = items;
+this.selection = items ? (Array.isArray(items) ? items : [items] ) : [];
}
registerOnChange(callback: any): void {
diff --git
a/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.html
b/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.html
index 7061defaa7..28cde2be3f 100644
---
a/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.html
+++
b/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.html
@@ -15,7 +15,7 @@
limitations under the License.
-->
-
+
diff --git
a/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.ts
b/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.ts
index faf2165c7a..317499146f 100644
---
a/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.ts
+++
b/ambari-logsearch-web/src/app/components/menu-button/menu-button.component.ts
@@ -127,6 +127,14 @@ export class MenuButtonComponent {
return this.subItems && this.subItems.filter((option: ListItem): boolean
=> option.isChecked);
}
+ get hasSelection(): boolean {
+return this.subItems && this.subItems.filter((option: ListItem): boolean
=> option.isChecked).length > 0;
+ }
+
+ get totalSelection(): number {
+return this.subItems ? this.subItems.filter((option: ListItem): boolean =>
option.isChecked).length : 0;
+ }
+
constructor(private utils: UtilsService) {}
findItemIndexInList(item: ListItem, itemList: ListItem[] = this.subItems):
number {
diff --git
a/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.html
b/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.html
index 714002854a..f4fdf4f1b6 100644
---
a/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.html
+++
b/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.html
@@ -15,17 +15,19 @@
limitations under the License.
-->
-
+
-
{{label}}
- {{selection[0].label |
translate}}
+
+{{ item.label | translate }}
+
diff --git
a/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.ts
b/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.ts
index 74341aeff4..2d04f210e5 100644
---
a/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.ts
+++
b/ambari-logsearch-web/src/app/modules/shared/components/dropdown-button/dropdown-button.component.ts
@@ -67,14 +67,31 @@ export class DropdownButtonComponent {
@Input()
[jira] [Updated] (AMBARI-24907) Update service metainfo to declare LDAP integration support
[
https://issues.apache.org/jira/browse/AMBARI-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24907:
Labels: pull-request-available (was: )
> Update service metainfo to declare LDAP integration support
> ---
>
> Key: AMBARI-24907
> URL: https://issues.apache.org/jira/browse/AMBARI-24907
> Project: Ambari
> Issue Type: Task
> Components: ambari-sever
>Affects Versions: 2.7.0
>Reporter: Sandor Molnar
>Assignee: Sandor Molnar
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> Update service metainfo to declare LDAP integration support. The following
> tag may be optionally set in a service's metainfo.xml file:
> {noformat}
>
> true
>
> {
> "equals": [
> "ranger-admin-site/ranger.authentication.method",
> "LDAP"
> ]
> }
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-2491) Security Wizard: show which principals and keytabs need to be created on which hosts
[
https://issues.apache.org/jira/browse/AMBARI-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688722#comment-16688722
]
ASF GitHub Bot commented on AMBARI-2491:
asfgit commented on issue #11: [AMBARI-2491] Grafana: Fix for Storm Components
and Storm Kafka Offse…
URL: https://github.com/apache/ambari-metrics/pull/11#issuecomment-439211171
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Metrics-Github-PR-Builder/23/
Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Security Wizard: show which principals and keytabs need to be created on
> which hosts
>
>
> Key: AMBARI-2491
> URL: https://issues.apache.org/jira/browse/AMBARI-2491
> Project: Ambari
> Issue Type: Improvement
> Components: ambari-web
>Affects Versions: 1.2.5
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.2.5
>
> Attachments: AMBARI-2491.patch,
> host-component-principal-keytab-display.png
>
>
> Currently it is very difficult to know what principals and keytabs need to be
> created on which hosts.
> We should present this information to the end user in a format that is easy
> to consume.
> The user running the wizard may not be the one who will be creating keytabs
> and principals. We can expose the capability to download a csv file and send
> it to the appropriate person who may parse the data to create a script to
> generate principals/keytabs (or do so manually).
> Display the attached as a popup after Configure Services step is done.
> Let's show it as a popup so that we don't affect any existing navigation/flow.
> For generating the content:
> Keytab paths are based on the user input
> Principal names are based on the user input
> NameNode host: show the nn and HTTP principals and keytab paths
> JobTracker host: show the jt principal and keytab path
> Oozie Server host: show the oozie and HTTP principals and keytab paths
> Nagios Server host: show the nagios principal and keytab path
> HBase Master host: show the hbase principal and keytab path
> Hive Server host: show the hive principal and keytab path
> WebHCat Server host: show the HTTP principal and keytab path
> ZooKeeper Server host: show the zookeeper principal and keytab path
> DataNode host: show the dn principal and keytab path
> TaskTracker host: show the tt principal and keytab path
> RegionServer host: show the hbase principal and keytab path
> If there are duplicated principals on the same host, display it only once.
> Clickng on "Download CSV" downloads the CSV file
> ("host-principal-keytab-list.csv"). The same content, except each row is a
> comma-delimited list with a \n at the end.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-2491) Security Wizard: show which principals and keytabs need to be created on which hosts
[
https://issues.apache.org/jira/browse/AMBARI-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688711#comment-16688711
]
ASF GitHub Bot commented on AMBARI-2491:
ishanbha opened a new pull request #11: [AMBARI-2491] Grafana: Fix for Storm
Components and Storm Kafka Offse…
URL: https://github.com/apache/ambari-metrics/pull/11
…t dashboards.
## What changes were proposed in this pull request?
Get correct values from the object templateSrv in datasource.js
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Security Wizard: show which principals and keytabs need to be created on
> which hosts
>
>
> Key: AMBARI-2491
> URL: https://issues.apache.org/jira/browse/AMBARI-2491
> Project: Ambari
> Issue Type: Improvement
> Components: ambari-web
>Affects Versions: 1.2.5
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.2.5
>
> Attachments: AMBARI-2491.patch,
> host-component-principal-keytab-display.png
>
>
> Currently it is very difficult to know what principals and keytabs need to be
> created on which hosts.
> We should present this information to the end user in a format that is easy
> to consume.
> The user running the wizard may not be the one who will be creating keytabs
> and principals. We can expose the capability to download a csv file and send
> it to the appropriate person who may parse the data to create a script to
> generate principals/keytabs (or do so manually).
> Display the attached as a popup after Configure Services step is done.
> Let's show it as a popup so that we don't affect any existing navigation/flow.
> For generating the content:
> Keytab paths are based on the user input
> Principal names are based on the user input
> NameNode host: show the nn and HTTP principals and keytab paths
> JobTracker host: show the jt principal and keytab path
> Oozie Server host: show the oozie and HTTP principals and keytab paths
> Nagios Server host: show the nagios principal and keytab path
> HBase Master host: show the hbase principal and keytab path
> Hive Server host: show the hive principal and keytab path
> WebHCat Server host: show the HTTP principal and keytab path
> ZooKeeper Server host: show the zookeeper principal and keytab path
> DataNode host: show the dn principal and keytab path
> TaskTracker host: show the tt principal and keytab path
> RegionServer host: show the hbase principal and keytab path
> If there are duplicated principals on the same host, display it only once.
> Clickng on "Download CSV" downloads the CSV file
> ("host-principal-keytab-list.csv"). The same content, except each row is a
> comma-delimited list with a \n at the end.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-2491) Security Wizard: show which principals and keytabs need to be created on which hosts
[
https://issues.apache.org/jira/browse/AMBARI-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-2491:
---
Labels: pull-request-available (was: )
> Security Wizard: show which principals and keytabs need to be created on
> which hosts
>
>
> Key: AMBARI-2491
> URL: https://issues.apache.org/jira/browse/AMBARI-2491
> Project: Ambari
> Issue Type: Improvement
> Components: ambari-web
>Affects Versions: 1.2.5
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.2.5
>
> Attachments: AMBARI-2491.patch,
> host-component-principal-keytab-display.png
>
>
> Currently it is very difficult to know what principals and keytabs need to be
> created on which hosts.
> We should present this information to the end user in a format that is easy
> to consume.
> The user running the wizard may not be the one who will be creating keytabs
> and principals. We can expose the capability to download a csv file and send
> it to the appropriate person who may parse the data to create a script to
> generate principals/keytabs (or do so manually).
> Display the attached as a popup after Configure Services step is done.
> Let's show it as a popup so that we don't affect any existing navigation/flow.
> For generating the content:
> Keytab paths are based on the user input
> Principal names are based on the user input
> NameNode host: show the nn and HTTP principals and keytab paths
> JobTracker host: show the jt principal and keytab path
> Oozie Server host: show the oozie and HTTP principals and keytab paths
> Nagios Server host: show the nagios principal and keytab path
> HBase Master host: show the hbase principal and keytab path
> Hive Server host: show the hive principal and keytab path
> WebHCat Server host: show the HTTP principal and keytab path
> ZooKeeper Server host: show the zookeeper principal and keytab path
> DataNode host: show the dn principal and keytab path
> TaskTracker host: show the tt principal and keytab path
> RegionServer host: show the hbase principal and keytab path
> If there are duplicated principals on the same host, display it only once.
> Clickng on "Download CSV" downloads the CSV file
> ("host-principal-keytab-list.csv"). The same content, except each row is a
> comma-delimited list with a \n at the end.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688697#comment-16688697 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #25: AMBARI-24833. Extend cloud log rolling options URL: https://github.com/apache/ambari-logsearch/pull/25#issuecomment-439202536 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/50/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688599#comment-16688599
]
ASF GitHub Bot commented on AMBARI-24895:
-
adoroszlai commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233995622
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
##
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger =
LoggerFactory.getLogger(InfraClient.class);
Review comment:
What you mean is a "compile-time constant".
https://docs.oracle.com/javase/tutorial/java/javaOO/classvars.html
No matter which definition of constant you prefer, the naming conventions I
was alluding to in my comment applies to `static final`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688589#comment-16688589
]
ASF GitHub Bot commented on AMBARI-24895:
-
oleewere commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233993364
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
##
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger =
LoggerFactory.getLogger(InfraClient.class);
Review comment:
@adoroszlai what the compiler thinks it is
(https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.28),
primitives/arrays or classes/objects where the state cannot change anyhow.
object immutability is not enough here, like Logger is an interface. but
another opposite example that can be constant is a list (created from a static
method), which has final add/remove methods. also worth to check what you can
use in annotations.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24876) BE: Performance Tune service Configs Pages
[
https://issues.apache.org/jira/browse/AMBARI-24876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24876:
Labels: pull-request-available (was: )
> BE: Performance Tune service Configs Pages
> --
>
> Key: AMBARI-24876
> URL: https://issues.apache.org/jira/browse/AMBARI-24876
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.7.0
>Reporter: Myroslav Papirkovskyi
>Assignee: Myroslav Papirkovskyi
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> Need to be able to show configuration in less than 2 seconds. The heaviest
> part of configurations loading is recommendations request. Now for 2k cluster
> recommendations request is taking about 12 seconds:
> {code:java}
> /api/v1/stacks/HDP/versions/3.0/recommendations
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24909) Parallel Client Fixes for Server Side Tasks
[ https://issues.apache.org/jira/browse/AMBARI-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24909: Labels: pull-request-available (was: ) > Parallel Client Fixes for Server Side Tasks > --- > > Key: AMBARI-24909 > URL: https://issues.apache.org/jira/browse/AMBARI-24909 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Reporter: Nate Cole >Assignee: Nate Cole >Priority: Critical > Labels: pull-request-available > Fix For: 2.8.0 > > > The fix for parallel clients (AMBARI-24873) was broken for server-side action > tasks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688526#comment-16688526
]
ASF GitHub Bot commented on AMBARI-24895:
-
adoroszlai commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233971124
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
##
@@ -41,7 +41,7 @@
import org.slf4j.LoggerFactory;
public class Solr {
- private static final Logger LOG = LoggerFactory.getLogger(Solr.class);
+ private static final Logger logger = LoggerFactory.getLogger(Solr.class);
Review comment:
Thanks @kasakrisz!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688525#comment-16688525
]
ASF GitHub Bot commented on AMBARI-24895:
-
adoroszlai commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233970961
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
##
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger =
LoggerFactory.getLogger(InfraClient.class);
Review comment:
@oleewere what's a constant?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688519#comment-16688519
]
ASF GitHub Bot commented on AMBARI-24895:
-
kasakrisz closed pull request #15: AMBARI-24895 - Infra Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/ambari-infra-manager-it/pom.xml b/ambari-infra-manager-it/pom.xml
index 68d43529..f1f296bc 100644
--- a/ambari-infra-manager-it/pom.xml
+++ b/ambari-infra-manager-it/pom.xml
@@ -157,6 +157,16 @@
hadoop-hdfs-client
${hadoop.version}
+
+ org.apache.logging.log4j
+ log4j-api
+ 2.11.1
+
+
+ org.apache.logging.log4j
+ log4j-core
+ 2.11.1
+
diff --git
a/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
b/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
index ddc4f000..fdccbab9 100644
---
a/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
+++
b/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
@@ -39,8 +39,8 @@
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.DefaultHttpRequestRetryHandler;
import org.apache.http.impl.client.HttpClientBuilder;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.core.type.TypeReference;
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger = LogManager.getLogger(InfraClient.class);
private final CloseableHttpClient httpClient;
private final URI baseUrl;
@@ -77,7 +77,7 @@ private HttpResponse execute(HttpRequestBase post) {
try (CloseableHttpResponse response = httpClient.execute(post)) {
String responseBodyText =
IOUtils.toString(response.getEntity().getContent(), Charset.defaultCharset());
int statusCode = response.getStatusLine().getStatusCode();
- LOG.info("Response code {} body {} ", statusCode, responseBodyText);
+ logger.info("Response code {} body {} ", statusCode, responseBodyText);
if (!(200 <= statusCode && statusCode <= 299))
throw new RuntimeException("Error while executing http request: " +
responseBodyText);
return new HttpResponse(statusCode, responseBodyText);
diff --git
a/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
b/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
index 0dcc91a9..7bc952ae 100644
--- a/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
+++ b/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
@@ -31,17 +31,17 @@
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.DefaultHttpRequestRetryHandler;
import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.LBHttpSolrClient;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrInputDocument;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
public class Solr {
- private static final Logger LOG = LoggerFactory.getLogger(Solr.class);
+ private static final Logger logger = LogManager.getLogger(Solr.class);
public static final String AUDIT_LOGS_COLLECTION = "audit_logs";
public static final String HADOOP_LOGS_COLLECTION = "hadoop_logs";
private static final int SOLR_PORT = 8983;
@@ -85,7 +85,7 @@ public void add(SolrInputDocument solrInputDocument) {
}
public void createSolrCollection(String collectionName) {
-LOG.info("Creating collection");
+logger.info("Creating collection");
runCommand(new String[]{"docker", "exec", "docker_solr_1", "solr",
"create_collection", "-force", "-c", collectionName, "-d",
Paths.get(configSetPath, "configsets", collectionName, "conf").toString(),
"-n", collectionName + "_conf"});
}
diff --git
a/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/TestUtil.java
b/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/TestUtil.java
index f48e1073..8f19ce98 100644
---
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[ https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688517#comment-16688517 ] ASF GitHub Bot commented on AMBARI-24895: - asfgit commented on issue #15: AMBARI-24895 - Infra Manager: code clean up URL: https://github.com/apache/ambari-infra/pull/15#issuecomment-439153527 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-Infra-Github-PR-Builder/23/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Infra Manager: code clean up > > > Key: AMBARI-24895 > URL: https://issues.apache.org/jira/browse/AMBARI-24895 > Project: Ambari > Issue Type: Task > Components: ambari-infra >Affects Versions: 2.8.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > * rename LOG -> logger > * remove slf4j use log4j2 > * merge Job Properties and Job Parameter classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688491#comment-16688491
]
ASF GitHub Bot commented on AMBARI-24833:
-
swagle commented on a change in pull request #25: AMBARI-24833. Extend cloud
log rolling options
URL: https://github.com/apache/ambari-logsearch/pull/25#discussion_r233960818
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/CloudStorageDestination.java
##
@@ -19,7 +19,7 @@
package org.apache.ambari.logfeeder.conf;
public enum CloudStorageDestination {
- HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), NONE("none");
+ HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), WASB("wasb"), NONE("none");
Review comment:
Makes sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688488#comment-16688488
]
ASF GitHub Bot commented on AMBARI-24833:
-
oleewere commented on a change in pull request #25: AMBARI-24833. Extend cloud
log rolling options
URL: https://github.com/apache/ambari-logsearch/pull/25#discussion_r233959595
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/CloudStorageDestination.java
##
@@ -19,7 +19,7 @@
package org.apache.ambari.logfeeder.conf;
public enum CloudStorageDestination {
- HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), NONE("none");
+ HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), WASB("wasb"), NONE("none");
Review comment:
it is something for logfeeder level, but as we are not supporting core
gcs/wasb yet it won't be needed anyway, default-fs is used, if hdfs client will
figure out the right filesystem
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688465#comment-16688465
]
ASF GitHub Bot commented on AMBARI-24833:
-
swagle commented on a change in pull request #25: AMBARI-24833. Extend cloud
log rolling options
URL: https://github.com/apache/ambari-logsearch/pull/25#discussion_r233954847
##
File path:
ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/CloudStorageDestination.java
##
@@ -19,7 +19,7 @@
package org.apache.ambari.logfeeder.conf;
public enum CloudStorageDestination {
- HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), NONE("none");
+ HDFS("hdfs"), S3("s3"), GCS("gcs"), ADLS("adls"), WASB("wasb"), NONE("none");
Review comment:
Do we need to add secure mode enums? (adlss, wasbs) are the ones that I know
exist.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
>
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
>Affects Versions: 2.7.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688436#comment-16688436 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #25: AMBARI-24833. Extend cloud log rolling options URL: https://github.com/apache/ambari-logsearch/pull/25#issuecomment-439130819 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/49/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688330#comment-16688330 ] ASF GitHub Bot commented on AMBARI-24833: - asfgit commented on issue #25: AMBARI-24833. Extend cloud log rolling options URL: https://github.com/apache/ambari-logsearch/pull/25#issuecomment-439106294 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/48/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24833) Log Feeder: send logs to cloud storage (gcs/s3 etc.)
[ https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688317#comment-16688317 ] ASF GitHub Bot commented on AMBARI-24833: - oleewere opened a new pull request #25: AMBARI-24833. Extend cloud log rolling options URL: https://github.com/apache/ambari-logsearch/pull/25 # What changes were proposed in this pull request? Extend rolling options: - add size format - add default rolling strategy ## How was this patch tested? Waiting for UTs This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Log Feeder: send logs to cloud storage (gcs/s3 etc.) > > > Key: AMBARI-24833 > URL: https://issues.apache.org/jira/browse/AMBARI-24833 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch >Affects Versions: 2.7.0 >Reporter: Olivér Szabó >Assignee: Olivér Szabó >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687987#comment-16687987
]
ASF GitHub Bot commented on AMBARI-24895:
-
kasakrisz commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233827919
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
##
@@ -41,7 +41,7 @@
import org.slf4j.LoggerFactory;
public class Solr {
- private static final Logger LOG = LoggerFactory.getLogger(Solr.class);
+ private static final Logger logger = LoggerFactory.getLogger(Solr.class);
Review comment:
I am going to replace it to make it consistent.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687949#comment-16687949
]
ASF GitHub Bot commented on AMBARI-24895:
-
oleewere commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233822759
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
##
@@ -41,7 +41,7 @@
import org.slf4j.LoggerFactory;
public class Solr {
- private static final Logger LOG = LoggerFactory.getLogger(Solr.class);
+ private static final Logger logger = LoggerFactory.getLogger(Solr.class);
Review comment:
we need the log4j2 thing in the end product (infra manager), on testing
side, it won't really matter, but on naming convention we want to do the same
(LOG -> logger)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687946#comment-16687946
]
ASF GitHub Bot commented on AMBARI-24895:
-
oleewere commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233821022
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
##
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger =
LoggerFactory.getLogger(InfraClient.class);
Review comment:
@adoroszlai , it is not a constant (only a final reference) - it is a common
misconception, also its not a log, its logger, so the name logger with
lowercase is the proper one.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687939#comment-16687939
]
ASF GitHub Bot commented on AMBARI-24895:
-
oleewere commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233821022
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
##
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger =
LoggerFactory.getLogger(InfraClient.class);
Review comment:
it is not a constant (only a final reference)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687935#comment-16687935
]
ASF GitHub Bot commented on AMBARI-24895:
-
oleewere commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233821022
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
##
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger =
LoggerFactory.getLogger(InfraClient.class);
Review comment:
it is not a constant
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24905) Service display name on left navigation bar should be suffixed with "Client" if only client service component is present for a service
[ https://issues.apache.org/jira/browse/AMBARI-24905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24905: Labels: pull-request-available (was: ) > Service display name on left navigation bar should be suffixed with "Client" > if only client service component is present for a service > -- > > Key: AMBARI-24905 > URL: https://issues.apache.org/jira/browse/AMBARI-24905 > Project: Ambari > Issue Type: Task > Components: ambari-web >Affects Versions: 2.7.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > Attachments: Displayname HDFS Client.png, Dsplayname HDFS.png > > > When a service only contains client service component then check service's > displayname. If service's displayname does not and with "Client" then suffix > service's displayname with "Client" > Current behavior: !Dsplayname HDFS.png! > Expected behavior: !Displayname HDFS Client.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24904) JAR does not exist: /var/lib/ambari-agent/lib/fast-hdfs-resource.jar
[ https://issues.apache.org/jira/browse/AMBARI-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24904: Labels: pull-request-available (was: ) > JAR does not exist: /var/lib/ambari-agent/lib/fast-hdfs-resource.jar > > > Key: AMBARI-24904 > URL: https://issues.apache.org/jira/browse/AMBARI-24904 > Project: Ambari > Issue Type: Bug >Reporter: Andrew Onischuk >Assignee: Andrew Onischuk >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > Attachments: AMBARI-24904.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687872#comment-16687872
]
ASF GitHub Bot commented on AMBARI-24895:
-
kasakrisz commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233805645
##
File path:
ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/security/HadoopCredentialStore.java
##
@@ -18,16 +18,16 @@
*/
package org.apache.ambari.infra.conf.security;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import static org.apache.commons.lang.StringUtils.isBlank;
+import static org.apache.commons.lang3.ArrayUtils.isNotEmpty;
import java.util.Optional;
-import static org.apache.commons.lang.StringUtils.isBlank;
-import static org.apache.commons.lang3.ArrayUtils.isNotEmpty;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
public class HadoopCredentialStore implements PasswordStore {
- private static final Logger LOG =
LoggerFactory.getLogger(InfraManagerSecurityConfig.class);
+ private static final Logger logger =
LogManager.getLogger(InfraManagerSecurityConfig.class);
Review comment:
LogManager.getLogger - log4j2
LoggerFactory.getLogger - slf4j
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687815#comment-16687815
]
ASF GitHub Bot commented on AMBARI-24895:
-
zeroflag commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233794856
##
File path:
ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/security/HadoopCredentialStore.java
##
@@ -18,16 +18,16 @@
*/
package org.apache.ambari.infra.conf.security;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import static org.apache.commons.lang.StringUtils.isBlank;
+import static org.apache.commons.lang3.ArrayUtils.isNotEmpty;
import java.util.Optional;
-import static org.apache.commons.lang.StringUtils.isBlank;
-import static org.apache.commons.lang3.ArrayUtils.isNotEmpty;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
public class HadoopCredentialStore implements PasswordStore {
- private static final Logger LOG =
LoggerFactory.getLogger(InfraManagerSecurityConfig.class);
+ private static final Logger logger =
LogManager.getLogger(InfraManagerSecurityConfig.class);
Review comment:
what's the difference between LogManager.getLogger and
LoggerFactory.getLogger (used at other places)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24894) Sensitive service configuration values should be decrypted when processing the Ambari agent command script, if enabled
[
https://issues.apache.org/jira/browse/AMBARI-24894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24894:
Labels: pull-request-available (was: )
> Sensitive service configuration values should be decrypted when processing
> the Ambari agent command script, if enabled
> --
>
> Key: AMBARI-24894
> URL: https://issues.apache.org/jira/browse/AMBARI-24894
> Project: Ambari
> Issue Type: Task
> Components: ambari-agent, ambari-server
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> Sensitive service configuration values should be decrypted when processing
> the Ambari agent command script, if enabled.
> During the processing of
> resource_management.libraries.script.script.Script#execute, the command data
> file is to be read in and the encrypted values in the JSON document are to be
> decrypted before executing the command.
> Each encrypted value will be in the form of
> ${enc=, value=}
> For example:
> ${enc=aes265_hex, value=5248...303d}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687741#comment-16687741
]
ASF GitHub Bot commented on AMBARI-24895:
-
adoroszlai commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233765518
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
##
@@ -49,7 +49,7 @@
// TODO: use swagger
public class InfraClient implements AutoCloseable {
- private static final Logger LOG = LoggerFactory.getLogger(InfraClient.class);
+ private static final Logger logger =
LoggerFactory.getLogger(InfraClient.class);
Review comment:
Why lowercase? It is a constant.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[
https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687742#comment-16687742
]
ASF GitHub Bot commented on AMBARI-24895:
-
adoroszlai commented on a change in pull request #15: AMBARI-24895 - Infra
Manager: code clean up
URL: https://github.com/apache/ambari-infra/pull/15#discussion_r233765973
##
File path:
ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/Solr.java
##
@@ -41,7 +41,7 @@
import org.slf4j.LoggerFactory;
public class Solr {
- private static final Logger LOG = LoggerFactory.getLogger(Solr.class);
+ private static final Logger logger = LoggerFactory.getLogger(Solr.class);
Review comment:
Is there a reason for keeping `slf4j` in integration tests?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Infra Manager: code clean up
>
>
> Key: AMBARI-24895
> URL: https://issues.apache.org/jira/browse/AMBARI-24895
> Project: Ambari
> Issue Type: Task
> Components: ambari-infra
>Affects Versions: 2.8.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> * rename LOG -> logger
> * remove slf4j use log4j2
> * merge Job Properties and Job Parameter classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24903) hdfsResource should create resources in file:/// if core-site is not available.
[ https://issues.apache.org/jira/browse/AMBARI-24903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24903: Labels: pull-request-available (was: ) > hdfsResource should create resources in file:/// if core-site is not > available. > --- > > Key: AMBARI-24903 > URL: https://issues.apache.org/jira/browse/AMBARI-24903 > Project: Ambari > Issue Type: Bug >Reporter: Andrew Onischuk >Assignee: Andrew Onischuk >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > Attachments: AMBARI-24903.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24901) Handle complex "Add Service" request in ServiceResourceProvider
[
https://issues.apache.org/jira/browse/AMBARI-24901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24901:
Labels: pull-request-available (was: )
> Handle complex "Add Service" request in ServiceResourceProvider
> ---
>
> Key: AMBARI-24901
> URL: https://issues.apache.org/jira/browse/AMBARI-24901
> Project: Ambari
> Issue Type: Task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> Related to AMBARI-24881, change {{ServiceResourceProvider}} to handle the
> complex "Add Service" request. So far only stub logic is needed, details of
> the service creation will be filled later.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24902) ATS 1.5 does not start in DL cluster without HDFS
[ https://issues.apache.org/jira/browse/AMBARI-24902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24902: Labels: pull-request-available (was: ) > ATS 1.5 does not start in DL cluster without HDFS > - > > Key: AMBARI-24902 > URL: https://issues.apache.org/jira/browse/AMBARI-24902 > Project: Ambari > Issue Type: Bug >Reporter: Andrew Onischuk >Assignee: Andrew Onischuk >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > Attachments: AMBARI-24902.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[ https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687580#comment-16687580 ] ASF GitHub Bot commented on AMBARI-24895: - kasakrisz opened a new pull request #15: AMBARI-24895 - Infra Manager: code clean up URL: https://github.com/apache/ambari-infra/pull/15 ## What changes were proposed in this pull request? - rename LOG -> logger - remove slf4j use log4j2 - merge Job Properties and Job Parameter classes ## How was this patch tested? ITs passed Manually: 1. Deploy Ambari and a cluster: zookeeper, infra-solr, logsearch, hdfs, infra-manager 2. Kerberize the cluster 3. Enable archiving for service_logs and set the destination to hdfs on Ambari UI, Infra Manager config 4. Run archiving of service_logs using infra managers rest api 5. check hdfs for archived service logs This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Infra Manager: code clean up > > > Key: AMBARI-24895 > URL: https://issues.apache.org/jira/browse/AMBARI-24895 > Project: Ambari > Issue Type: Task > Components: ambari-infra >Affects Versions: 2.8.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > * rename LOG -> logger > * remove slf4j use log4j2 > * merge Job Properties and Job Parameter classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24895) Infra Manager: code clean up
[ https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687582#comment-16687582 ] ASF GitHub Bot commented on AMBARI-24895: - asfgit commented on issue #15: AMBARI-24895 - Infra Manager: code clean up URL: https://github.com/apache/ambari-infra/pull/15#issuecomment-438942589 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-Infra-Github-PR-Builder/22/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Infra Manager: code clean up > > > Key: AMBARI-24895 > URL: https://issues.apache.org/jira/browse/AMBARI-24895 > Project: Ambari > Issue Type: Task > Components: ambari-infra >Affects Versions: 2.8.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > * rename LOG -> logger > * remove slf4j use log4j2 > * merge Job Properties and Job Parameter classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24895) Infra Manager: code clean up
[ https://issues.apache.org/jira/browse/AMBARI-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24895: Labels: pull-request-available (was: ) > Infra Manager: code clean up > > > Key: AMBARI-24895 > URL: https://issues.apache.org/jira/browse/AMBARI-24895 > Project: Ambari > Issue Type: Task > Components: ambari-infra >Affects Versions: 2.8.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > * rename LOG -> logger > * remove slf4j use log4j2 > * merge Job Properties and Job Parameter classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24878) Infra Manager: kerberos support
[
https://issues.apache.org/jira/browse/AMBARI-24878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687469#comment-16687469
]
ASF GitHub Bot commented on AMBARI-24878:
-
kasakrisz closed pull request #14: AMBARI-24878 - Infra Manager: kerberos
support
URL: https://github.com/apache/ambari-infra/pull/14
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
b/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
index 0118c769..ddc4f000 100644
---
a/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
+++
b/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java
@@ -18,10 +18,16 @@
*/
package org.apache.ambari.infra;
-import com.fasterxml.jackson.core.JsonParseException;
-import com.fasterxml.jackson.core.type.TypeReference;
-import com.fasterxml.jackson.databind.JsonMappingException;
-import com.fasterxml.jackson.databind.ObjectMapper;
+import static org.apache.commons.lang.StringUtils.isBlank;
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.Charset;
+import java.util.HashMap;
+import java.util.Map;
+
import org.apache.commons.io.IOUtils;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.CloseableHttpResponse;
@@ -36,15 +42,10 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import java.io.IOException;
-import java.io.UncheckedIOException;
-import java.net.URI;
-import java.net.URISyntaxException;
-import java.nio.charset.Charset;
-import java.util.HashMap;
-import java.util.Map;
-
-import static org.apache.commons.lang.StringUtils.isBlank;
+import com.fasterxml.jackson.core.JsonParseException;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.JsonMappingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
// TODO: use swagger
public class InfraClient implements AutoCloseable {
@@ -96,6 +97,12 @@ public JobExecutionInfo startJob(String jobName, String
parameters) {
try {
String responseText = execute(new
HttpPost(uriBuilder.build())).getBody();
Map responseContent = new
ObjectMapper().readValue(responseText, new
TypeReference>() {});
+ if (!responseContent.containsKey("jobId"))
+throw new NullPointerException("jobId is not found in start job
responseContent");
+ if (!responseContent.containsKey("jobExecutionData"))
+throw new NullPointerException("jobExecutionData is not found in start
job responseContent");
+ if (!((Map)responseContent.get("jobExecutionData")).containsKey("id"))
+throw new NullPointerException("id is not found in jobExecutionData");
return new JobExecutionInfo(responseContent.get("jobId").toString(),
((Map)responseContent.get("jobExecutionData")).get("id").toString());
} catch (URISyntaxException | JsonParseException | JsonMappingException e)
{
throw new RuntimeException(e);
diff --git
a/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/ArchivingParameters.java
b/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/ArchivingParameters.java
index 6a36f724..5c783d6e 100644
---
a/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/ArchivingParameters.java
+++
b/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/ArchivingParameters.java
@@ -54,6 +54,8 @@
@JsonSerialize(converter = FsPermissionToStringConverter.class)
@JsonDeserialize(converter = StringToFsPermissionConverter.class)
private FsPermission hdfsFilePermission;
+ private String hdfsKerberosPrincipal;
+ private String hdfsKerberosKeytabPath;
private String start;
private String end;
@JsonSerialize(converter = DurationToStringConverter.class)
@@ -172,6 +174,22 @@ public void setHdfsFilePermission(FsPermission
hdfsFilePermission) {
this.hdfsFilePermission = hdfsFilePermission;
}
+ public String getHdfsKerberosPrincipal() {
+return hdfsKerberosPrincipal;
+ }
+
+ public void setHdfsKerberosPrincipal(String hdfsKerberosPrincipal) {
+this.hdfsKerberosPrincipal = hdfsKerberosPrincipal;
+ }
+
+ public String getHdfsKerberosKeytabPath() {
+return hdfsKerberosKeytabPath;
+ }
+
+ public void setHdfsKerberosKeytabPath(String hdfsKerberosKeytabPath) {
+this.hdfsKerberosKeytabPath = hdfsKerberosKeytabPath;
+ }
+
public Optional s3Properties() {
if (isBlank(s3BucketName))
return Optional.empty();
@@ -183,6 +201,18 @@ public
[jira] [Commented] (AMBARI-24899) [Log Search UI] Change capture countdown text
[ https://issues.apache.org/jira/browse/AMBARI-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687319#comment-16687319 ] ASF GitHub Bot commented on AMBARI-24899: - asfgit commented on issue #24: [AMBARI-24899] [Log Search UI] Change capture countdown text URL: https://github.com/apache/ambari-logsearch/pull/24#issuecomment-438869660 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/47/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > [Log Search UI] Change capture countdown text > - > > Key: AMBARI-24899 > URL: https://issues.apache.org/jira/browse/AMBARI-24899 > Project: Ambari > Issue Type: Task > Components: ambari-logsearch, logsearch >Affects Versions: 2.7.3 >Reporter: Istvan Tobias >Assignee: Istvan Tobias >Priority: Trivial > Labels: pull-request-available > Original Estimate: 1h > Remaining Estimate: 1h > > The text should be: "Please wait while the logs for your capture period are > indexed..." -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24899) [Log Search UI] Change capture countdown text
[ https://issues.apache.org/jira/browse/AMBARI-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24899: Labels: pull-request-available (was: ) > [Log Search UI] Change capture countdown text > - > > Key: AMBARI-24899 > URL: https://issues.apache.org/jira/browse/AMBARI-24899 > Project: Ambari > Issue Type: Task > Components: ambari-logsearch, logsearch >Affects Versions: 2.7.3 >Reporter: Istvan Tobias >Assignee: Istvan Tobias >Priority: Trivial > Labels: pull-request-available > Original Estimate: 1h > Remaining Estimate: 1h > > The text should be: "Please wait while the logs for your capture period are > indexed..." -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24899) [Log Search UI] Change capture countdown text
[ https://issues.apache.org/jira/browse/AMBARI-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687308#comment-16687308 ] ASF GitHub Bot commented on AMBARI-24899: - tobias-istvan opened a new pull request #24: [AMBARI-24899] [Log Search UI] Change capture countdown text URL: https://github.com/apache/ambari-logsearch/pull/24 # What changes were proposed in this pull request? Simple text change. ## How was this patch tested? Manually and by unit tests: ``` PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 292 of 292 SUCCESS (9.172 secs / 9.074 secs) ✨ Done in 35.67s. ``` Please review [Ambari Contributing Guide](https://cwiki.apache.org/confluence/display/AMBARI/How+to+Contribute) before opening a pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > [Log Search UI] Change capture countdown text > - > > Key: AMBARI-24899 > URL: https://issues.apache.org/jira/browse/AMBARI-24899 > Project: Ambari > Issue Type: Task > Components: ambari-logsearch, logsearch >Affects Versions: 2.7.3 >Reporter: Istvan Tobias >Assignee: Istvan Tobias >Priority: Trivial > Labels: pull-request-available > Original Estimate: 1h > Remaining Estimate: 1h > > The text should be: "Please wait while the logs for your capture period are > indexed..." -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24898) Add cluster drop down to Grafana aggregate dashboards.
[
https://issues.apache.org/jira/browse/AMBARI-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687279#comment-16687279
]
ASF GitHub Bot commented on AMBARI-24898:
-
ishanbha closed pull request #10: [AMBARI-24898] Add cluster drop down to
Grafana aggregate dashboards.
URL: https://github.com/apache/ambari-metrics/pull/10
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/ambari-metrics-grafana/ambari-metrics/datasource.js
b/ambari-metrics-grafana/ambari-metrics/datasource.js
index 6e14f33..c6885da 100644
--- a/ambari-metrics-grafana/ambari-metrics/datasource.js
+++ b/ambari-metrics-grafana/ambari-metrics/datasource.js
@@ -689,14 +689,9 @@ define([
});
}
// To speed up querying on templatized dashboards.
- var indexOfHosts = -1;
- for (var i = 0; i < templateSrv.variables.length; i++) {
-if (templateSrv.variables[i].name == 'hosts' &&
templateSrv.index.hosts) {
- indexOfHosts = i;
-}
- }
- if (indexOfHosts >= 0) {
-let allHosts = [];
+ let allHosts;
+ if (templateSrv.index.hosts) {
+allHosts = [];
const hosts = templateSrv.index.hosts.options
for (let host of hosts) {
if (host.text.toLowerCase() === "all" && host.selected) {
@@ -705,13 +700,13 @@ define([
} else if (host.selected) {
allHosts.push(host.text);
}
-};
-metricsPromises.push(_.map(options.targets, function (target) {
- target.templatedHost = allHosts ? allHosts : '';
- target.templatedCluster = templatedCluster;
- return getAllHostData(target);
-}));
+}
}
+ metricsPromises.push(_.map(options.targets, function (target) {
+target.templatedHost = allHosts ? allHosts : '';
+target.templatedCluster = templatedCluster;
+return getAllHostData(target);
+ }));
metricsPromises = _.flatten(metricsPromises);
} else {
// Non Templatized Dashboards
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add cluster drop down to Grafana aggregate dashboards.
> --
>
> Key: AMBARI-24898
> URL: https://issues.apache.org/jira/browse/AMBARI-24898
> Project: Ambari
> Issue Type: Bug
> Components: ambari-metrics
>Reporter: Ishan Bhatt
>Assignee: Ishan Bhatt
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> Currently, the HOST based dashboards (System-Servers, HDFS Namenodes,
> HBase-Regionservers etc) have a dropdown for the cluster level selection.
> This selection determines the set of hosts in the HOST dropdown, and the
> cluster is tagged in the API call made by every graph in the dashboard
> through the "instanceId" query parameter.
> We need to add the cluster dropdown to aggregate dashboards as well. This
> cluster selection will also be tagged in the API call (instanceId) like in
> other dashboards.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24898) Add cluster drop down to Grafana aggregate dashboards.
[ https://issues.apache.org/jira/browse/AMBARI-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687094#comment-16687094 ] ASF GitHub Bot commented on AMBARI-24898: - asfgit commented on issue #10: [AMBARI-24898] Add cluster drop down to Grafana aggregate dashboards. URL: https://github.com/apache/ambari-metrics/pull/10#issuecomment-438811143 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-Metrics-Github-PR-Builder/22/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add cluster drop down to Grafana aggregate dashboards. > -- > > Key: AMBARI-24898 > URL: https://issues.apache.org/jira/browse/AMBARI-24898 > Project: Ambari > Issue Type: Bug > Components: ambari-metrics >Reporter: Ishan Bhatt >Assignee: Ishan Bhatt >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > Currently, the HOST based dashboards (System-Servers, HDFS Namenodes, > HBase-Regionservers etc) have a dropdown for the cluster level selection. > This selection determines the set of hosts in the HOST dropdown, and the > cluster is tagged in the API call made by every graph in the dashboard > through the "instanceId" query parameter. > We need to add the cluster dropdown to aggregate dashboards as well. This > cluster selection will also be tagged in the API call (instanceId) like in > other dashboards. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24898) Add cluster drop down to Grafana aggregate dashboards.
[ https://issues.apache.org/jira/browse/AMBARI-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687087#comment-16687087 ] ASF GitHub Bot commented on AMBARI-24898: - ishanbha opened a new pull request #10: [AMBARI-24898] Add cluster drop down to Grafana aggregate dashboards. URL: https://github.com/apache/ambari-metrics/pull/10 ## What changes were proposed in this pull request? Added logic to include templatized dashboards to proceed without the host variable. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add cluster drop down to Grafana aggregate dashboards. > -- > > Key: AMBARI-24898 > URL: https://issues.apache.org/jira/browse/AMBARI-24898 > Project: Ambari > Issue Type: Bug > Components: ambari-metrics >Reporter: Ishan Bhatt >Assignee: Ishan Bhatt >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > Currently, the HOST based dashboards (System-Servers, HDFS Namenodes, > HBase-Regionservers etc) have a dropdown for the cluster level selection. > This selection determines the set of hosts in the HOST dropdown, and the > cluster is tagged in the API call made by every graph in the dashboard > through the "instanceId" query parameter. > We need to add the cluster dropdown to aggregate dashboards as well. This > cluster selection will also be tagged in the API call (instanceId) like in > other dashboards. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-24898) Add cluster drop down to Grafana aggregate dashboards.
[ https://issues.apache.org/jira/browse/AMBARI-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AMBARI-24898: Labels: pull-request-available (was: ) > Add cluster drop down to Grafana aggregate dashboards. > -- > > Key: AMBARI-24898 > URL: https://issues.apache.org/jira/browse/AMBARI-24898 > Project: Ambari > Issue Type: Bug > Components: ambari-metrics >Reporter: Ishan Bhatt >Assignee: Ishan Bhatt >Priority: Major > Labels: pull-request-available > Fix For: 2.8.0 > > > Currently, the HOST based dashboards (System-Servers, HDFS Namenodes, > HBase-Regionservers etc) have a dropdown for the cluster level selection. > This selection determines the set of hosts in the HOST dropdown, and the > cluster is tagged in the API call made by every graph in the dashboard > through the "instanceId" query parameter. > We need to add the cluster dropdown to aggregate dashboards as well. This > cluster selection will also be tagged in the API call (instanceId) like in > other dashboards. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24884) AMS Grafana query editor panel does not work in upgraded version.
[
https://issues.apache.org/jira/browse/AMBARI-24884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687001#comment-16687001
]
ASF GitHub Bot commented on AMBARI-24884:
-
ishanbha closed pull request #9: [AMBARI-24884] AMS Grafana query editor panel
does not work in upgraded version.
URL: https://github.com/apache/ambari-metrics/pull/9
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/ambari-metrics-grafana/ambari-metrics/query_ctrl.js
b/ambari-metrics-grafana/ambari-metrics/query_ctrl.js
index 5b5c4a9..b466167 100644
--- a/ambari-metrics-grafana/ambari-metrics/query_ctrl.js
+++ b/ambari-metrics-grafana/ambari-metrics/query_ctrl.js
@@ -31,7 +31,7 @@ var __extends = (this && this.__extends) || (function () {
})();
exports.__esModule = true;
///
-var angular_1 = require("angular");
+require("angular");
var lodash_1 = require("lodash");
var sdk_1 = require("app/plugins/sdk");
var AmbariMetricsQueryCtrl = /** @class */ (function (_super) {
@@ -39,75 +39,6 @@ var AmbariMetricsQueryCtrl = /** @class */ (function
(_super) {
/** @ngInject **/
function AmbariMetricsQueryCtrl($scope, $injector) {
var _this = _super.call(this, $scope, $injector) || this;
-_this.targetBlur = function () {
-this.target.errors = this.validateTarget(this.target);
-// this does not work so good
-if (!lodash_1["default"].isEqual(this.oldTarget, this.target) &&
lodash_1["default"].isEmpty(this.target.errors)) {
-this.oldTarget = angular_1["default"].copy(this.target);
-this.get_data();
-}
-};
-_this.getTextValues = function (metricFindResult) {
-return lodash_1["default"].map(metricFindResult, function (value)
{ return value.text; });
-};
-_this.suggestApps = function (query, callback) {
-this.datasource.suggestApps(query)
-.then(this.getTextValues)
-.then(callback);
-};
-_this.suggestClusters = function (query, callback) {
-this.datasource.suggestClusters(this.target.app)
-.then(this.getTextValues)
-.then(callback);
-};
-_this.suggestHosts = function (query, callback) {
-this.datasource.suggestHosts(this.target.app, this.target.cluster)
-.then(this.getTextValues)
-.then(callback);
-};
-_this.suggestMetrics = function (query, callback) {
-this.datasource.suggestMetrics(query, this.target.app)
-.then(this.getTextValues)
-.then(callback);
-};
-_this.suggestTagKeys = function (query, callback) {
-this.datasource.metricFindQuery('tag_names(' + this.target.metric
+ ')')
-.then(this.getTextValues)
-.then(callback);
-};
-_this.suggestTagValues = function (query, callback) {
-this.datasource.metricFindQuery('tag_values(' + this.target.metric
+ ',' + this.target.currentTagKey + ')')
-.then(this.getTextValues)
-.then(callback);
-};
-_this.addTag = function () {
-if (!this.addTagMode) {
-this.addTagMode = true;
-return;
-}
-if (!this.target.tags) {
-this.target.tags = {};
-}
-this.target.errors = this.validateTarget(this.target);
-if (!this.target.errors.tags) {
-this.target.tags[this.target.currentTagKey] =
this.target.currentTagValue;
-this.target.currentTagKey = '';
-this.target.currentTagValue = '';
-this.targetBlur();
-}
-this.addTagMode = false;
-};
-_this.removeTag = function (key) {
-delete this.target.tags[key];
-this.targetBlur();
-};
-_this.validateTarget = function (target) {
-var errs = {};
-if (target.tags && lodash_1["default"].has(target.tags,
target.currentTagKey)) {
-errs.tags = "Duplicate tag key '" + target.currentTagKey +
"'.";
-}
-return errs;
-};
_this.errors = _this.validateTarget(_this.target);
_this.aggregators = ['none', 'avg', 'sum', 'min', 'max'];
_this.precisions = ['default', 'seconds', 'minutes', 'hours', 'days'];
@@ -131,7 +62,7 @@ var AmbariMetricsQueryCtrl = /** @class */ (function
(_super) {
this.target.seriesAggregator = "none";
}
};
-_this.$watch('target.app', function (newValue) {
+
[jira] [Updated] (AMBARI-23478) YARN Cluster CPU Usage Graph Always Shows High CPU Usage
[
https://issues.apache.org/jira/browse/AMBARI-23478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-23478:
Labels: pull-request-available (was: )
> YARN Cluster CPU Usage Graph Always Shows High CPU Usage
>
>
> Key: AMBARI-23478
> URL: https://issues.apache.org/jira/browse/AMBARI-23478
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0
>Reporter: Jonathan Hurley
>Assignee: Aravindan Vijayan
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
> Attachments: image-2018-03-19-20-26-44-325.png,
> image-2018-03-19-20-27-19-160.png
>
>
> h3. ISSUE
> In Ambari, YARN's Cluster CPU widget always shows relatively high CPU usage,
> when NodeManager in a cluster is more than one.
> !image-2018-03-19-20-26-44-325.png|thumbnail!
> !image-2018-03-19-20-27-19-160.png|thumbnail!
> (started another node at around 19:00)
> h3. REPRODUCE STEPS
> # Install a cluster with one NodeManager and AMS.
> # Confirm "Cluster CPU" widget looks OK
> # Add one more node with NodeManager, and wait for a while
> h3. INVESTIGATION
> AMS side looks OK
> {code}
> curl -s -k http://sandbox-hdp.hortonworks.com:6188/ws/v1/timeline/metrics -G
> --data-urlencode metricNames=cpu_idle._sum --data-urlencode appId=NODEMANAGER
> --data-urlencode startTime=1521454794 --data-urlencode endTime=1521455394
> --data-urlencode precision=MINUTES
> ...
> {
> "metrics": [
> {
> "appid": "nodemanager",
> "metadata": {},
> "metricname": "cpu_idle._sum",
> "metrics": {
> "152145480": 198.990001,
> "152145510": 192.56
> },
> "starttime": 152145480,
> "timestamp": 152145480
> }
> ]
> }
> {code}
> But via Ambari, cpu_idle._sum becomes *{color:#d04437}100 times{color}*
> smaller
> {code}
> curl -s -k -u admin:admin
> http://sandbox-hdp.hortonworks.com:8080/api/v1/clusters/Sandbox/services/YARN/components/NODEMANAGER
> -G --data-urlencode
> 'fields=metrics/cpu/cpu_idle._sum[1521454950,152140,15]'
> ...(snip)...
> "metrics" : {
> "cpu" : {
> "cpu_idle._sum" : [
> [
> 1.8687,
> 1521454950
> ],
> [
> 1.9843,
> 1521454980
> ],
> [
> 1.9,
> 1521455010
> ],
> [
> 1.9844,
> 1521455040
> ],
> [
> 1.8925,
> 1521455070
> ],
> ...(snip)...
> {code}
> Somehow 'cpu_idle._sum' is always wrong for this Widget:
> {code}
> curl -s -k -u admin:admin
> http://sandbox-hdp.hortonworks.com:8080/api/v1/clusters/Sandbox/services/YARN/components/NODEMANAGER
> -G --data-urlencode
> 'fields=metrics/cpu/cpu_nice._sum[1521196167,1521199767,15],metrics/cpu/cpu_idle._avg[1521196167,1521199767,15],metrics/cpu/cpu_wio._sum[1521196167,1521199767,15],metrics/cpu/cpu_idle._sum[1521196167,1521199767,15],metrics/cpu/cpu_user._sum[1521196167,1521199767,15],metrics/cpu/cpu_system._sum[1521196167,1521199767,15]'
> -o ./ambari_NODEMANAGER_metrics.json
> [root@sandbox-hdp ~]# grep -E -B1 '"cpu_|152145'
> ambari_NODEMANAGER_metrics.json | grep -vE -- '(--|\],)'
> "cpu" : {
> "cpu_idle._avg" : [
> 85.549998,
> 1521199500
> "cpu_idle._sum" : [
> 1.7106, <<< need to multiply 100
> 1521199500
> "cpu_nice._sum" : [
> 0.0,
> 1521199500
> "cpu_system._sum" : [
> 21.902,
> 1521199500
> "cpu_user._sum" : [
> 6.666,
> 1521199500
> "cpu_wio._sum" : [
> 0.2,
> 1521199500
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-24897) Reduce Visible Artifacts from the SPI
[
https://issues.apache.org/jira/browse/AMBARI-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AMBARI-24897:
Labels: pull-request-available (was: )
> Reduce Visible Artifacts from the SPI
> -
>
> Key: AMBARI-24897
> URL: https://issues.apache.org/jira/browse/AMBARI-24897
> Project: Ambari
> Issue Type: Task
> Components: ambari-server-spi
>Affects Versions: 2.8.0
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
> The SPI exposes too many internal dependencies to its consumers. Some
> dependencies, such as {{commons-lang}} might be useful and should still be
> provided. Others, such as {{gson}} should be hidden.
> - Remove non-essential dependencies from being seen by consumers of the SPI
> - Prevent consumers of the SPI from double-copying dependencies which Ambari
> Server already has
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMBARI-24891) [Log Serach UI] The Log Index Filter panel does not work
[ https://issues.apache.org/jira/browse/AMBARI-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686639#comment-16686639 ] ASF GitHub Bot commented on AMBARI-24891: - asfgit commented on issue #23: [AMBARI-24891] [Log Serach UI] The Log Index Filter panel does not work URL: https://github.com/apache/ambari-logsearch/pull/23#issuecomment-438691842 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/Ambari-LogSearch-Github-PR-Builder/46/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > [Log Serach UI] The Log Index Filter panel does not work > > > Key: AMBARI-24891 > URL: https://issues.apache.org/jira/browse/AMBARI-24891 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, logsearch >Affects Versions: 2.7.3 >Reporter: Istvan Tobias >Assignee: Istvan Tobias >Priority: Blocker > Labels: pull-request-available > > The Log Index Filter settings modal does not load the configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-24891) [Log Serach UI] The Log Index Filter panel does not work
[ https://issues.apache.org/jira/browse/AMBARI-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686629#comment-16686629 ] ASF GitHub Bot commented on AMBARI-24891: - tobias-istvan commented on issue #23: [AMBARI-24891] [Log Serach UI] The Log Index Filter panel does not work URL: https://github.com/apache/ambari-logsearch/pull/23#issuecomment-438689807 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > [Log Serach UI] The Log Index Filter panel does not work > > > Key: AMBARI-24891 > URL: https://issues.apache.org/jira/browse/AMBARI-24891 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, logsearch >Affects Versions: 2.7.3 >Reporter: Istvan Tobias >Assignee: Istvan Tobias >Priority: Blocker > Labels: pull-request-available > > The Log Index Filter settings modal does not load the configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
