[jira] [Commented] (HAWQ-1638) Issues with website
[ https://issues.apache.org/jira/browse/HAWQ-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537890#comment-16537890 ] ASF GitHub Bot commented on HAWQ-1638: -- Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq-site/pull/16 > Issues with website > --- > > Key: HAWQ-1638 > URL: https://issues.apache.org/jira/browse/HAWQ-1638 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Sebb >Assignee: Radar Lei >Priority: Major > > The HAWQ page looks nice, however there are a few problems with it. > The phrase > "Plus, HAWQ® works Apache MADlib) machine learning libraries" > does not read well. Something missing? > The first/main references to ASF products such as Hadoop, YARN etc must use > the full name, i.e. Apache Hadoop etc. > The download section does not have any link to the KEYS file, nor any > instructions on how to use the KEYS+sig or hashes to validate downloads. > The download section still includes references to MD5 hashes. > These are deprecated and can be removed for older releases that have other > hashes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] incubator-hawq-site pull request #16: HAWQ-1638. Correct typo and use full n...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq-site/pull/16 ---
[jira] [Closed] (HAWQ-1627) Support setting the max protocol message size when talking with HDFS
[ https://issues.apache.org/jira/browse/HAWQ-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongxu Ma closed HAWQ-1627. --- Resolution: Fixed fixed > Support setting the max protocol message size when talking with HDFS > > > Key: HAWQ-1627 > URL: https://issues.apache.org/jira/browse/HAWQ-1627 > Project: Apache HAWQ > Issue Type: Improvement > Components: libhdfs >Reporter: Hongxu Ma >Assignee: Hongxu Ma >Priority: Major > Fix For: 2.4.0.0-incubating > > > Now, the max size of protocol message in libhdfs is 64MB, and it cannot be > adjusted. > When the max size is reached (accessing a very big HDFS table/file), will see > the following lines in hawq master log: > {code} > 2018-06-20 11:21:56.768003 > CST,,,p75703,th-8481004160,,,seg-1,"LOG","0","3rd party error > log: > [libprotobuf ERROR google/protobuf/io/coded_stream.cc:208] A protocol message > was rejected because it was too big (more than 67108864 bytes). To increase > the limit (or to > disable these warnings), see CodedInputStream::SetTotalBytesLimit() in > google/protobuf/io/coded_stream.h.""SysLoggerMain","syslogger.c",518, > 2018-06-20 11:21:56.771657 > CST,,,p75703,th-8481004160,,,seg-1,"LOG","0","3rd party error > log: > 2018-06-20 11:21:56.771492, p75751, th0x7fffcd7303c0, ERROR Failed to invoke > RPC call ""getFsStats"" on server ""localhost:9000"": > RpcChannel.cpp: 783: HdfsRpcException: RPC channel to ""localhost:9000"" got > protocol mismatch: RPC channel cannot parse response header. > @ Hdfs::Internal::RpcChannelImpl::readOneResponse(bool) > @ Hdfs::Internal::RpcChannelImpl::checkOneResponse() > @ > Hdfs::Internal::RpcChannelImpl::invokeInternal(std::__1::shared_ptr) > @ > Hdfs::Internal::RpcChannelImpl:""SysLoggerMain","syslogger.c",518, > 2018-06-20 11:21:56.771711 > CST,,,p75703,th-8481004160,,,seg-1,"LOG","0","3rd party error > log: > :invoke(Hdfs::Internal::RpcCall const&) > @ Hdfs::Internal::NamenodeImpl::invoke(Hdfs::Internal::RpcCall const&) > @ Hdfs::Internal::NamenodeImpl::getFsStats() > @ Hdfs::Internal::NamenodeProxy::getFsStats() > @ Hdfs::Internal::FileSystemImpl::getFsStats() > @ Hdfs::Internal::FileSystemImpl::connect() > @ Hdfs::FileSystem::connect(char const*, char const*, char const*) > @ Hdfs::FileSystem::connect(char const*) > @ hdfsBuilderConnect > @ gpfs_hdfs_connect > @ HdfsConnect > @ HdfsGetConnection > @ HdfsGetFileBlockLocati""SysLoggerMain","syslogger.c",518, > {code} > Considering HDFS has a guc *"ipc.maximum.data.length"* to set it, HAWQ should > also add a guc for it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HAWQ-1638) Issues with website
[ https://issues.apache.org/jira/browse/HAWQ-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537839#comment-16537839 ] ASF GitHub Bot commented on HAWQ-1638: -- Github user changleicn commented on the issue: https://github.com/apache/incubator-hawq-site/pull/16 looks good! > Issues with website > --- > > Key: HAWQ-1638 > URL: https://issues.apache.org/jira/browse/HAWQ-1638 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Sebb >Assignee: Radar Lei >Priority: Major > > The HAWQ page looks nice, however there are a few problems with it. > The phrase > "Plus, HAWQ® works Apache MADlib) machine learning libraries" > does not read well. Something missing? > The first/main references to ASF products such as Hadoop, YARN etc must use > the full name, i.e. Apache Hadoop etc. > The download section does not have any link to the KEYS file, nor any > instructions on how to use the KEYS+sig or hashes to validate downloads. > The download section still includes references to MD5 hashes. > These are deprecated and can be removed for older releases that have other > hashes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] incubator-hawq-site issue #16: HAWQ-1638. Correct typo and use full name for...
Github user changleicn commented on the issue: https://github.com/apache/incubator-hawq-site/pull/16 looks good! ---
[GitHub] incubator-hawq pull request #1379: HAWQ-1622. Cache PXF proxy UGI so that cl...
Github user lavjain commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1379#discussion_r201091933 --- Diff: pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/UGICache.java --- @@ -0,0 +1,129 @@ +package org.apache.hawq.pxf.service; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import java.io.IOException; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.DelayQueue; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.fs.FileSystem; +import org.apache.hadoop.security.UserGroupInformation; + +public class UGICache { + +private static final Log LOG = LogFactory.getLog(UGICache.class); +private static Map cache = new ConcurrentHashMap<>(); +//private static DelayQueue delayQueue = new DelayQueue<>(); +private static DelayQueue[] delayQueues = new DelayQueue<>[64]; +public static long UGI_CACHE_EXPIRY = 15 * 1 * 1000L; // 15 Minutes + +public UGICache() { +for (int i = 0; i < delayQueues.length; i++) { +delayQueues[i] = new DelayQueue<>(); --- End diff -- w\ @benchristel Will change `DelayQueues` to be not static. ---
[GitHub] incubator-hawq pull request #1379: HAWQ-1622. Cache PXF proxy UGI so that cl...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1379#discussion_r201090455 --- Diff: pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/SegmentTransactionId.java --- @@ -0,0 +1,60 @@ +package org.apache.hawq.pxf.service; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +public class SegmentTransactionId { + +private Integer segmentId; +private String transactionId; --- End diff -- We might need it in the future. It doesn't hurt to leave in there for now. ---
[jira] [Commented] (HAWQ-1638) Issues with website
[ https://issues.apache.org/jira/browse/HAWQ-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536746#comment-16536746 ] ASF GitHub Bot commented on HAWQ-1638: -- Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq-site/pull/16 Download links fix will be update later. @changleicn @edespino @jiny2 Would you help to review? Thanks. > Issues with website > --- > > Key: HAWQ-1638 > URL: https://issues.apache.org/jira/browse/HAWQ-1638 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Sebb >Assignee: Radar Lei >Priority: Major > > The HAWQ page looks nice, however there are a few problems with it. > The phrase > "Plus, HAWQ® works Apache MADlib) machine learning libraries" > does not read well. Something missing? > The first/main references to ASF products such as Hadoop, YARN etc must use > the full name, i.e. Apache Hadoop etc. > The download section does not have any link to the KEYS file, nor any > instructions on how to use the KEYS+sig or hashes to validate downloads. > The download section still includes references to MD5 hashes. > These are deprecated and can be removed for older releases that have other > hashes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] incubator-hawq-site issue #16: HAWQ-1638. Correct typo and use full name for...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq-site/pull/16 Download links fix will be update later. @changleicn @edespino @jiny2 Would you help to review? Thanks. ---
[GitHub] incubator-hawq-site pull request #16: HAWQ-1638. Correct typo and use full n...
GitHub user radarwave opened a pull request: https://github.com/apache/incubator-hawq-site/pull/16 HAWQ-1638. Correct typo and use full name for ASF products Correct typo for madlib and use full name for ASF products with the first/main references. You can merge this pull request into a Git repository by running: $ git pull https://github.com/radarwave/incubator-hawq-site HAWQ-1638 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq-site/pull/16.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16 commit c6a20e535d9a36d39977269b00b2e6cea05687f8 Author: rlei Date: 2018-07-09T09:52:04Z HAWQ-1638. Correct typo and use full name for ASF products with the first/main references. ---
[jira] [Commented] (HAWQ-1638) Issues with website
[ https://issues.apache.org/jira/browse/HAWQ-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536742#comment-16536742 ] ASF GitHub Bot commented on HAWQ-1638: -- GitHub user radarwave opened a pull request: https://github.com/apache/incubator-hawq-site/pull/16 HAWQ-1638. Correct typo and use full name for ASF products Correct typo for madlib and use full name for ASF products with the first/main references. You can merge this pull request into a Git repository by running: $ git pull https://github.com/radarwave/incubator-hawq-site HAWQ-1638 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq-site/pull/16.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16 commit c6a20e535d9a36d39977269b00b2e6cea05687f8 Author: rlei Date: 2018-07-09T09:52:04Z HAWQ-1638. Correct typo and use full name for ASF products with the first/main references. > Issues with website > --- > > Key: HAWQ-1638 > URL: https://issues.apache.org/jira/browse/HAWQ-1638 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Sebb >Assignee: Radar Lei >Priority: Major > > The HAWQ page looks nice, however there are a few problems with it. > The phrase > "Plus, HAWQ® works Apache MADlib) machine learning libraries" > does not read well. Something missing? > The first/main references to ASF products such as Hadoop, YARN etc must use > the full name, i.e. Apache Hadoop etc. > The download section does not have any link to the KEYS file, nor any > instructions on how to use the KEYS+sig or hashes to validate downloads. > The download section still includes references to MD5 hashes. > These are deprecated and can be removed for older releases that have other > hashes -- This message was sent by Atlassian JIRA (v7.6.3#76005)