[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59269276
--- Diff: phoenix-hive/pom.xml ---
@@ -0,0 +1,224 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+ 4.8.0-HBase-1.1-SNAPSHOT
+
+ phoenix-hive
+ Phoenix - Hive
+
+
+
+ org.apache.phoenix
+ phoenix-core
+
+
+ org.apache.phoenix
+ phoenix-core
+ tests
+ test
+
+
+ joda-time
+ joda-time
+
+
+ org.apache.hive
+ hive-exec
+ ${hive.version}
+
+
+ org.apache.calcite
+ *
+
+
+
+
+ org.apache.calcite
+ calcite-core
+ ${calcite.version}
+
+
+ org.apache.calcite
+ calcite-avatica
+ ${calcite.version}
+
+
+ org.apache.hive
+ hive-common
+ ${hive.version}
+
+
+ org.apache.hive
+ hive-cli
+ ${hive.version}
+
+
+ jline
+ jline
+ 0.9.94
+
+
+ commons-lang
+ commons-lang
+ ${commons-lang.version}
+
+
+ commons-logging
+ commons-logging
+ ${commons-logging.version}
+
+
+ org.apache.hbase
+ hbase-testing-util
+ test
+ true
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-it
+ test-jar
+ test
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-common
+
+
+ org.apache.hbase
+ hbase-protocol
+
+
+ org.apache.hbase
+ hbase-client
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test-jar
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test-jar
+ test
+
+
+ org.apache.hadoop
+ hadoop-common
+
+
+ org.apache.hadoop
+ hadoop-annotations
--- End diff --
I don't see this actively referenced in your changes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59269299
--- Diff: phoenix-hive/pom.xml ---
@@ -0,0 +1,224 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+ 4.8.0-HBase-1.1-SNAPSHOT
+
+ phoenix-hive
+ Phoenix - Hive
+
+
+
+ org.apache.phoenix
+ phoenix-core
+
+
+ org.apache.phoenix
+ phoenix-core
+ tests
+ test
+
+
+ joda-time
+ joda-time
+
+
+ org.apache.hive
+ hive-exec
+ ${hive.version}
+
+
+ org.apache.calcite
+ *
+
+
+
+
+ org.apache.calcite
+ calcite-core
+ ${calcite.version}
+
+
+ org.apache.calcite
+ calcite-avatica
+ ${calcite.version}
+
+
+ org.apache.hive
+ hive-common
+ ${hive.version}
+
+
+ org.apache.hive
+ hive-cli
+ ${hive.version}
+
+
+ jline
+ jline
+ 0.9.94
+
+
+ commons-lang
+ commons-lang
+ ${commons-lang.version}
+
+
+ commons-logging
+ commons-logging
+ ${commons-logging.version}
+
+
+ org.apache.hbase
+ hbase-testing-util
+ test
+ true
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-it
+ test-jar
+ test
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-common
+
+
+ org.apache.hbase
+ hbase-protocol
+
+
+ org.apache.hbase
+ hbase-client
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test-jar
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test-jar
+ test
+
+
+ org.apache.hadoop
+ hadoop-common
+
+
+ org.apache.hadoop
+ hadoop-annotations
+
+
+ org.apache.hadoop
+ hadoop-mapreduce-client-core
+
+
+ org.apache.hadoop
+ hadoop-minicluster
--- End diff --
Should this be test scope?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59269345
--- Diff: phoenix-hive/pom.xml ---
@@ -0,0 +1,224 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+ 4.8.0-HBase-1.1-SNAPSHOT
+
+ phoenix-hive
+ Phoenix - Hive
+
+
+
+ org.apache.phoenix
+ phoenix-core
+
+
+ org.apache.phoenix
+ phoenix-core
+ tests
+ test
+
+
+ joda-time
+ joda-time
+
+
+ org.apache.hive
+ hive-exec
+ ${hive.version}
+
+
+ org.apache.calcite
+ *
+
+
+
+
+ org.apache.calcite
+ calcite-core
+ ${calcite.version}
+
+
+ org.apache.calcite
+ calcite-avatica
+ ${calcite.version}
+
+
+ org.apache.hive
+ hive-common
+ ${hive.version}
+
+
+ org.apache.hive
+ hive-cli
+ ${hive.version}
+
+
+ jline
+ jline
+ 0.9.94
+
+
+ commons-lang
+ commons-lang
+ ${commons-lang.version}
+
+
+ commons-logging
+ commons-logging
+ ${commons-logging.version}
+
+
+ org.apache.hbase
+ hbase-testing-util
+ test
+ true
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-it
+ test-jar
+ test
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-common
+
+
+ org.apache.hbase
+ hbase-protocol
+
+
+ org.apache.hbase
+ hbase-client
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test-jar
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test-jar
+ test
+
+
+ org.apache.hadoop
+ hadoop-common
+
+
+ org.apache.hadoop
+ hadoop-annotations
+
+
+ org.apache.hadoop
+ hadoop-mapreduce-client-core
+
+
+ org.apache.hadoop
+ hadoop-minicluster
+
+
+
+ org.mockito
+ mockito-all
+ test
+
+
+ junit
+ junit
+ test
+
+
+
+
+
+
+org.codehaus.mojo
+build-helper-maven-plugin
+
+
+org.apache.maven.plugins
+maven-failsafe-plugin
+
+
+maven-dependency-plugin
+${maven-dependency-plugin.version}
--- End diff --
Likely unnecessary version specification
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59269404
--- Diff: phoenix-hive/pom.xml ---
@@ -0,0 +1,224 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+ 4.8.0-HBase-1.1-SNAPSHOT
+
+ phoenix-hive
+ Phoenix - Hive
+
+
+
+ org.apache.phoenix
+ phoenix-core
+
+
+ org.apache.phoenix
+ phoenix-core
+ tests
+ test
+
+
+ joda-time
+ joda-time
+
+
+ org.apache.hive
+ hive-exec
+ ${hive.version}
+
+
+ org.apache.calcite
+ *
+
+
+
+
+ org.apache.calcite
+ calcite-core
+ ${calcite.version}
+
+
+ org.apache.calcite
+ calcite-avatica
+ ${calcite.version}
+
+
+ org.apache.hive
+ hive-common
+ ${hive.version}
+
+
+ org.apache.hive
+ hive-cli
+ ${hive.version}
+
+
+ jline
+ jline
+ 0.9.94
+
+
+ commons-lang
+ commons-lang
+ ${commons-lang.version}
+
+
+ commons-logging
+ commons-logging
+ ${commons-logging.version}
+
+
+ org.apache.hbase
+ hbase-testing-util
+ test
+ true
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-it
+ test-jar
+ test
+
+
+ org.jruby
+ jruby-complete
+
+
+
+
+ org.apache.hbase
+ hbase-common
+
+
+ org.apache.hbase
+ hbase-protocol
+
+
+ org.apache.hbase
+ hbase-client
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop-compat
+ test-jar
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test
+
+
+ org.apache.hbase
+ hbase-hadoop2-compat
+ test-jar
+ test
+
+
+ org.apache.hadoop
+ hadoop-common
+
+
+ org.apache.hadoop
+ hadoop-annotations
+
+
+ org.apache.hadoop
+ hadoop-mapreduce-client-core
+
+
+ org.apache.hadoop
+ hadoop-minicluster
+
+
+
+ org.mockito
+ mockito-all
+ test
+
+
+ junit
+ junit
+ test
+
+
+
+
+
+
+org.codehaus.mojo
+build-helper-maven-plugin
+
+
+org.apache.maven.plugins
+maven-failsafe-plugin
+
+
+maven-dependency-plugin
+${maven-dependency-plugin.version}
+
--- End diff --
Wonky indentation here. Tabs maybe?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59269559
--- Diff: phoenix-hive/pom.xml ---
@@ -0,0 +1,224 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+ 4.8.0-HBase-1.1-SNAPSHOT
+
+ phoenix-hive
+ Phoenix - Hive
+
+
+
+ org.apache.phoenix
+ phoenix-core
+
+
+ org.apache.phoenix
+ phoenix-core
+ tests
+ test
+
+
+ joda-time
+ joda-time
+
+
+ org.apache.hive
+ hive-exec
+ ${hive.version}
+
+
+ org.apache.calcite
+ *
+
+
+
+
+ org.apache.calcite
+ calcite-core
+ ${calcite.version}
+
+
+ org.apache.calcite
+ calcite-avatica
--- End diff --
Is this used?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59269604
--- Diff: phoenix-hive/pom.xml ---
@@ -0,0 +1,224 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+ 4.8.0-HBase-1.1-SNAPSHOT
+
+ phoenix-hive
+ Phoenix - Hive
+
+
+
+ org.apache.phoenix
+ phoenix-core
+
+
+ org.apache.phoenix
+ phoenix-core
+ tests
+ test
+
+
+ joda-time
+ joda-time
+
+
+ org.apache.hive
+ hive-exec
+ ${hive.version}
+
+
+ org.apache.calcite
+ *
+
+
+
+
+ org.apache.calcite
+ calcite-core
--- End diff --
Same here, unused?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59269847
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/PhoenixMetaHook.java ---
@@ -0,0 +1,245 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive;
+
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.hive.metastore.HiveMetaHook;
+import org.apache.hadoop.hive.metastore.TableType;
+import org.apache.hadoop.hive.metastore.api.FieldSchema;
+import org.apache.hadoop.hive.metastore.api.MetaException;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.phoenix.hive.constants.PhoenixStorageHandlerConstants;
+import org.apache.phoenix.hive.util.PhoenixConnectionUtil;
+import org.apache.phoenix.hive.util.PhoenixStorageHandlerUtil;
+import org.apache.phoenix.hive.util.PhoenixUtil;
+
+import com.google.common.base.CharMatcher;
+import com.google.common.base.Splitter;
+import com.google.common.collect.Lists;
+
+public class PhoenixMetaHook implements HiveMetaHook {
+
+private static final Log LOG =
LogFactory.getLog(PhoenixMetaHook.class);
+
+@Override
+public void preCreateTable(Table table) throws MetaException {
+if (LOG.isDebugEnabled()) {
+LOG.debug("Precreate table : " + table.getTableName());
+}
+
+try (Connection conn = PhoenixConnectionUtil.getConnection(table))
{
+String tableType = table.getTableType();
+String tableName =
PhoenixStorageHandlerUtil.getTargetTableName(table);
+
+if (TableType.EXTERNAL_TABLE.name().equals(tableType)) {
+// Check whether phoenix table exists.
+if (!PhoenixUtil.existTable(conn, tableName)) {
+// Error if phoenix table not exist.
+throw new MetaException("Phoenix table " + tableName +
" doesn't exist");
+}
+} else if (TableType.MANAGED_TABLE.name().equals(tableType)) {
+// Check whether phoenix table exists.
+if (PhoenixUtil.existTable(conn, tableName)) {
+// Error if phoenix table already exist.
+throw new MetaException("Phoenix table " + tableName +
" already exist.");
+}
+
+PhoenixUtil.createTable(conn, createTableStatement(table));
+} else {
+throw new MetaException("Unsupported table Type: " +
table.getTableType());
+}
+
+if (LOG.isDebugEnabled()) {
+LOG.debug("Phoenix table " + tableName + " was created");
+}
+} catch (SQLException e) {
+throw new MetaException(e.getMessage());
+}
+}
+
+private String createTableStatement(Table table) throws MetaException {
+Map tableParameterMap = table.getParameters();
+
+String tableName =
PhoenixStorageHandlerUtil.getTargetTableName(table);
+StringBuilder ddl = new StringBuilder("create table
").append(tableName).append(" (\n");
+
+String phoenixRowKeys =
tableParameterMap.get(PhoenixStorageHandlerConstants
+.PHOENIX_ROWKEYS);
+StringBuilder realRowKeys = new StringBuilder();
+List phoenixRowKeyList = Lists.newArrayList(Splitter.on
+
(PhoenixStorageHandlerConstants.COMMA).trimResults().split(phoenixRowKeys));
+Map columnMappingMap =
getColumnMappingMap(tableParameterMap.get
+
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59270355
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/PhoenixRowKey.java ---
@@ -0,0 +1,69 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive;
+
+import com.google.common.collect.Maps;
+import org.apache.hadoop.hive.ql.io.RecordIdentifier;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+import java.io.OutputStream;
+import java.util.Map;
+
+public class PhoenixRowKey extends RecordIdentifier {
+
+ private Map rowKeyMap = Maps.newHashMap();
+
+ public PhoenixRowKey() {
+
+ }
+
+ // public void add(String columnName, Object value) {
--- End diff --
Dead code
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59271033
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/constants/PhoenixStorageHandlerConstants.java
---
@@ -0,0 +1,101 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive.constants;
+
+import java.util.List;
+
+import org.apache.hadoop.io.IntWritable;
+
+import com.google.common.collect.Lists;
+
+/**
+ * Constants using for Hive Storage Handler implementation
+ */
+public class PhoenixStorageHandlerConstants {
+
+ public static final String HBASE_INPUT_FORMAT_CLASS =
"phoenix.input.format.class";
+
+ public static final String PHOENIX_TABLE_NAME = "phoenix.table.name";
+
+ public static final String DEFAULT_PHOENIX_INPUT_CLASS =
"com.lgcns.bigdata.platform.hive.hbase.phoenix.mapreduce.PhoenixResultWritable";
+
+ public static final String ZOOKEEPER_QUORUM =
"phoenix.zookeeper.quorum";
+public static final String ZOOKEEPER_PORT =
"phoenix.zookeeper.client.port";
+public static final String ZOOKEEPER_PARENT =
"phoenix.zookeeper.znode.parent";
+public static final String DEFAULT_ZOOKEEPER_QUORUM = "localhost";
+public static final int DEFAULT_ZOOKEEPER_PORT = 2181;
+public static final String DEFAULT_ZOOKEEPER_PARENT = "/hbase";
+
+public static final String PHOENIX_ROWKEYS = "phoenix.rowkeys";
+public static final String PHOENIX_COLUMN_MAPPING =
"phoenix.column.mapping";
+public static final String PHOENIX_TABLE_OPTIONS =
"phoenix.table.options";
+
+public static final String PHOENIX_TABLE_QUERY_HINT = ".query.hint";
+public static final String PHOENIX_REDUCER_NUMBER = ".reducer.count";
+public static final String DISABLE_WAL = ".disable.wal";
+public static final String BATCH_MODE = "batch.mode";
+public static final String AUTO_FLUSH = ".auto.flush";
+
+public static final String COLON = ":";
+ public static final String COMMA = ",";
+ public static final String EMPTY_STRING = "";
+ public static final String SPACE = " ";
+ public static final String LEFT_ROUND_BRACKET = "(";
+ public static final String RIGHT_ROUND_BRACKET = ")";
+ public static final String QUOTATION_MARK = "'";
+ public static final String EQUAL = "=";
+ public static final String IS = "is";
+ public static final String QUESTION = "?";
+
+ public static final String SPLIT_BY_STATS = "split.by.stats";
+ public static final String HBASE_SCAN_CACHE = "hbase.scan.cache";
+ public static final String HBASE_SCAN_CACHEBLOCKS =
"hbase.scan.cacheblock";
+ public static final String HBASE_DATE_FORMAT = "hbase.date.format";
+ public static final String HBASE_TIMESTAMP_FORMAT =
"hbase.timestamp.format";
+ public static final String DEFAULT_DATE_FORMAT = "-MM-dd";
+ public static final String DEFAULT_TIMESTAMP_FORMAT = "-MM-dd
HH:mm:ss.SSS";
+
+ public static final String IN_OUT_WORK = "in.out.work";
+ public static final String IN_WORK = "input";
+ public static final String OUT_WORK = "output";
+
+ public static final String MR = "mr";
+ public static final String TEZ = "tez";
+ public static final String SPARK = "spark";
+
+ public static final String DATE_TYPE = "date";
+ public static final String TIMESTAMP_TYPE = "timestamp";
+ public static final String BETWEEN_COMPARATOR = "between";
+ public static fi
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59272080
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/objectinspector/PhoenixObjectInspectorFactory.java
---
@@ -0,0 +1,150 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive.objectinspector;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters;
+import
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyObjectInspectorFactory;
+import
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.ObjectInspectorOptions;
+import org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfo;
+
+public class PhoenixObjectInspectorFactory {
--- End diff --
I'm wondering why you need this factory (and the accompanying OI
implementations for the different types) and you can't reuse the Hive
ObjectInspector implementations. Something to do with strongly-typed values
from Phoenix (as opposed to HBase/Accumulo's implementations)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59273613
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/mapreduce/PhoenixResultWritable.java
---
@@ -0,0 +1,215 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive.mapreduce;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.sql.SQLException;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configurable;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.io.Writable;
+import org.apache.hadoop.mapreduce.lib.db.DBWritable;
+import org.apache.phoenix.hive.PhoenixRowKey;
+import org.apache.phoenix.hive.constants.PhoenixStorageHandlerConstants;
+import org.apache.phoenix.hive.util.PhoenixStorageHandlerUtil;
+import org.apache.phoenix.hive.util.PhoenixUtil;
+import org.apache.phoenix.util.ColumnInfo;
+
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+/**
+ * Serialized class for SerDe
+ *
+ */
+public class PhoenixResultWritable implements Writable, DBWritable,
Configurable {
+
+private static final Log LOG =
LogFactory.getLog(PhoenixResultWritable.class);
+
+private List columnMetadataList;
+private List valueList;// for output
+private Map rowMap = Maps.newHashMap(); // for input
+
+private int columnCount = -1;
+
+private Configuration config;
+private boolean isTransactional;
+private Map rowKeyMap = Maps.newLinkedHashMap();
+private List primaryKeyColumnList;
+
+public PhoenixResultWritable() {
+}
+
+public PhoenixResultWritable(Configuration config) throws IOException {
+setConf(config);
+}
+
+public PhoenixResultWritable(Configuration config, List
columnMetadataList) throws IOException {
+this(config);
+this.columnMetadataList = columnMetadataList;
+
+valueList =
Lists.newArrayListWithExpectedSize(columnMetadataList.size());
+}
+
+@Override
+public void write(DataOutput out) throws IOException {
+}
--- End diff --
No need to implement these?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59273812
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/ql/index/IndexPredicateAnalyzer.java
---
@@ -0,0 +1,486 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive.ql.index;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Stack;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.hive.ql.exec.FunctionRegistry;
+import org.apache.hadoop.hive.ql.lib.DefaultGraphWalker;
+import org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher;
+import org.apache.hadoop.hive.ql.lib.Dispatcher;
+import org.apache.hadoop.hive.ql.lib.GraphWalker;
+import org.apache.hadoop.hive.ql.lib.Node;
+import org.apache.hadoop.hive.ql.lib.NodeProcessor;
+import org.apache.hadoop.hive.ql.lib.NodeProcessorCtx;
+import org.apache.hadoop.hive.ql.lib.Rule;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+import org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDescUtils;
+import org.apache.hadoop.hive.ql.plan.ExprNodeFieldDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFBaseCompare;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFBetween;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNot;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToBinary;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToChar;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDecimal;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUtcTimestamp;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToVarchar;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory;
+
+import com.google.common.collect.Lists;
+
+/**
+ * Clone of org.apache.hadoop.hive.ql.index.IndexPredicateAnalyzer with
modifying
+ * analyzePredicate method.
+ *
+ *
+ */
+public class IndexPredicateAnalyzer {
+
+private static final Log LOG =
LogFactory.getLog(IndexPredicateAnalyzer.class);
+
+private final Set udfNames;
+private final Map> columnToUDFs;
+private FieldValidator fieldValidator;
+
+private boolean acceptsFields;
+
+public IndexPredicateAnalyzer() {
+udfNames = new HashSet();
+columnToUDFs = new HashMap>();
+}
+
+public void setFieldValidator(FieldValidator fieldValidator) {
+this.fieldValidator = fieldValidator;
+}
+
+/**
+ * Registers a comparison operator as one which can be satisfied by an
index
+ * search. Unless this is called, analyzePredicate will never find any
+ * indexable conditions.
+ *
+ * @param udfName name of comparison operator as returned by either
+ *{@link GenericUDFBridge#getUdfName} (for simple
UDF's) or
+ *udf.getClass().getName() (for generic UDF's).
+ */
+public
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on the pull request: https://github.com/apache/phoenix/pull/155#issuecomment-208542873 Some general thoughts (I stopped leaving them inline everytime I saw them). I'm guessing you "inherited" some of these from JeongMin's original work. * Dbl-check indentations * Try to remove commented out code * Some class-level javadoc comments would be *amazing* * Not a single unit test? :) Other things that I remember biting me previously: * Make sure you try to run with Tez as well. Both in the "uber" (local job) mode and a normal tez task. There are.. subtleties between them, sadly (as sadly, I don't remember the specifics anymore). Other general thoughts: * The RecordUpdater implementation looks pretty cool. Didn't know they made this available for StorageHandlers. * Hive has a decent suite for running Hive tests as a part of their build (which includes tests for StorageHandlers) with this qtest/itest modules. You might be able to take some inspiration from these for testing. Looks good so far. It will be a nice bridge between Phoenix and Hive (as we work towards a common-core of Calcite). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on the pull request: https://github.com/apache/phoenix/pull/155#issuecomment-208626268 > it's hard to get all those miniclusters run together. I still hope to get it resolved. That's why the pom contains dependencies on calcite, hbase-test and others. If it's just at the test scope, you might be able to circumvent the issue by just overriding the hbase and hadoop versions to compatible versions only at the test scope. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59558405
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/objectinspector/PhoenixObjectInspectorFactory.java
---
@@ -0,0 +1,150 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive.objectinspector;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters;
+import
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyObjectInspectorFactory;
+import
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.ObjectInspectorOptions;
+import org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfo;
+
+public class PhoenixObjectInspectorFactory {
--- End diff --
Haha, ok. I want to say that I think they are superfluous -- Hive already
provides implementations for the OI's. {HBase,Accumulo}StorageHandler should
already have examples on how they work. It would reduce the amount of code to
maintain here which is great.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/155#discussion_r59558697
--- Diff:
phoenix-hive/src/main/java/org/apache/phoenix/hive/mapreduce/PhoenixResultWritable.java
---
@@ -0,0 +1,215 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.hive.mapreduce;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.sql.SQLException;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configurable;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.io.Writable;
+import org.apache.hadoop.mapreduce.lib.db.DBWritable;
+import org.apache.phoenix.hive.PhoenixRowKey;
+import org.apache.phoenix.hive.constants.PhoenixStorageHandlerConstants;
+import org.apache.phoenix.hive.util.PhoenixStorageHandlerUtil;
+import org.apache.phoenix.hive.util.PhoenixUtil;
+import org.apache.phoenix.util.ColumnInfo;
+
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+/**
+ * Serialized class for SerDe
+ *
+ */
+public class PhoenixResultWritable implements Writable, DBWritable,
Configurable {
+
+private static final Log LOG =
LogFactory.getLog(PhoenixResultWritable.class);
+
+private List columnMetadataList;
+private List valueList;// for output
+private Map rowMap = Maps.newHashMap(); // for input
+
+private int columnCount = -1;
+
+private Configuration config;
+private boolean isTransactional;
+private Map rowKeyMap = Maps.newLinkedHashMap();
+private List primaryKeyColumnList;
+
+public PhoenixResultWritable() {
+}
+
+public PhoenixResultWritable(Configuration config) throws IOException {
+setConf(config);
+}
+
+public PhoenixResultWritable(Configuration config, List
columnMetadataList) throws IOException {
+this(config);
+this.columnMetadataList = columnMetadataList;
+
+valueList =
Lists.newArrayListWithExpectedSize(columnMetadataList.size());
+}
+
+@Override
+public void write(DataOutput out) throws IOException {
+}
--- End diff --
Maybe add `throws new UnsupportedOperationException()` then?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/155#discussion_r59953160 --- Diff: phoenix-hive/pom.xml --- @@ -117,66 +93,44 @@ - org.apache.hbase - hbase-common - - - org.apache.hbase - hbase-protocol - - - org.apache.hbase - hbase-client - - - org.apache.hbase - hbase-hadoop-compat - test - - - org.apache.hbase - hbase-hadoop-compat - test-jar - test - - - org.apache.hbase - hbase-hadoop2-compat + org.apache.hadoop + hadoop-hdfs + 2.7.1 test - org.apache.hbase - hbase-hadoop2-compat + org.apache.hadoop + hadoop-hdfs + 2.7.1 --- End diff -- Given James' previous comment, I think you can safely change the hadoop version at the top-level pom. Then, you won't have to specify 2.7.1 down here in phoenix-hive. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/155#discussion_r59953321 --- Diff: phoenix-hive/src/main/java/org/apache/phoenix/hive/PhoenixMetaHook.java --- @@ -6,9 +6,9 @@ * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * + * --- End diff -- Adding in the paragraph tags might break the RAT check. Best to revert that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on the pull request: https://github.com/apache/phoenix/pull/155#issuecomment-210678972 Recent improvements look good! The new integration test is *very* nice. The ObjectInspector stuff isn't a huge deal to replace, but it's something that can be consolidated later. Left a few minor comments, but I this is good to go after those are fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-2743 Hive Storage support
Github user joshelser commented on the pull request:
https://github.com/apache/phoenix/pull/155#issuecomment-211454505
@ss77892 one more nit-pick, it seems like you removed the version for some
of the entries in the pom but then added `${hadoop-two.version}` for others. I
would assume that it's not necessary for any of them, but let me know if I'm
wrong. I can also just fix this while merging it in as long as @JamesRTaylor
thinks this good to go too :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/159#discussion_r66094488
--- Diff: phoenix-client/pom.xml ---
@@ -0,0 +1,381 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+4.8.0-HBase-1.2-SNAPSHOT
+
+ phoenix-client
+ Phoenix Client
+ Phoenix Client
+ jar
+
+
+true
+
+true
+true
+${project.basedir}/..
+org.apache.phoenix.shaded
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-site-plugin
+
+ true
+
+
+
+
+org.apache.maven.plugins
+maven-jar-plugin
+
+
+default-jar
+none
+
+
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
+
+
+package
+
+ shade
+
+
+ phoenix-${project.version}-client
+ false
+
true
+ false
+
+
+
+
+
+LICENSE.txt
+ASL2.0
+
+
+
+ false
+
+
+
+ csv-bulk-load-config.properties
+
+
${project.basedir}/../config/csv-bulk-load-config.properties
+
+
+
+ README.md
+ ${project.basedir}/../README.md
+
+
+ LICENSE.txt
+ ${project.basedir}/../LICENSE.txt
+
+
+
+
+ *:*
+
+
+ org.apache.phoenix:phoenix-client
+
+
+
+
+
+
+
+ com.codahale
+
${shaded.package}.com.codahale
+
+
+ com.fasterxml
+
${shaded.package}.com.fasterxml
+
+
+ com.google
+
${shaded.package}.com.google
+
+
+
+
+
+
+ com.jamesmurty
+
${shaded.package}.com.jamesmurty
+
+
+ com.jcraft
+
${shaded.package}.com.jcraft
+
+
+ com.lmax
+ ${shaded.package}.com.lmax
+
+
+ com.thoughtworks
+
${shaded.package}.com.thoughtworks
+
+
+ com.sun.jersey
+
${shaded.package}.com.sun.jersey
+
+
+ com.yammer
+
${shaded.package}.com.yammer
+
+
+
+
+ io.netty
+ ${shaded.package}.io.netty
+
+
+
+
+ org.antlr
+
${shaded.package}.org.antlr
+
+
+ org.aopalliance
+
${shaded.package}.org.aopalliance
+
+
+ org.codehaus
+
${shaded.package}.org.codehaus
+
+
+ org.eclipse.jetty
+
${shaded.package}.org.eclipse.jetty
+
+
+ org.fusesource
+
${shaded.package}.org.fusesource
+
+
+ org.hamcrest
+
${shaded.package}.org.hamcrest
+
+
+
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/159#discussion_r66094042 --- Diff: phoenix-server/pom.xml --- @@ -7,52 +30,98 @@ 4.8.0-HBase-1.2-SNAPSHOT phoenix-server - Phoenix Query Server - A query server for exposing Phoenix to thin clients - - - - The Apache Software License, Version 2.0 - http://www.apache.org/licenses/LICENSE-2.0.txt - repo - - - - - -Apache Software Foundation -http://www.apache.org - - + Phoenix Server + Phoenix Server --- End diff -- Nit: A better description would be awesome --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/159#discussion_r66094212 --- Diff: bin/queryserver.py --- @@ -119,7 +119,7 @@ # The command is run through subprocess so environment variables are automatically inherited java_cmd = '%(java)s -cp ' + hbase_config_path + os.pathsep + hadoop_config_path + os.pathsep + \ -phoenix_utils.phoenix_queryserver_jar + os.pathsep + phoenix_utils.phoenix_client_jar + \ +phoenix_utils.phoenix_client_jar + \ --- End diff -- Why is the queryserver jar no longer included when launching the queryserver? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/159#discussion_r66093991 --- Diff: phoenix-client/pom.xml --- @@ -0,0 +1,381 @@ + + + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + 4.0.0 + +org.apache.phoenix +phoenix +4.8.0-HBase-1.2-SNAPSHOT + + phoenix-client + Phoenix Client + Phoenix Client --- End diff -- Nit: A better description would be awesome --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/159#discussion_r66096699
--- Diff: phoenix-client/pom.xml ---
@@ -0,0 +1,381 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+4.8.0-HBase-1.2-SNAPSHOT
+
+ phoenix-client
+ Phoenix Client
+ Phoenix Client
+ jar
+
+
+true
+
+true
+true
+${project.basedir}/..
+org.apache.phoenix.shaded
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-site-plugin
+
+ true
+
+
+
+
+org.apache.maven.plugins
+maven-jar-plugin
+
+
+default-jar
+none
+
+
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
+
+
+package
+
+ shade
+
+
+ phoenix-${project.version}-client
+ false
+
true
+ false
+
+
+
+
+
+LICENSE.txt
+ASL2.0
+
+
+
+ false
+
+
+
+ csv-bulk-load-config.properties
+
+
${project.basedir}/../config/csv-bulk-load-config.properties
+
+
+
+ README.md
+ ${project.basedir}/../README.md
+
+
+ LICENSE.txt
+ ${project.basedir}/../LICENSE.txt
+
+
+
+
+ *:*
+
+
+ org.apache.phoenix:phoenix-client
+
+
+
+
+
+
+
+ com.codahale
+
${shaded.package}.com.codahale
+
+
+ com.fasterxml
+
${shaded.package}.com.fasterxml
+
+
+ com.google
+
${shaded.package}.com.google
+
+
+
+
+
+
+ com.jamesmurty
+
${shaded.package}.com.jamesmurty
+
+
+ com.jcraft
+
${shaded.package}.com.jcraft
+
+
+ com.lmax
+ ${shaded.package}.com.lmax
+
+
+ com.thoughtworks
+
${shaded.package}.com.thoughtworks
+
+
+ com.sun.jersey
+
${shaded.package}.com.sun.jersey
+
+
+ com.yammer
+
${shaded.package}.com.yammer
+
+
+
+
+ io.netty
+ ${shaded.package}.io.netty
+
+
+
+
+ org.antlr
+
${shaded.package}.org.antlr
+
+
+ org.aopalliance
+
${shaded.package}.org.aopalliance
+
+
+ org.codehaus
+
${shaded.package}.org.codehaus
+
+
+ org.eclipse.jetty
+
${shaded.package}.org.eclipse.jetty
+
+
+ org.fusesource
+
${shaded.package}.org.fusesource
+
+
+ org.hamcrest
+
${shaded.package}.org.hamcrest
+
+
+
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/159#discussion_r66097370
--- Diff: phoenix-assembly/src/build/components/all-common-jars.xml ---
@@ -88,12 +93,12 @@
${project.basedir}/../phoenix-server/target/
--- End diff --
Need to change this directory. There is no phoenix-queryserver*.jar
included in the lib/ directory in the tarball.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/159#discussion_r66099573
--- Diff: phoenix-client/pom.xml ---
@@ -0,0 +1,381 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+4.8.0-HBase-1.2-SNAPSHOT
+
+ phoenix-client
+ Phoenix Client
+ Phoenix Client
+ jar
+
+
+true
+
+true
+true
+${project.basedir}/..
+org.apache.phoenix.shaded
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-site-plugin
+
+ true
+
+
+
+
+org.apache.maven.plugins
+maven-jar-plugin
+
+
+default-jar
+none
+
+
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
+
+
+package
+
+ shade
+
+
+ phoenix-${project.version}-client
+ false
+
true
+ false
+
+
+
+
+
+LICENSE.txt
+ASL2.0
+
+
+
+ false
+
+
+
+ csv-bulk-load-config.properties
+
+
${project.basedir}/../config/csv-bulk-load-config.properties
+
+
+
+ README.md
+ ${project.basedir}/../README.md
+
+
+ LICENSE.txt
+ ${project.basedir}/../LICENSE.txt
+
+
+
+
+ *:*
+
+
+ org.apache.phoenix:phoenix-client
+
+
+
+
+
+
+
+ com.codahale
+
${shaded.package}.com.codahale
+
+
+ com.fasterxml
+
${shaded.package}.com.fasterxml
+
+
+ com.google
+
${shaded.package}.com.google
+
+
+
+
+
+
+ com.jamesmurty
+
${shaded.package}.com.jamesmurty
+
+
+ com.jcraft
+
${shaded.package}.com.jcraft
+
+
+ com.lmax
+ ${shaded.package}.com.lmax
+
+
+ com.thoughtworks
+
${shaded.package}.com.thoughtworks
+
+
+ com.sun.jersey
+
${shaded.package}.com.sun.jersey
+
+
+ com.yammer
+
${shaded.package}.com.yammer
+
+
+
+
+ io.netty
+ ${shaded.package}.io.netty
+
+
+
+
+ org.antlr
+
${shaded.package}.org.antlr
+
+
+ org.aopalliance
+
${shaded.package}.org.aopalliance
+
+
+ org.codehaus
+
${shaded.package}.org.codehaus
+
+
+ org.eclipse.jetty
+
${shaded.package}.org.eclipse.jetty
+
+
+ org.fusesource
+
${shaded.package}.org.fusesource
+
+
+ org.hamcrest
+
${shaded.package}.org.hamcrest
+
+
+
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/159#discussion_r66099747 --- Diff: bin/queryserver.py --- @@ -119,7 +119,7 @@ # The command is run through subprocess so environment variables are automatically inherited java_cmd = '%(java)s -cp ' + hbase_config_path + os.pathsep + hadoop_config_path + os.pathsep + \ -phoenix_utils.phoenix_queryserver_jar + os.pathsep + phoenix_utils.phoenix_client_jar + \ +phoenix_utils.phoenix_client_jar + \ --- End diff -- Right, realized that later :). Given my opinion on removing the queryserver classes from the client jar, I think this should be switched back to queryserver + client. It makes a more natural usage, IMO. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/159#discussion_r66113124
--- Diff: phoenix-client/pom.xml ---
@@ -0,0 +1,381 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ 4.0.0
+
+org.apache.phoenix
+phoenix
+4.8.0-HBase-1.2-SNAPSHOT
+
+ phoenix-client
+ Phoenix Client
+ Phoenix Client
+ jar
+
+
+true
+
+true
+true
+${project.basedir}/..
+org.apache.phoenix.shaded
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-site-plugin
+
+ true
+
+
+
+
+org.apache.maven.plugins
+maven-jar-plugin
+
+
+default-jar
+none
+
+
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
+
+
+package
+
+ shade
+
+
+ phoenix-${project.version}-client
+ false
+
true
+ false
+
+
+
+
+
+LICENSE.txt
+ASL2.0
+
+
+
+ false
+
+
+
+ csv-bulk-load-config.properties
+
+
${project.basedir}/../config/csv-bulk-load-config.properties
+
+
+
+ README.md
+ ${project.basedir}/../README.md
+
+
+ LICENSE.txt
+ ${project.basedir}/../LICENSE.txt
+
+
+
+
+ *:*
+
+
+ org.apache.phoenix:phoenix-client
+
+
+
+
+
+
+
+ com.codahale
+
${shaded.package}.com.codahale
+
+
+ com.fasterxml
+
${shaded.package}.com.fasterxml
+
+
+ com.google
+
${shaded.package}.com.google
+
+
+
+
+
+
+ com.jamesmurty
+
${shaded.package}.com.jamesmurty
+
+
+ com.jcraft
+
${shaded.package}.com.jcraft
+
+
+ com.lmax
+ ${shaded.package}.com.lmax
+
+
+ com.thoughtworks
+
${shaded.package}.com.thoughtworks
+
+
+ com.sun.jersey
+
${shaded.package}.com.sun.jersey
+
+
+ com.yammer
+
${shaded.package}.com.yammer
+
+
+
+
+ io.netty
+ ${shaded.package}.io.netty
+
+
+
+
+ org.antlr
+
${shaded.package}.org.antlr
+
+
+ org.aopalliance
+
${shaded.package}.org.aopalliance
+
+
+ org.codehaus
+
${shaded.package}.org.codehaus
+
+
+ org.eclipse.jetty
+
${shaded.package}.org.eclipse.jetty
+
+
+ org.fusesource
+
${shaded.package}.org.fusesource
+
+
+ org.hamcrest
+
${shaded.package}.org.hamcrest
+
+
+
[GitHub] phoenix pull request #159: PHOENIX-2535 Create shaded clients (thin + thick)
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/159#discussion_r66280325
--- Diff: phoenix-queryserver/pom.xml ---
@@ -24,6 +26,50 @@
org.apache.maven.plugins
maven-failsafe-plugin
+
+org.apache.maven.plugins
+maven-shade-plugin
+
+
+query-server
+package
+
+ shade
+
+
+ phoenix-${project.version}-queryserver
+ false
+
true
+ false
+
+
+ org.apache.phoenix:phoenix-server
--- End diff --
Shouldn't these be phoenix-queryserver and phoenix-queryserver-client?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request #183: PHOENIX-3084 source release licensing issues.
GitHub user joshelser opened a pull request: https://github.com/apache/phoenix/pull/183 PHOENIX-3084 source release licensing issues. Fixes for the Apache Phoenix source release You can merge this pull request into a Git repository by running: $ git pull https://github.com/joshelser/phoenix licensing Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/183.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #183 commit 50b655e3c8ba5a4887da2b2168ea0afbabd00d42 Author: Josh Elser Date: 2016-07-18T21:56:05Z PHOENIX-3086 Release artifacts should be "apache-phoenix*" commit b6b58467b2a9cb5a3bb4a74d978aea504788285f Author: Josh Elser Date: 2016-07-18T22:18:07Z PHOENIX-3085 Remove Copyright ASF from license headers commit 7f54cd8e14511b618ec528a42311d35f6b93d59e Author: Josh Elser Date: 2016-07-18T22:36:30Z PHOENIX-3087 Add missing license headers to files commit f91cc84ff3f8101c52c7b3343262ffea3436dbc7 Author: Josh Elser Date: 2016-07-18T22:55:08Z PHOENIX-3088 Drastically reduce apache-rat-plugin excludes The apache-rat-plugin is already configured to exclude many of the things that we were overriding the exclusions in pluginManagement to exclude for us. All of these were removed. Exclusion configuration moved from pluginMangement to plugins to avoid unintentional exclusions in child modules (that were only meant to applied at the parent pom) Also removes manual configuration of apache-rat-plugin version to use the version set in apache parent pom. commit ad856b2b25afa71ee5c0664b1b4beefd42797e54 Author: Josh Elser Date: 2016-07-19T02:57:16Z PHOENIX-3089 PHOENIX-3090 Fix L&N for source release --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #183: PHOENIX-3084 source release licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71396256 --- Diff: examples/STOCK_SYMBOL.sql --- @@ -1,3 +1,19 @@ +-- Licensed to the Apache Software Foundation (ASF) under one +-- or more contributor license agreements. See the NOTICE file +-- distributed with this work for additional information +-- regarding copyright ownership. The ASF licenses this file +-- to you under the Apache License, Version 2.0 (the +-- "License"); you may not use this file except in compliance +-- with the License. You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, software +-- distributed under the License is distributed on an "AS IS" BASIS, +-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +-- See the License for the specific language governing permissions and +-- limitations under the License. + --- End diff -- Great. Thanks for the catch! I totally did not use it. Let me re-add these to the exclusions and make an issue to track this information. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #183: PHOENIX-3084 source release licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71396909 --- Diff: LICENSE --- @@ -200,3 +200,90 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. + +--- + +This product bundles Sqlline (https://github.com/julianhyde/sqlline) +which is licensed under the 3-clause BSD license + +Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux +Copyright (c) 2004-2010 The Eigenbase Project +Copyright (c) 2013-2014 Julian Hyde +All rights reserved. + +--- + +This product bundles portions of AngularJS (https://angularjs.org/) which --- End diff -- The text added here should be all that is needed for 4.8.0's source release for the trace ui stuff. The accompanying binary artifact just needs the same text copied into its LICENSE and NOTICE. While we can move this out for 4.8.0, I think I'd say we can decide what to do in a 4.9.0 (does this code get dropped, moved into some other repo, etc). We certainly can just remove it now if desired. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #183: PHOENIX-3084 source release licensing issues.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/183 7d9adab removes the license headers from `examples/**/*.sql` and re-adds the exclusions to apache-rat-plugin. Filed https://issues.apache.org/jira/browse/PHOENIX-3099 to track it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #183: PHOENIX-3084 source release licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71407328 --- Diff: LICENSE --- @@ -200,3 +200,90 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. + +--- + +This product bundles Sqlline (https://github.com/julianhyde/sqlline) +which is licensed under the 3-clause BSD license + +Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux +Copyright (c) 2004-2010 The Eigenbase Project +Copyright (c) 2013-2014 Julian Hyde +All rights reserved. + +--- + +This product bundles portions of AngularJS (https://angularjs.org/) which --- End diff -- Oh right. That was done in 7f54cd8. Still though, that was straightforward to fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #183: 4.8.0 licensing issues.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/183 Ok, I think this one is good to go. Any chance I could bug some folks to glance over these changes (I know they're massive). @billierinaldi, @joewitt, @apurtell, @busbey FYI @JamesRTaylor and @ankitsinghal (to know things are close). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71724014 --- Diff: NOTICE --- @@ -1,37 +1,18 @@ +Apache Phoenix +Copyright 2013-2016 The Apache Software Foundation + This product includes software developed by The Apache Software Foundation (http://www.apache.org/). -It includes software from other Apache Software Foundation projects, -including, but not limited to: - - Apache HBase - - Apache Hadoop - - Apache Commons - -In addition, this product includes/uses software developed by: - -JLine (http://jline.sourceforge.net/), -Copyright (c) 2002-2006, Marc Prud'hommeaux . - -SQLLine ((http://jline.sourceforge.net/), -Copyright (c) 2002-2006, Marc Prud'hommeaux . - -SLF4J (http://www.slf4j.org/), -Copyright (c) 2004-2008 QOS.ch - -ANTLR (http://www.antlr.org/), -Copyright (c) 2003-2008, Terrence Parr. - -JUnit (http://www.junit.org/) included under the Common Public License v1.0. -See the full text here: http://junit.sourceforge.net/cpl-v10.html +This also includes: The phoenix-spark module has been adapted from the phoenix-spark library distributed under the terms of the Apache 2 license. Original source copyright: Copyright 2014 Simply Measured, Inc. Copyright 2015 Interset Software Inc. The file bin/daemon.py is based on the file of the same name in python-daemon 2.0.5 -(https://pypi.python.org/pypi/python-daemon/) distributed under the terms of -the Apache 2 license. Original source copyright: --- End diff -- No, it is ASLv2. Omitting because it's unnecessary to state because it is ASLv2. Just propagating the copyright information. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71724215 --- Diff: NOTICE --- @@ -1,37 +1,18 @@ +Apache Phoenix +Copyright 2013-2016 The Apache Software Foundation + This product includes software developed by The Apache Software Foundation (http://www.apache.org/). -It includes software from other Apache Software Foundation projects, -including, but not limited to: - - Apache HBase - - Apache Hadoop - - Apache Commons - -In addition, this product includes/uses software developed by: - -JLine (http://jline.sourceforge.net/), -Copyright (c) 2002-2006, Marc Prud'hommeaux . - -SQLLine ((http://jline.sourceforge.net/), -Copyright (c) 2002-2006, Marc Prud'hommeaux . - -SLF4J (http://www.slf4j.org/), -Copyright (c) 2004-2008 QOS.ch - -ANTLR (http://www.antlr.org/), -Copyright (c) 2003-2008, Terrence Parr. - -JUnit (http://www.junit.org/) included under the Common Public License v1.0. -See the full text here: http://junit.sourceforge.net/cpl-v10.html +This also includes: The phoenix-spark module has been adapted from the phoenix-spark library distributed under the terms of the Apache 2 license. Original source copyright: --- End diff -- I suppose under the same logic as the daemon.py entry, I could drop that too. AFAIK, these were not verbatim "copies" from other NOTICE files, just stating that they were copied and not original works. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71724269 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71724651 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71727042 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71727352 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71727249 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71728385 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71729382 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71731040 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/183#discussion_r71731461 --- Diff: dev/release_files/LICENSE --- @@ -201,28 +201,1238 @@ See the License for the specific language governing permissions and limitations under the License. -== +-- + +Apache Phoenix binary artifact bundled components. Full licenses +for bundled software is available at the end of this file if not +included inline. + +-- +This product bundles the following products which are licensed with +the 3-Clause BSD License. -BSD Clause 3/New BSD License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-3-Clause +Sqlline (https://github.com/julianhyde/sqlline) + + Copyright (c) 2002,2003,2004,2005,2006,2007 Marc Prud'hommeaux + Copyright (c) 2004-2010 The Eigenbase Project + Copyright (c) 2013-2014 Julian Hyde + +ASM 3.1 (http://asm.ow2.org) Copyright (c) 2000-2011 INRIA, France Telecom Snappy, version 1.1.0.1, http://code.google.com/p/snappy/ -Protobuf, version 2.4.0, https://code.google.com/p/protobuf/ -SQLLine, version 1.1.2, https://github.com/julianhyde/sqlline/ -== +Google Protobuf Java, version 2.5.0, (https://code.google.com/p/protobuf/), Copyright 2008, Google Inc. +Google Protobuf Java, version 3.0.0-beta-1, (https://code.google.com/p/protobuf/), Copyright 2014, Google Inc. -BSD Clause 2 License, applies to the following bundled libraries: -LICENSE: http://opensource.org/licenses/BSD-2-Clause +JSch, version 0.1.42 (http://www.jcraft.com/jsch) Copyright 2002-2015 Atsuhiko Yamanaka, JCraft,Inc. -JLine, version 2.11, http://jline.sourceforge.net/ -ANTLR, version 3.5, http://www.antlr.org/ +Paranamer, version 2.3 (https://github.com/paul-hammant/paranamer/) Copyright (c) 2006 Paul Hammant & ThoughtWorks Inc -== +JLine version 2.11 (https://github.com/jline/jline2) Copyright (c) 2002-2016, the original author or authors. -MIT License, applies to the following bundled library: -LICENSE: http://opensource.org/licenses/MIT +ANTLR String Template 4.0.4 (http://www.stringtemplate.org) Copyright StringTemplate / Terence Parr 2013 -SLF4J, version 1.4.3, http://www.slf4j.org/ +ANTLR Runtime 3.5.2 (https://github.com/antlr/antlr3) Copyright (c) 2013 Terence Parr + +leveldbjni 1.8 (http://fusesource.com) Copyright (C) 2011, FuseSource Corp + +Hamcrest 1.3 (www.hamcrest.org) Copyright (c) 2000-2015 www.hamcrest.org + +Snappy 0.3 (https://github.com/dain/snappy) + Copyright 2011 Dain Sundstrom d...@iq80.com + Copyright 2011, Google inc.opensou...@google.com + +OWASP AntiSamy 1.4.3 (https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project) + +OWASP ESAPI 2.1.0 (https://github.com/ESAPI/esapi-java-legacy) Copyright (c) 2007, The OWASP Foundation + +xmlenc (http://xmlenc.sourceforge.net/) Copyright 2003-2005, Ernst de Haan + +Janino Compiler (https://github.com/janino-compiler/janino) + Copyright (c) 2001-2016, Arno Unkrig + Copyright (c) 2015-2016 TIBCO Software Inc. + +Hamcrest-core 1.3 (http://www.hamcrest.org) Copyright (c) 2000-2006, www.hamcrest.org + +--- + +This product bundles the following products which are licensed with +the MIT License + +AngularJS (https://angularjs.org/) Copyright (c) 2010-2016 Google, Inc. + +Bootstrap (http://getbootstrap.com/) Copyright (c) 2011-2015 Twitter, Inc. +GLYPHICONS Halflings is also included as a part of Bootstrap. + +jQuery (https://jquery.org/) Copyright 2005, 2014 jQuery Foundation, Inc. and +other contributors + +sizzle.js (http://sizzlejs.com) Copyright 2008, 2014 jQuery Foundation, Inc. and +other contributors + +Angular Google Chart (https://github.com/angular-google-chart/angular-google-chart) +Copyright (c) 2013 Nicolas Bouillon + +angular-ui-bootstrap (http://angular-ui.github.io/bootstrap/) Copyright (c) 2012-2016 the AngularUI +Team, https://github.com/organizations/angular-ui/teams/291112 + +Font Awesome CSS (http://fontawesome.io/). + +JRuby jcodings 1.0.8 (https://github.com/jruby/jcodings) + +JRuby joni 2.1.2 (https://github.com/jruby/joni) + +SLF4j (http://slf4j.org) Copyright (c) 2004-2013 QOS.ch + +HSQLDB (http://hsqldb.org/) + + For work developed by the HSQL Developm
[GitHub] phoenix issue #183: 4.8.0 licensing issues.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/183 Thanks again for taking a look at this, Sean! You found some really nice brain farts from me. Changes made: * Clarified CDDL 1.0 and 1.1 references in LICENSE * Included CDDL 1.0 and 1.1 text in LICENSE * Corrected the LICENSE mentions for hbase-server (bad copy-paste) * Clarified CC by A 2.5 and 3.0 products in LICENSE * Included CC by A 2.5 and 3.0 texts in LICENSE * Remove an unnecessary mention to ASLv2 in NOTICE * Removed all trailing whitespace in L&N --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #183: 4.8.0 licensing issues.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/183 I think this is good to go. Does anyone else want to look at this before I merge it in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #183: 4.8.0 licensing issues.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/183 @apurtell I got some emails from you on the naming of the tarballs, but I'm struggling to find the thread here on GH. I'm not actually 100% positive if the artifacts being named "phoenix" and not "apache-phoenix" is against policy. I would be OK reverting that change if it's not actually required that the names are "apache-foo" (and only recommended) -- seeing if I can find the relevant policy docs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #183: 4.8.0 licensing issues.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/183 Thanks for all of the help everyone. This was merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #183: 4.8.0 licensing issues.
Github user joshelser closed the pull request at: https://github.com/apache/phoenix/pull/183 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 Hi @btbytes! Thanks for the changes so far. A couple of things: 1. Have you opened a JIRA issue which captures the changes you'd like to make? We want to make sure that JIRA is the source system of record for discussing changes. You should be able to create an "Improvement" issue under https://issues.apache.org/jira/browse/PHOENIX. If you haven't already, please do so, and then let me know the URL and I can assign it to you to for credit. 2. Any thoughts about making the corresponding changes to sqlline.py that you made here to sqlline-thin.py? 3. Any thoughts about testing these changes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 Great! I've given you karma and assigned PHOENIX-3132 to you. > I was waiting for a response do this PR before I took on more refactoring ;) Hah! Smart :) > I have tested in so far as the output of the two versions of the program are the same before they are shelled out in the last line. So, yes. Let me clarify. I was wondering if, in your changes to be a bit more idiomatic, you had made any changes which would help us work towards automated unit tests which we could run during the normal Phoenix Maven build. I've had it on my backburner to look at what Apache Ambari does to invoke unit tests on python code and try to borrow some of that code so that we can try to improve the quality of our python scripts and increase our confidence in future changes (such as your's). I don't want to push that onto you, but was just curious if you had written any sort of tests as a part of these changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/186#discussion_r73024052 --- Diff: bin/sqlline-thin.py --- @@ -18,64 +18,58 @@ # limitations under the License. # - +from __future__ import print_function --- End diff -- I was curious what this was doing and it looks like it's some magic to make print in python3 work like it does in python2 (http://stackoverflow.com/questions/4560804/why-do-we-invoke-print-after-importing-print-function-in-python-2-6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/186#discussion_r73024268
--- Diff: bin/sqlline-thin.py ---
@@ -86,73 +80,83 @@ def get_serialization():
return default_serialization
return stdout
-if len(sys.argv) == 1:
-pass
-elif len(sys.argv) == 2:
-if os.path.isfile(sys.argv[1]):
-sqlfile = sys.argv[1]
-else:
+
+def main():
+url = 'localhost:8765'
+sqlfile = ''
+if len(sys.argv) == 1:
+pass
+elif len(sys.argv) == 2:
+if os.path.isfile(sys.argv[1]):
+sqlfile = sys.argv[1]
+else:
+url = sys.argv[1]
+elif len(sys.argv) == 3:
url = sys.argv[1]
-elif len(sys.argv) == 3:
-url = sys.argv[1]
-sqlfile = sys.argv[2]
-else:
-usage_and_exit()
-
-url = cleanup_url(url)
-
-if sqlfile != "":
-sqlfile = "--run=" + sqlfile
-
-colorSetting = "true"
-# disable color setting for windows OS
-if os.name == 'nt':
-colorSetting = "false"
-
-# HBase configuration folder path (where hbase-site.xml reside) for
-# HBase/Phoenix client side property override
-hbase_config_path = os.getenv('HBASE_CONF_DIR', phoenix_utils.current_dir)
-
-serialization = get_serialization()
-
-java_home = os.getenv('JAVA_HOME')
-
-# load hbase-env.??? to extract JAVA_HOME, HBASE_PID_DIR, HBASE_LOG_DIR
-hbase_env_path = None
-hbase_env_cmd = None
-if os.name == 'posix':
-hbase_env_path = os.path.join(hbase_config_path, 'hbase-env.sh')
-hbase_env_cmd = ['bash', '-c', 'source %s && env' % hbase_env_path]
-elif os.name == 'nt':
-hbase_env_path = os.path.join(hbase_config_path, 'hbase-env.cmd')
-hbase_env_cmd = ['cmd.exe', '/c', 'call %s & set' % hbase_env_path]
-if not hbase_env_path or not hbase_env_cmd:
-print >> sys.stderr, "hbase-env file unknown on platform %s" % os.name
-sys.exit(-1)
-
-hbase_env = {}
-if os.path.isfile(hbase_env_path):
-p = subprocess.Popen(hbase_env_cmd, stdout = subprocess.PIPE)
-for x in p.stdout:
-(k, _, v) = x.partition('=')
-hbase_env[k.strip()] = v.strip()
-
-if hbase_env.has_key('JAVA_HOME'):
-java_home = hbase_env['JAVA_HOME']
-
-if java_home:
-java = os.path.join(java_home, 'bin', 'java')
-else:
-java = 'java'
-
-java_cmd = java + ' $PHOENIX_OPTS ' + \
-' -cp "' + phoenix_utils.hbase_conf_dir + os.pathsep +
phoenix_utils.phoenix_thin_client_jar + \
-os.pathsep + phoenix_utils.hadoop_conf + os.pathsep +
phoenix_utils.hadoop_classpath + '" -Dlog4j.configuration=file:' + \
-os.path.join(phoenix_utils.current_dir, "log4j.properties") + \
-" org.apache.phoenix.queryserver.client.SqllineWrapper -d
org.apache.phoenix.queryserver.client.Driver " + \
-" -u \"jdbc:phoenix:thin:url=" + url + ";serialization=" +
serialization + "\"" + \
-" -n none -p none --color=" + colorSetting + " --fastConnect=false
--verbose=true " + \
-" --incremental=false --isolation=TRANSACTION_READ_COMMITTED " +
sqlfile
-
-exitcode = subprocess.call(java_cmd, shell=True)
-sys.exit(exitcode)
+sqlfile = sys.argv[2]
+else:
+sys.exit('usage: sqlline-thin.py [host[:port]] [sql_file]')
+
+# disable color setting for windows OS
+color_setting = 'false' if os.name == 'nt' else 'true'
+
+# HBase configuration folder path (where hbase-site.xml reside) for
+# HBase/Phoenix client side property override
+hbase_config_path = os.getenv('HBASE_CONF_DIR',
phoenix_utils.current_dir)
+serialization = get_serialization()
+java_home = os.getenv('JAVA_HOME')
+
+# load hbase-env.??? to extract JAVA_HOME, HBASE_PID_DIR, HBASE_LOG_DIR
+hbase_env_path = None
+hbase_env_cmd = None
+if os.name == 'posix':
+hbase_env_path = os.path.join(hbase_config_path, 'hbase-env.sh')
+hbase_env_cmd = ['bash', '-c', 'source %s && env' % hbase_env_path]
+elif os.name == 'nt':
+hbase_env_path = os.path.join(hbase_config_path, 'hbase-env.cmd')
+hbase
[GitHub] phoenix pull request #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/186#discussion_r73024725
--- Diff: bin/sqlline-thin.py ---
@@ -18,64 +18,58 @@
# limitations under the License.
#
-
+from __future__ import print_function
import os
+import phoenix_utils
import subprocess
import sys
-import phoenix_utils
-import atexit
import urlparse
-global childProc
-childProc = None
-def kill_child():
-if childProc is not None:
-childProc.terminate()
-childProc.kill()
-if os.name != 'nt':
-os.system("reset")
-atexit.register(kill_child)
-
phoenix_utils.setPath()
-url = "localhost:8765"
-sqlfile = ""
-serialization_key = 'phoenix.queryserver.serialization'
-
-def usage_and_exit():
-sys.exit("usage: sqlline-thin.py [host[:port]] [sql_file]")
def cleanup_url(url):
-parsed = urlparse.urlparse(url)
-if parsed.scheme == "":
-url = "http://"; + url
-parsed = urlparse.urlparse(url)
-if ":" not in parsed.netloc:
-url = url + ":8765"
+p = urlparse.urlparse(url)
+scheme = p.scheme if p.scheme else 'http'
+netloc = p.netloc if ':' in p.netloc else p.netloc + '8765'
+url = '{scheme}://{netloc}'.format(scheme, netloc)
--- End diff --
This is using the arguments positionally for substitution even though you
provided named substitutions in the format string, right? Would it make sense
to change the args to kwargs?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/186#discussion_r73025653
--- Diff: bin/sqlline-thin.py ---
@@ -18,64 +18,58 @@
# limitations under the License.
#
-
+from __future__ import print_function
import os
+import phoenix_utils
import subprocess
import sys
-import phoenix_utils
-import atexit
import urlparse
-global childProc
-childProc = None
-def kill_child():
-if childProc is not None:
-childProc.terminate()
-childProc.kill()
-if os.name != 'nt':
-os.system("reset")
-atexit.register(kill_child)
--- End diff --
It looks like you had dropped this logic completely. Is there a reason we
don't need it or was it unintentional?
Ultimately, we'd want to make sure that if this script is
killed/interrupted that it would also terminate the Java program it spawns. I'm
not sure how subprocess would handle this now (and if it was even necessary).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 Left a few comments for you inline. Overall, I like the changes. Would love to see corresponding ones made for sqlline.py initially (we tend to make changes to one in the other as well). I don't think we would need to revisit all of the scripts for the first round (unless you're up to the task!). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #186: cleanup sqlline-thin.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 > quite a bit of this script is to poke the environment .. which is I/O. Unit testing may not yield much. But, i'll take a fresh look. Yeah, I can understand where you're coming from. It would probably require some indirection to do it well. One concrete example I saw again was that we use JAVA_HOME from the environment unless it exists in the `hbase-env.*` script. These are the sorts of things that stand out at me that, long term, we should have tests to prove work as expected. Again, I don't want to push them all on you in this changeset, but I'd be more than happy to work with someone who is a better python dev than myself :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #186: cleanup sqlline*.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 > I have added what I think is cross-platform way of handling zombie? child processes. *nux is definitely the one we care about the most :). I would expect that subprocess would already have the platform-specific stuff abstracted away for us (but maybe not)? > However, I realise there are more than one such call in each program. I'll have atleast one more push to address this. Cool. > Except for a few lines, both the programs are quite similar. There is an opportunity to reuse code. But this is not an immediate concern. Agreed. Definitely an area for continued improvements! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #186: cleanup sqlline*.py using python idioms.
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/186#discussion_r74614217 --- Diff: bin/sqlline.py --- @@ -1,4 +1,5 @@ #!/usr/bin/env python +"""sqlline.py""" --- End diff -- Is this a convention I don't know about? From a licensing perspective, I think it would be best to make sure the ASLv2 header be the first thing after the shebang. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #186: cleanup sqlline*.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 Sorry for the delay @btbytes, I missed your latest push where you addressed sqlline.py too. Thanks! I think this is looking good. I left one comment for you. I will try to test this out locally today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #186: cleanup sqlline*.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 Alright, I'll try to poke at this one today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #186: cleanup sqlline*.py using python idioms.
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/186 ``` % ./bin/sqlline.py Traceback (most recent call last): File "./bin/sqlline.py", line 139, in main() File "./bin/sqlline.py", line 132, in main preexec_fn=os.setsid) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child raise child_exception OSError: [Errno 63] File name too long ``` Looks like there is an upper limit on the command that can be executed. Can you turn the heredoc-style string into an array? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #191: PHOENIX-3189 Perform Kerberos login before Connec...
GitHub user joshelser opened a pull request: https://github.com/apache/phoenix/pull/191 PHOENIX-3189 Perform Kerberos login before ConnectionInfo is constructed Now that ConnectionInfo has the current User/UGI stored inside, we must make sure that any automatic Kerberos login occurs before the ConnectionInfo object is constructed. Otherwise, we will have multiple instances of ConnectionInfo that differ only by the User, which will leak HBase/ZK connections in the connectionQueryServicesMap. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joshelser/phoenix 3189-secure-cnxninfo Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/191.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #191 commit ac4184b9ec52ab261fc4d9f73edc6676eb421d79 Author: Josh Elser Date: 2016-08-17T17:34:59Z PHOENIX-3189 Perform Kerberos login before ConnectionInfo is constructed Now that ConnectionInfo has the current User/UGI stored inside, we must make sure that any automatic Kerberos login occurs before the ConnectionInfo object is constructed. Otherwise, we will have multiple instances of ConnectionInfo that differ only by the User, which will leak HBase/ZK connections in the connectionQueryServicesMap. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/191 > Would an alternative be to not rely on the User.equals method, but instead modify ConnectionInfo.equals to do some other kind of equality check on User? I think I initially had this idea too (wondering why User doesn't implement a "normal" value-based equals/hashCode implementation). Devaraj didn't remember exactly why, but he recalled that there was a reason that UGI (and thus User also) did this. I could try to dig into the Hadoop archives to see if I can understand why this was done. Mostly, I think this approach keeps us in line with what HDFS and HBase does now which has some value. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #191: PHOENIX-3189 Perform Kerberos login before Connec...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/191#discussion_r76484922
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
---
@@ -333,6 +337,55 @@ public ConnectionInfo normalize(ReadOnlyProps props)
throws SQLException {
keytab =
props.get(QueryServices.HBASE_CLIENT_KEYTAB);
}
}
+if (!isConnectionless()) {
+boolean credsProvidedInUrl = null != principal && null !=
keytab;
+boolean credsProvidedInProps =
info.containsKey(QueryServices.HBASE_CLIENT_PRINCIPAL) &&
info.containsKey(QueryServices.HBASE_CLIENT_KEYTAB);
+if (credsProvidedInUrl || credsProvidedInProps) {
+// PHOENIX-3189 Because ConnectionInfo is immutable,
we must make sure all parts of it are correct before
+// construction; this also requires the Kerberos user
credentials object (since they are compared by reference
+// and not by value. If the user provided a principal
and keytab via the JDBC url, we must make sure that the
+// Kerberos login happens *before* we construct the
ConnectionInfo object. Otherwise, the use of ConnectionInfo
+// to determine when ConnectionQueryServices impl's
should be reused will be broken.
+Configuration config =
HBaseFactoryProvider.getConfigurationFactory().getConfiguration();
+// Add QueryServices properties
+for (Entry entry : props) {
+config.set(entry.getKey(), entry.getValue());
+}
+// Add any user-provided properties (via DriverManager)
+if (info != null) {
+for (Object key : info.keySet()) {
+config.set((String) key,
info.getProperty((String) key));
+}
+}
+// Set the principal and keytab if provided from the
URL (overriding those provided in Properties)
+if (null != principal) {
+config.set(QueryServices.HBASE_CLIENT_PRINCIPAL,
principal);
+}
+if (null != keytab) {
+config.set(QueryServices.HBASE_CLIENT_KEYTAB,
keytab);
+}
+try {
+// Check if we need to authenticate with kerberos
so that we cache the correct ConnectionInfo
+UserGroupInformation currentUser =
UserGroupInformation.getCurrentUser();
+if (!currentUser.hasKerberosCredentials() ||
!currentUser.getUserName().equals(principal)) {
--- End diff --
Yes, good catch. Can definitely do this only when necessary.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/191 Just pushed d17a8d8 to address the issue you mentioned, @JamesRTaylor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/203 > That is caused by UserInformationGroup loginUserFromKeytab being called multiple time from different threads if using a multi threaded environment. this fix ensures that there will only be one login per process. `UGI.loginUserFromKeytab` never spawns a renewal thread so as it is. I don't think this change has the effect you intend it to have. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/203 > This fix has been tested and it solves the issue, the same fix has been applied to the storm hdfs and hbase connectors. But I still don't understand what you're trying to fix. https://github.com/apache/hadoop/blob/94225152399e6e89fa7b4cff6d17d33e544329a3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L957-L958 `UserGroupInformation` does *not* spawn any renewal thread for ticket renewal. Can you clarify what doesn't work? Given your description on JIRA, it doesn't make sense to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/203 Ignoring the aforementioned issue, I don't think this change is correctly handling multiple users. It would be re-introducing the bug that was talked about in PHOENIX-3126. If there was a user that was already logged in and then a different URL was provided with different credentials, the old user's credentials would be used instead of the new user's credentials. This would be a security vulnerability. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/203
> Regarding the renewal, I understand from,
http://stackoverflow.com/questions/34616676/should-i-call-ugi-checktgtandreloginfromkeytab-before-every-action-on-hadoop,
that the RPC layer takes care of that.
Well, if you're talking to HDFS directly it would take care of it :). But
we're talking about accessing HBase here. I'm not sure if the same holds true.
I know there is something similar in the HBase RPC level, but I'd have to find
it again in code to double check.
> I am trying to fix the scenario in which multiple threads call
loginUserFromKeytab concurrently and then the renewal process no longer works
as expected.
> If only one login happens the renewal works properly.
Is this the same principal over and over again? Are you essentially
providing the same principal and keytab in the JDBC URL, expecting Phoenix to
do everything for you instead of doing the login in Storm?
> Your concern regarding security is correct.
Ok. I would like to redirect your efforts to PHOENIX-3189 then. We cannot
sacrifice security for multi-threading (as you can already handle the Kerberos
login yourself). Can you take a look at the changes I have staged on #191? If
this is the above case I outlined, we can add some concurrency control to
prevent concurrent logins from happening.
> you can see that this class is not thread safe and not designed to have
different users login in the same JVM as loginUser is defined in this way.
Phoenix itself is not well-designed to support concurrent (different) users
accessing HBase because of how UGI works. If your application (Storm) needs to
provide this functionality, Storm should perform logins itself, cache the UGI
instances, and use {{UGI.doAs(..)}} instead of relying on the static state in
UGI.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/191 > This solution is not thread safe and will not allow to safely create multiple instances of a driver on different threads in the JVM. Yes, that's why I directed you over here, bud. That wasn't an initial goal of these changes. > With that said I am not sure that you can support multiple users and support renewals with the way the UGI works. Right.. you're catching on to what I was pointing out. This is something that you should be managing inside of Storm. We cannot do this effectively inside of Phoenix. We can only put a bandaid on top. > Do we want the Phoenix driver to allow multiple instances instantiated with a different logged in user for each in the same JVM ? The only change I think we can do here is to prevent multiple clients from doing what you're suggesting and hope they don't shoot themselves in the foot. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/191 @dbahir I believe b5be8d8 would address your concerns. WDYT? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #:
Github user joshelser commented on the pull request:
https://github.com/apache/phoenix/commit/b5be8d85bfd05aeb3ea17b1c734eda06d914f516#commitcomment-18831881
> UserGroupInformation.getCurrentUser() will not be thread safe.
```
@InterfaceAudience.Public
@InterfaceStability.Evolving
public synchronized
static UserGroupInformation getCurrentUser() throws IOException {
AccessControlContext context = AccessController.getContext();
Subject subject = Subject.getSubject(context);
if (subject == null || subject.getPrincipals(User.class).isEmpty()) {
return getLoginUser();
} else {
return new UserGroupInformation(subject);
}
}
```
Am I missing something, @dbahir?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request #205: PHOENIX-3223 Add `hadoop classpath` to PQS classp...
GitHub user joshelser opened a pull request: https://github.com/apache/phoenix/pull/205 PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available You can merge this pull request into a Git repository by running: $ git pull https://github.com/joshelser/phoenix PHOENIX-3223-pqs-classpath Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/205.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #205 commit d65593b194f6bb89bc0e69e744d2c0e92971cdb8 Author: Josh Elser Date: 2016-08-30T19:58:50Z PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/191 > Did you get to look at the comment regarding the user login ? Assuming you mean the below.. > With that said I am not sure that you can support multiple users and support renewals with the way the UGI works. > If in the same JVM a driver is instantiated for User A and then another driver is instantiated for User B the last call to loginUserFromKeytab will set the the user information in the UGI. Yes, the best way to approach this is that Phoenix's automatic Kerberos login should _only_ be used when there a single user accessing Phoenix at one time. I was thinking that Phoenix could likely use its own docs page on the interactions with Kerberos. That would be a good place to warn people and instruct them how they need to properly support multiple concurrent users on their own. > Can you allow another to login with a different principal? Would that cause a security issue? If we create one driver(One) with user A and then create another driver(Two) with user B the info in the UGI now is that of user B. So there can be a situation where driver One will be using credentials of user B. My thinking is this (but I should probably test it): says userA uses phoenix, then userB starts using Phoenix. This implies that userA would be logged out and I would hope that subsequent interactions with Phoenix by the "human" userA would fail because its credentials are no longer there (the RPCs themselves would fail). In other words, my thinking is that the RPC authentication would protect us in Phoenix from this being an issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/191 > Thanks for pursuing this tricky issue, @joshelser. I think what you have here is definitely an improvement and should be pulled in for 4.8.1 Thanks as always for your careful eye. > but I do think we should look at doing a value-based equality check for User instead. There's a fair amount of overhead in what you're doing and Phoenix does not do connection pooling but relies on being able to quickly/cheaply get a connection. Would you have some cycles to investigate that a bit first? To my point in the last comment, I believe it's become apparent that people are misusing this functionality. So, I believe there is some education that needs to happen about how clients of Phoenix who need to support multiple Kerberos users concurrently should interact with the system. I think this documentation is point 1. It flat-out shouldn't be used to concurrently support multiple Kerberos users -- developers must write their own code to do this correctly. It's difficult to handle this for them because of the required doAs() call. Can investigate this further too, maybe I'm overlooking something obvious. I do need to make good on my promise to dig through Hadoop/UGI git-log and try to piece together why exactly UGI isn't doing value-based checks already. This is point 2. I'd like to move ahead and push then (rather then let it rot on the vine), but will try to address both of these points this week if that's OK with you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/191 > My thinking is this (but I should probably test it): says userA uses phoenix, then userB starts using Phoenix. This implies that userA would be logged out and I would hope that subsequent interactions with Phoenix by the "human" userA would fail because its credentials are no longer there (the RPCs themselves would fail). In other words, my thinking is that the RPC authentication would protect us in Phoenix from this being an issue. No dice. This is still a hole (although not a new hole as I understand it). Will track this separately. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/307#discussion_r201076875
--- Diff:
phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerPhoenixDBIT.java
---
@@ -0,0 +1,423 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Maps;
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.LocalHBaseCluster;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hdfs.DFSConfigKeys;
+import org.apache.hadoop.http.HttpConfig;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.authentication.util.KerberosName;
+import org.apache.phoenix.query.ConfigurationFactory;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.queryserver.client.ThinClientUtil;
+import org.apache.phoenix.queryserver.server.QueryServer;
+import org.apache.phoenix.util.InstanceResolver;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import java.io.*;
+import java.lang.reflect.Field;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map.Entry;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import java.nio.file.Paths;
+import java.util.Map;
+
+import static org.junit.Assert.*;
+
+@Category(NeedsOwnMiniClusterTest.class)
+public class SecureQueryServerPhoenixDBIT {
+private static final Log LOG =
LogFactory.getLog(SecureQueryServerPhoenixDBIT.class);
+
+private static final File TEMP_DIR = new File(getTempDirForClass());
+private static final File KEYTAB_DIR = new File(TEMP_DIR, "keytabs");
+private static final List USER_KEYTAB_FILES = new ArrayList<>();
+
+private static final String SPNEGO_PRINCIPAL = "HTTP/localhost";
+private static final String PQS_PRINCIPAL = "phoenixqs/localhost";
+private static final String SERVICE_PRINCIPAL =
"securecluster/localhost";
+private static File KEYTAB;
+
+private static MiniKdc KDC;
+private static HBaseTestingUtility UTIL = new HBaseTestingUtility();
+private static LocalHBaseCluster HBASE_CLUSTER;
+private static int NUM_CREATED_USERS;
+
+private static ExecutorService PQS_EXECUTOR;
+private static QueryServer PQS;
+private static int PQS_PORT;
+private static String PQS_URL;
+
+private static String getTempDirForClass() {
+StringBuilder sb = new StringBuilder(32);
+sb.append(System.getProperty("user.dir")).append(File.separator);
+sb.append("target").append(File.separator);
+sb.append(SecureQueryServerPhoenixDBIT.class.getSimpleName());
+return sb.toString();
+}
+
+private static void updateDefaultRealm() throws Exception {
+// (at
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/307#discussion_r201078376 --- Diff: python/requests-kerberos/LICENSE --- @@ -0,0 +1,15 @@ +ISC License --- End diff -- Just calling out that this is allowed: ISC is a Category-A license per https://www.apache.org/legal/resolved.html ---
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/307#discussion_r201077800 --- Diff: python/phoenixdb-module/phoenixdb/__init__.py --- @@ -1,11 +1,10 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at +# Copyright 2015 Lukas Lalinsky --- End diff -- Any reason for the re-add of this? We don't need this after the IP Clearance process, I think. NOTICE file should be sufficient. ---
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/307#discussion_r201076404
--- Diff: phoenix-queryserver/src/it/bin/test_phoenixdb.sh ---
@@ -0,0 +1,59 @@
+#/usr/bin/env bash
+
+set -u
+set -x
+set -e
+
+function cleanup {
+set +e
+set +u
+kdestroy
--- End diff --
If we use a custom directory for the `kinit`, then this just becomes
removing that custom directory.
---
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/307#discussion_r201075953
--- Diff: phoenix-queryserver/src/it/bin/test_phoenixdb.sh ---
@@ -0,0 +1,59 @@
+#/usr/bin/env bash
+
+set -u
+set -x
+set -e
+
+function cleanup {
+set +e
+set +u
+kdestroy
+pushd ${PY_ENV_PATH}/bin
+. deactivate ""
+popd
+rm -rf $PY_ENV_PATH
+}
+
+trap cleanup EXIT
+
+echo "LAUNCHING SCRIPT"
+
+LOCAL_PY=$1
+PRINC=$2
+KEYTAB_LOC=$3
+KRB5_CFG_FILE=$4
+PQS_PORT=$5
+PYTHON_SCRIPT=$6
+
+PY_ENV_PATH=$( mktemp -d )
+
+conda create -y -p $PY_ENV_PATH || virtualenv $PY_ENV_PATH
+
+pushd ${PY_ENV_PATH}/bin
+
+# conda activate does stuff with unbound variables :(
+set +u
+. activate ""
+
+popd
+
+set -u
+echo "INSTALLING COMPONENTS"
+pip install -e file:///${LOCAL_PY}/requests-kerberos
+pip install -e file:///${LOCAL_PY}/phoenixdb-module
+
+export KRB5_CONFIG=$KRB5_CFG_FILE
+cat $KRB5_CONFIG
+export KRB5_TRACE=/dev/stdout
+
+#echo "RUNNING KINIT"
+kinit -kt $KEYTAB_LOC $PRINC
--- End diff --
Can we kinit to a custom location? e.g. the `-c` option. Then, later, we
just set the variable `KRB5CCNAME` in the shell ENV.
This would help prevent us from bashing the user's ticket (if they already
have one).
---
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/307#discussion_r201077202
--- Diff:
phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerPhoenixDBIT.java
---
@@ -0,0 +1,423 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Maps;
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.LocalHBaseCluster;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hdfs.DFSConfigKeys;
+import org.apache.hadoop.http.HttpConfig;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.authentication.util.KerberosName;
+import org.apache.phoenix.query.ConfigurationFactory;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.queryserver.client.ThinClientUtil;
+import org.apache.phoenix.queryserver.server.QueryServer;
+import org.apache.phoenix.util.InstanceResolver;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import java.io.*;
+import java.lang.reflect.Field;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map.Entry;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import java.nio.file.Paths;
+import java.util.Map;
+
+import static org.junit.Assert.*;
+
+@Category(NeedsOwnMiniClusterTest.class)
+public class SecureQueryServerPhoenixDBIT {
+private static final Log LOG =
LogFactory.getLog(SecureQueryServerPhoenixDBIT.class);
+
+private static final File TEMP_DIR = new File(getTempDirForClass());
+private static final File KEYTAB_DIR = new File(TEMP_DIR, "keytabs");
+private static final List USER_KEYTAB_FILES = new ArrayList<>();
+
+private static final String SPNEGO_PRINCIPAL = "HTTP/localhost";
+private static final String PQS_PRINCIPAL = "phoenixqs/localhost";
+private static final String SERVICE_PRINCIPAL =
"securecluster/localhost";
+private static File KEYTAB;
+
+private static MiniKdc KDC;
+private static HBaseTestingUtility UTIL = new HBaseTestingUtility();
+private static LocalHBaseCluster HBASE_CLUSTER;
+private static int NUM_CREATED_USERS;
+
+private static ExecutorService PQS_EXECUTOR;
+private static QueryServer PQS;
+private static int PQS_PORT;
+private static String PQS_URL;
+
+private static String getTempDirForClass() {
+StringBuilder sb = new StringBuilder(32);
+sb.append(System.getProperty("user.dir")).append(File.separator);
+sb.append("target").append(File.separator);
+sb.append(SecureQueryServerPhoenixDBIT.class.getSimpleName());
+return sb.toString();
+}
+
+private static void updateDefaultRealm() throws Exception {
+// (at
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/307#discussion_r201077355
--- Diff:
phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerPhoenixDBIT.java
---
@@ -0,0 +1,423 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Maps;
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.LocalHBaseCluster;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hdfs.DFSConfigKeys;
+import org.apache.hadoop.http.HttpConfig;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.authentication.util.KerberosName;
+import org.apache.phoenix.query.ConfigurationFactory;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.queryserver.client.ThinClientUtil;
+import org.apache.phoenix.queryserver.server.QueryServer;
+import org.apache.phoenix.util.InstanceResolver;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import java.io.*;
+import java.lang.reflect.Field;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map.Entry;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import java.nio.file.Paths;
+import java.util.Map;
+
+import static org.junit.Assert.*;
+
+@Category(NeedsOwnMiniClusterTest.class)
+public class SecureQueryServerPhoenixDBIT {
+private static final Log LOG =
LogFactory.getLog(SecureQueryServerPhoenixDBIT.class);
+
+private static final File TEMP_DIR = new File(getTempDirForClass());
+private static final File KEYTAB_DIR = new File(TEMP_DIR, "keytabs");
+private static final List USER_KEYTAB_FILES = new ArrayList<>();
+
+private static final String SPNEGO_PRINCIPAL = "HTTP/localhost";
+private static final String PQS_PRINCIPAL = "phoenixqs/localhost";
+private static final String SERVICE_PRINCIPAL =
"securecluster/localhost";
+private static File KEYTAB;
+
+private static MiniKdc KDC;
+private static HBaseTestingUtility UTIL = new HBaseTestingUtility();
+private static LocalHBaseCluster HBASE_CLUSTER;
+private static int NUM_CREATED_USERS;
+
+private static ExecutorService PQS_EXECUTOR;
+private static QueryServer PQS;
+private static int PQS_PORT;
+private static String PQS_URL;
+
+private static String getTempDirForClass() {
+StringBuilder sb = new StringBuilder(32);
+sb.append(System.getProperty("user.dir")).append(File.separator);
+sb.append("target").append(File.separator);
+sb.append(SecureQueryServerPhoenixDBIT.class.getSimpleName());
+return sb.toString();
+}
+
+private static void updateDefaultRealm() throws Exception {
+// (at
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/307#discussion_r201418146 --- Diff: phoenix-queryserver/pom.xml --- @@ -47,6 +47,11 @@ org.apache.maven.plugins maven-failsafe-plugin + + +**/SecureQueryServerPhoenixDBIT.java --- End diff -- You not intending for this test to be executed during the normal build process? ---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/307 > How do we make sure requests-kerberos is built in a way that phoenixdb will grab it? (e.g. 0.13.0.dev0-phoenixdb) Ah, I see this is updated in python/requests-kerberos/requests_kerberos/__init__.py. We should update python/phoenixdb/requirements.txt to make sure we pull that 0.13.0.dev0-phoenixdb, right? ---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/307 Tried to run the test, but it failed with: ``` 2018-07-10 13:43:55,599 ERROR [main] end2end.SecureQueryServerPhoenixDBIT(359): + . deactivate '' 2018-07-10 13:43:55,599 ERROR [main] end2end.SecureQueryServerPhoenixDBIT(359): /Users/jelser/projects/phoenix.git/phoenix-queryserver/./src/it/bin/test_phoenixdb.sh: line 12: deactivate: No such file or directory ``` Might have been me hacking on things... digging more. ---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r201465583
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/ClientHashAggregatingResultIterator.java
---
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.iterate;
+
+import static org.apache.phoenix.query.QueryConstants.AGG_TIMESTAMP;
+import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN;
+import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY;
+
+import java.io.IOException;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
+import org.apache.phoenix.expression.Expression;
+import org.apache.phoenix.expression.aggregator.Aggregator;
+import org.apache.phoenix.expression.aggregator.Aggregators;
+import org.apache.phoenix.schema.tuple.MultiKeyValueTuple;
+import org.apache.phoenix.schema.tuple.Tuple;
+import org.apache.phoenix.util.KeyValueUtil;
+import org.apache.phoenix.util.TupleUtil;
+
+/**
+ *
+ * This class implements client-side hash aggregation in memory.
+ * Issue https://issues.apache.org/jira/browse/PHOENIX-4751.
+ *
+ */
+public class ClientHashAggregatingResultIterator
+implements AggregatingResultIterator {
+
+private static final int HASH_AGG_INIT_SIZE = 64*1024;
+private static final byte[] UNITIALIZED_KEY_BUFFER = new byte[0];
+private final ResultIterator resultIterator;
+private final Aggregators aggregators;
+private final List groupByExpressions;
+private HashMap hash;
+private List keyList;
+private Iterator keyIterator;
+
+public ClientHashAggregatingResultIterator(ResultIterator
resultIterator, Aggregators aggregators, List groupByExpressions) {
+if (resultIterator == null) throw new NullPointerException();
--- End diff --
Could consolidate this down into `Objects.requireNonNull(resultIterator)`
from java.util.Objects.
---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r201468535
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/ClientHashAggregatingResultIterator.java
---
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.iterate;
+
+import static org.apache.phoenix.query.QueryConstants.AGG_TIMESTAMP;
+import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN;
+import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY;
+
+import java.io.IOException;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
+import org.apache.phoenix.expression.Expression;
+import org.apache.phoenix.expression.aggregator.Aggregator;
+import org.apache.phoenix.expression.aggregator.Aggregators;
+import org.apache.phoenix.schema.tuple.MultiKeyValueTuple;
+import org.apache.phoenix.schema.tuple.Tuple;
+import org.apache.phoenix.util.KeyValueUtil;
+import org.apache.phoenix.util.TupleUtil;
+
+/**
+ *
+ * This class implements client-side hash aggregation in memory.
+ * Issue https://issues.apache.org/jira/browse/PHOENIX-4751.
+ *
+ */
+public class ClientHashAggregatingResultIterator
+implements AggregatingResultIterator {
+
+private static final int HASH_AGG_INIT_SIZE = 64*1024;
+private static final byte[] UNITIALIZED_KEY_BUFFER = new byte[0];
+private final ResultIterator resultIterator;
+private final Aggregators aggregators;
+private final List groupByExpressions;
+private HashMap hash;
+private List keyList;
+private Iterator keyIterator;
+
+public ClientHashAggregatingResultIterator(ResultIterator
resultIterator, Aggregators aggregators, List groupByExpressions) {
+if (resultIterator == null) throw new NullPointerException();
+if (aggregators == null) throw new NullPointerException();
+if (groupByExpressions == null) throw new NullPointerException();
+this.resultIterator = resultIterator;
+this.aggregators = aggregators;
+this.groupByExpressions = groupByExpressions;
+}
+
+@Override
+public Tuple next() throws SQLException {
+if (keyIterator == null) {
+populateHash();
--- End diff --
A comment here would be nice to note the side-effect that `hash` and
`keyList` are guaranteed to be non-null (and thus, the lack of the defensive
null-checks below is OK).
---
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/307#discussion_r202106445
--- Diff: phoenix-queryserver/src/it/bin/test_phoenixdb.sh ---
@@ -0,0 +1,59 @@
+#/usr/bin/env bash
+
+set -u
+set -x
+set -e
+
+function cleanup {
+set +e
+set +u
+kdestroy
--- End diff --
Ok, cool. I didn't think kdestroy was doing more than just cleaning up
those token :)
---
[GitHub] phoenix pull request #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/307#discussion_r202108057
--- Diff: phoenix-queryserver/src/it/bin/test_phoenixdb.sh ---
@@ -0,0 +1,59 @@
+#/usr/bin/env bash
+
+set -u
+set -x
+set -e
+
+function cleanup {
+set +e
+set +u
+kdestroy
+pushd ${PY_ENV_PATH}/bin
+. deactivate ""
+popd
+rm -rf $PY_ENV_PATH
+}
+
+trap cleanup EXIT
+
+echo "LAUNCHING SCRIPT"
+
+LOCAL_PY=$1
+PRINC=$2
+KEYTAB_LOC=$3
+KRB5_CFG_FILE=$4
+PQS_PORT=$5
+PYTHON_SCRIPT=$6
+
+PY_ENV_PATH=$( mktemp -d )
+
+conda create -y -p $PY_ENV_PATH || virtualenv $PY_ENV_PATH
+
+pushd ${PY_ENV_PATH}/bin
+
+# conda activate does stuff with unbound variables :(
+set +u
+. activate ""
+
+popd
+
+set -u
+echo "INSTALLING COMPONENTS"
+pip install -e file:///${LOCAL_PY}/requests-kerberos
+pip install -e file:///${LOCAL_PY}/phoenixdb-module
+
+export KRB5_CONFIG=$KRB5_CFG_FILE
+cat $KRB5_CONFIG
+export KRB5_TRACE=/dev/stdout
+
+#echo "RUNNING KINIT"
+kinit -kt $KEYTAB_LOC $PRINC
--- End diff --
> I just tried on the command line and MAC OS (Heimdal) kinit does not
require a directory
Yeah, convention is to use `${tmpdir}/krb5cc_$(current-user uid)`.
> pass the when executing python or just continue running in the same
shell, which is why I stopped attempts to make ny further reductions to the
shell script
Oh right, I forgot they would bash the environment. Let's just let this be
for now. Will be easier to come back to it later.
---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/307 > If you are still unable to get .deactivate to work, remove it from the shell script for now and we can revisit it. Again I am being overly pedantic here, but a the shell exits, we really do not need to clean up any environment. Ok. I'm looking at this again today. Thanks for the feedback so far. The test/code looks good to me. I think we just need to update documentation to explain what we're doing (maybe a README.md in `python/`?) ---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/307 Ugh, I'm getting frustrated: I have MIT kerberos on my Mac, so I unblocked myself first by just forcing the minikdc config file to be made instead of the if-branch you added, Lev. The next thing, I get a failure trying to launch python from the virtualenv: ``` /Users/jelser/projects/phoenix.git/phoenix-queryserver/./src/it/bin/test_phoenixdb.sh: line 59: 66933 Illegal instruction: 4 python $PYTHON_SCRIPT $PQS_PORT ``` This was reproducible doing it by hand, so I thought maybe it was related to me using python-2.7.14 (old). So, I switched over to Python-3.6.4, reinstalled everything, and I got this. ``` Traceback (most recent call last): File "/private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskhgp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/requests_kerberos/kerberos_.py", line 2, in import kerberos ImportError: dlopen(/private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskhgp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/kerberos.cpython-36m-darwin.so, 2): Symbol not found: _mempcpy Referenced from: /private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskhgp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/kerberos.cpython-36m-darwin.so Expected in: flat namespace in /private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskhgp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/kerberos.cpython-36m-darwin.so ``` I ran into another GH issue saying that pykerberos==1.1.14 fixed it for them, but I'm not seeing a difference locally. How do you feel about requiring Docker, @pu239ppy? ;) ---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/307 > Not sure how you got MIT on Mac OS X, is it in some ports package Yeah, homebrew has a `krb5` package which I use for running stuff locally (e.g. I put it on the `PATH` before the Heimdal, osx-provided variant) > I suppose integrating docker into the mix would make things interesting. I'll try it over the weekend if I get the time. I'm playing with different versions of python now, but am just worried about the feasibility of this actually working on the general person's machine given how much I'm struggling :\ ---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/307 Ah, I think my issue might have been cruft sitting in `python/requests-kerberos/`, the `build` dir and the `.egg-info` dir. Getting a straightforward HTTP/401 error now. That I know how to deal with :) ---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/307
Ok, where I'm at now:
* Python 2.7.15 (installed via pyenv)
* Using virtualenv to circumvent the .sh script
* Modified the junit test to just leave it running
* Modified the junit test to just use the minikdc's kdc.conf
* Pulled back the pykerberos dependency to 1.1.14 to get past an "illegal
instruction error" that I get with pykerberos-1.2.1 (or whatever pip found)
This gets the phoenixdb client to actually submit the initial POST and get
the `WWW-Authenticate: Negotiate` header back. However, my client seems to be
unable to generate its challenge data from our mini kdc:
```
DEBUG:phoenixdb.avatica.client:POST http://localhost:60358/
'\n@org.apache.calcite.avatica.proto.Requests$CloseConnectionRequest\x12&\n$f71fb5c5-a814-4766-9691-8aeddfc0eea4'
{'content-type': 'application/x-google-protobuf'}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
localhost:60358
send: 'POST / HTTP/1.1\r\nHost: localhost:60358\r\nConnection:
keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent:
python-requests/2.19.1\r\ncontent-type:
application/x-google-protobuf\r\nContent-Length:
106\r\n\r\n\n@org.apache.calcite.avatica.proto.Requests$CloseConnectionRequest\x12&\n$f71fb5c5-a814-4766-9691-8aeddfc0eea4'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Fri, 13 Jul 2018 17:06:02 GMT
header: WWW-Authenticate: Negotiate
header: Cache-Control: must-revalidate,no-cache,no-store
header: Content-Type: text/html; charset=ISO-8859-1
header: Content-Length: 281
header: Server: Jetty(9.2.19.v20160908)
DEBUG:urllib3.connectionpool:http://localhost:60358 "POST / HTTP/1.1" 401
281
DEBUG:requests_kerberos.kerberos_:handle_401(): Handling: 401
ERROR:requests_kerberos.kerberos_:generate_request_header():
authGSSClientStep() failed:
Traceback (most recent call last):
File
"/Users/jelser/projects/phoenix.git/python/requests-kerberos/requests_kerberos/kerberos_.py",
line 235, in generate_request_header
negotiate_resp_value)
GSSError: (('Unspecified GSS failure. Minor code may provide more
information', 851968), ('Message stream modified', 11))
ERROR:requests_kerberos.kerberos_:(('Unspecified GSS failure. Minor code
may provide more information', 851968), ('Message stream modified', 11))
Traceback (most recent call last):
File
"/Users/jelser/projects/phoenix.git/python/requests-kerberos/requests_kerberos/kerberos_.py",
line 235, in generate_request_header
negotiate_resp_value)
GSSError: (('Unspecified GSS failure. Minor code may provide more
information', 851968), ('Message stream modified', 11))
```
I can't seem to unwrap what's wrong with the request to the KDC which is
preventing that from happening. Need to find more debug...
---
[GitHub] phoenix issue #307: PHOENIX-4688 Kerberize python phoenixdb
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/307
Turning back on `KRB5_TRACE`...
```
DEBUG:phoenixdb.avatica.client:POST http://localhost:60358/
'\n?org.apache.calcite.avatica.proto.Requests$OpenConnectionRequest\x12&\n$386e3317-e23e-4a0e-9fc6-2efaa546ffc4'
{'content-type': 'application/x-google-protobuf'}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
localhost:60358
send: 'POST / HTTP/1.1\r\nHost: localhost:60358\r\nConnection:
keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent:
python-requests/2.19.1\r\ncontent-type:
application/x-google-protobuf\r\nContent-Length:
105\r\n\r\n\n?org.apache.calcite.avatica.proto.Requests$OpenConnectionRequest\x12&\n$386e3317-e23e-4a0e-9fc6-2efaa546ffc4'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Fri, 13 Jul 2018 17:23:46 GMT
header: WWW-Authenticate: Negotiate
header: Cache-Control: must-revalidate,no-cache,no-store
header: Content-Type: text/html; charset=ISO-8859-1
header: Content-Length: 281
header: Server: Jetty(9.2.19.v20160908)
DEBUG:urllib3.connectionpool:http://localhost:60358 "POST / HTTP/1.1" 401
281
DEBUG:requests_kerberos.kerberos_:handle_401(): Handling: 401
[28575] 1531502626.856661: ccselect module realm chose cache
FILE:/tmp/krb5cc_502 with client principal us...@example.com for server
principal HTTP/localh...@example.com
[28575] 1531502626.856662: Getting credentials us...@example.com ->
HTTP/localhost@ using ccache FILE:/tmp/krb5cc_502
[28575] 1531502626.856663: Retrieving us...@example.com -> HTTP/localhost@
from FILE:/tmp/krb5cc_502 with result: -1765328243/Matching credential not
found (filename: /tmp/krb5cc_502)
[28575] 1531502626.856664: Retrying us...@example.com ->
HTTP/localh...@example.com with result: -1765328243/Matching credential not
found (filename: /tmp/krb5cc_502)
[28575] 1531502626.856665: Server has referral realm; starting with
HTTP/localh...@example.com
[28575] 1531502626.85: Retrieving us...@example.com ->
krbtgt/example@example.com from FILE:/tmp/krb5cc_502 with result: 0/Success
[28575] 1531502626.856667: Starting with TGT for client realm:
us...@example.com -> krbtgt/example@example.com
[28575] 1531502626.856668: Requesting tickets for
HTTP/localh...@example.com, referrals on
[28575] 1531502626.856669: Generated subkey for TGS request: aes128-cts/86C4
[28575] 1531502626.856670: etypes requested in TGS request: aes256-cts,
aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts,
camellia256-cts
[28575] 1531502626.856672: Encoding request body and padata into FAST
request
[28575] 1531502626.856673: Sending request (807 bytes) to EXAMPLE.COM
[28575] 1531502626.856674: Resolving hostname localhost
[28575] 1531502626.856675: Initiating TCP connection to stream ::1:60299
[28575] 1531502626.856676: Terminating TCP connection to stream ::1:60299
[28575] 1531502626.856677: Initiating TCP connection to stream
127.0.0.1:60299
[28575] 1531502626.856678: Sending TCP request to stream 127.0.0.1:60299
[28575] 1531502626.856679: Received answer (119 bytes) from stream
127.0.0.1:60299
[28575] 1531502626.856680: Terminating TCP connection to stream
127.0.0.1:60299
[28575] 1531502626.856681: Sending DNS URI query for _kerberos.EXAMPLE.COM.
[28575] 1531502626.856682: No URI records found
[28575] 1531502626.856683: Sending DNS SRV query for
_kerberos-master._udp.EXAMPLE.COM.
[28575] 1531502626.856684: Sending DNS SRV query for
_kerberos-master._tcp.EXAMPLE.COM.
[28575] 1531502626.856685: No SRV records found
[28575] 1531502626.856686: Response was not from master KDC
[28575] 1531502626.856687: TGS request result: -1765328343/Message stream
modified
[28575] 1531502626.856688: Requesting tickets for
HTTP/localh...@example.com, referrals off
[28575] 1531502626.856689: Generated subkey for TGS request: aes128-cts/F96F
[28575] 1531502626.856690: etypes requested in TGS request: aes256-cts,
aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts,
camellia256-cts
[28575] 1531502626.856692: Encoding request body and padata into FAST
request
[28575] 1531502626.856693: Sending request (807 bytes) to EXAMPLE.COM
[28575] 1531502626.856694: Resolving hostname localhost
[28575] 1531502626.856695: Initiating TCP connection to stream ::1:60299
[28575] 1531502626.856696: Terminating TCP connection to stream ::1:60299
[28575] 1531502626.856697: Initiating TCP connection to stream
127.0.0.1:60299
[28575] 1531502626.856698: Sending TCP request to stream 127.0.0.1:60299
[28575] 1531502626.856699: Received answer (119 bytes) from stream
127.0.0.1:60299
[28575] 1531502626.856700: Terminating TCP c
