[jira] [Commented] (PHOENIX-6441) Remove TSOMockModule reference from OmidTransactionProvider
[
https://issues.apache.org/jira/browse/PHOENIX-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485042#comment-17485042
]
ASF GitHub Bot commented on PHOENIX-6441:
-
stoty commented on a change in pull request #1374:
URL: https://github.com/apache/phoenix/pull/1374#discussion_r796268062
##
File path:
phoenix-core/src/main/java/org/apache/phoenix/transaction/OmidTransactionProvider.java
##
@@ -120,84 +101,20 @@ public void close() throws IOException {}
return commitTableClient;
}
-@Override
-public PhoenixTransactionService getTransactionService(Configuration
config, ConnectionInfo connectionInfo, int port) throws SQLException{
-TSOServerConfig tsoConfig = new TSOServerConfig();
-TSOServer tso;
-
-tsoConfig.setPort(port);
-tsoConfig.setConflictMapSize(config.getInt(OMID_TSO_CONFLICT_MAP_SIZE,
DEFAULT_OMID_TSO_CONFLICT_MAP_SIZE));
-tsoConfig.setTimestampType(config.get(OMID_TSO_TIMESTAMP_TYPE,
DEFAULT_OMID_TSO_TIMESTAMP_TYPE));
-tsoConfig.setWaitStrategy(WAIT_STRATEGY.LOW_CPU.toString());
-
-Injector injector = Guice.createInjector(new TSOMockModule(tsoConfig));
-tso = injector.getInstance(TSOServer.class);
-tso.startAsync();
-tso.awaitRunning();
-
-OmidClientConfiguration clientConfig = new OmidClientConfiguration();
-clientConfig.setConnectionString("localhost:" + port);
-
clientConfig.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-
-InMemoryCommitTable commitTable = (InMemoryCommitTable)
injector.getInstance(CommitTable.class);
-
-try {
-// Create the associated Handler
-TSOClient client = TSOClient.newInstance(clientConfig);
-
-HBaseOmidClientConfiguration clientConf = new
HBaseOmidClientConfiguration();
-clientConf.setConnectionString("localhost:" + port);
-
clientConf.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-clientConf.setHBaseConfiguration(config);
-commitTableClient = commitTable.getClient();
-commitTableWriter = commitTable.getWriter();
-transactionManager = HBaseTransactionManager.builder(clientConf)
-.commitTableClient(commitTableClient)
-.commitTableWriter(commitTableWriter)
-.tsoClient(client).build();
-} catch (IOException | InterruptedException e) {
-throw new SQLExceptionInfo.Builder(
-SQLExceptionCode.TRANSACTION_FAILED)
-.setMessage(e.getMessage()).setRootCause(e).build()
-.buildException();
-}
-
-return new OmidTransactionService(tso, transactionManager);
-}
-
-static class OmidTransactionService implements PhoenixTransactionService {
-private final HBaseTransactionManager transactionManager;
-private TSOServer tso;
-
-public OmidTransactionService(TSOServer tso, HBaseTransactionManager
transactionManager) {
-this.tso = tso;
-this.transactionManager = transactionManager;
-}
-
-public void start() {
-
-}
-
-@Override
-public void close() throws IOException {
-if (transactionManager != null) {
-transactionManager.close();
-}
-if (tso != null) {
-tso.stopAsync();
-tso.awaitTerminated();
-}
-}
+// For testing only
+public void injectTestService(HBaseTransactionManager transactionManager,
CommitTable.Client commitTableClient) {
+this.transactionManager = transactionManager;
+this.commitTableClient = commitTableClient;
}
@Override
-public Class getCoprocessor() {
-return OmidTransactionalProcessor.class;
Review comment:
Originally I did this to handle missing classes better, but it's not
really needed.
However, returning strings here simplifies the resto fo the code, as all
HBase coprocessor admin methods expect and return class names.
I'm going to replace these with class.getName()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Remove TSOMockModule reference from OmidTransactionProvider
> ---
>
> Key: PHOENIX-6441
> URL: https://issues.apache.org/jira/browse/PHOENIX-6441
> Project: Phoenix
> Issue Type: Bug
> Components: core, omid
>Affects Versions: 5.2.0
>
[GitHub] [phoenix] stoty commented on a change in pull request #1374: PHOENIX-6441 Remove TSOMockModule reference from OmidTransactionProvider
stoty commented on a change in pull request #1374:
URL: https://github.com/apache/phoenix/pull/1374#discussion_r796268062
##
File path:
phoenix-core/src/main/java/org/apache/phoenix/transaction/OmidTransactionProvider.java
##
@@ -120,84 +101,20 @@ public void close() throws IOException {}
return commitTableClient;
}
-@Override
-public PhoenixTransactionService getTransactionService(Configuration
config, ConnectionInfo connectionInfo, int port) throws SQLException{
-TSOServerConfig tsoConfig = new TSOServerConfig();
-TSOServer tso;
-
-tsoConfig.setPort(port);
-tsoConfig.setConflictMapSize(config.getInt(OMID_TSO_CONFLICT_MAP_SIZE,
DEFAULT_OMID_TSO_CONFLICT_MAP_SIZE));
-tsoConfig.setTimestampType(config.get(OMID_TSO_TIMESTAMP_TYPE,
DEFAULT_OMID_TSO_TIMESTAMP_TYPE));
-tsoConfig.setWaitStrategy(WAIT_STRATEGY.LOW_CPU.toString());
-
-Injector injector = Guice.createInjector(new TSOMockModule(tsoConfig));
-tso = injector.getInstance(TSOServer.class);
-tso.startAsync();
-tso.awaitRunning();
-
-OmidClientConfiguration clientConfig = new OmidClientConfiguration();
-clientConfig.setConnectionString("localhost:" + port);
-
clientConfig.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-
-InMemoryCommitTable commitTable = (InMemoryCommitTable)
injector.getInstance(CommitTable.class);
-
-try {
-// Create the associated Handler
-TSOClient client = TSOClient.newInstance(clientConfig);
-
-HBaseOmidClientConfiguration clientConf = new
HBaseOmidClientConfiguration();
-clientConf.setConnectionString("localhost:" + port);
-
clientConf.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-clientConf.setHBaseConfiguration(config);
-commitTableClient = commitTable.getClient();
-commitTableWriter = commitTable.getWriter();
-transactionManager = HBaseTransactionManager.builder(clientConf)
-.commitTableClient(commitTableClient)
-.commitTableWriter(commitTableWriter)
-.tsoClient(client).build();
-} catch (IOException | InterruptedException e) {
-throw new SQLExceptionInfo.Builder(
-SQLExceptionCode.TRANSACTION_FAILED)
-.setMessage(e.getMessage()).setRootCause(e).build()
-.buildException();
-}
-
-return new OmidTransactionService(tso, transactionManager);
-}
-
-static class OmidTransactionService implements PhoenixTransactionService {
-private final HBaseTransactionManager transactionManager;
-private TSOServer tso;
-
-public OmidTransactionService(TSOServer tso, HBaseTransactionManager
transactionManager) {
-this.tso = tso;
-this.transactionManager = transactionManager;
-}
-
-public void start() {
-
-}
-
-@Override
-public void close() throws IOException {
-if (transactionManager != null) {
-transactionManager.close();
-}
-if (tso != null) {
-tso.stopAsync();
-tso.awaitTerminated();
-}
-}
+// For testing only
+public void injectTestService(HBaseTransactionManager transactionManager,
CommitTable.Client commitTableClient) {
+this.transactionManager = transactionManager;
+this.commitTableClient = commitTableClient;
}
@Override
-public Class getCoprocessor() {
-return OmidTransactionalProcessor.class;
Review comment:
Originally I did this to handle missing classes better, but it's not
really needed.
However, returning strings here simplifies the resto fo the code, as all
HBase coprocessor admin methods expect and return class names.
I'm going to replace these with class.getName()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (PHOENIX-6441) Remove TSOMockModule reference from OmidTransactionProvider
[
https://issues.apache.org/jira/browse/PHOENIX-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484992#comment-17484992
]
ASF GitHub Bot commented on PHOENIX-6441:
-
chrajeshbabu commented on a change in pull request #1374:
URL: https://github.com/apache/phoenix/pull/1374#discussion_r796136425
##
File path:
phoenix-core/src/main/java/org/apache/phoenix/transaction/OmidTransactionProvider.java
##
@@ -120,84 +101,20 @@ public void close() throws IOException {}
return commitTableClient;
}
-@Override
-public PhoenixTransactionService getTransactionService(Configuration
config, ConnectionInfo connectionInfo, int port) throws SQLException{
-TSOServerConfig tsoConfig = new TSOServerConfig();
-TSOServer tso;
-
-tsoConfig.setPort(port);
-tsoConfig.setConflictMapSize(config.getInt(OMID_TSO_CONFLICT_MAP_SIZE,
DEFAULT_OMID_TSO_CONFLICT_MAP_SIZE));
-tsoConfig.setTimestampType(config.get(OMID_TSO_TIMESTAMP_TYPE,
DEFAULT_OMID_TSO_TIMESTAMP_TYPE));
-tsoConfig.setWaitStrategy(WAIT_STRATEGY.LOW_CPU.toString());
-
-Injector injector = Guice.createInjector(new TSOMockModule(tsoConfig));
-tso = injector.getInstance(TSOServer.class);
-tso.startAsync();
-tso.awaitRunning();
-
-OmidClientConfiguration clientConfig = new OmidClientConfiguration();
-clientConfig.setConnectionString("localhost:" + port);
-
clientConfig.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-
-InMemoryCommitTable commitTable = (InMemoryCommitTable)
injector.getInstance(CommitTable.class);
-
-try {
-// Create the associated Handler
-TSOClient client = TSOClient.newInstance(clientConfig);
-
-HBaseOmidClientConfiguration clientConf = new
HBaseOmidClientConfiguration();
-clientConf.setConnectionString("localhost:" + port);
-
clientConf.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-clientConf.setHBaseConfiguration(config);
-commitTableClient = commitTable.getClient();
-commitTableWriter = commitTable.getWriter();
-transactionManager = HBaseTransactionManager.builder(clientConf)
-.commitTableClient(commitTableClient)
-.commitTableWriter(commitTableWriter)
-.tsoClient(client).build();
-} catch (IOException | InterruptedException e) {
-throw new SQLExceptionInfo.Builder(
-SQLExceptionCode.TRANSACTION_FAILED)
-.setMessage(e.getMessage()).setRootCause(e).build()
-.buildException();
-}
-
-return new OmidTransactionService(tso, transactionManager);
-}
-
-static class OmidTransactionService implements PhoenixTransactionService {
-private final HBaseTransactionManager transactionManager;
-private TSOServer tso;
-
-public OmidTransactionService(TSOServer tso, HBaseTransactionManager
transactionManager) {
-this.tso = tso;
-this.transactionManager = transactionManager;
-}
-
-public void start() {
-
-}
-
-@Override
-public void close() throws IOException {
-if (transactionManager != null) {
-transactionManager.close();
-}
-if (tso != null) {
-tso.stopAsync();
-tso.awaitTerminated();
-}
-}
+// For testing only
+public void injectTestService(HBaseTransactionManager transactionManager,
CommitTable.Client commitTableClient) {
+this.transactionManager = transactionManager;
+this.commitTableClient = commitTableClient;
}
@Override
-public Class getCoprocessor() {
-return OmidTransactionalProcessor.class;
Review comment:
Any reason to use static string to specify the coprocessor class name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Remove TSOMockModule reference from OmidTransactionProvider
> ---
>
> Key: PHOENIX-6441
> URL: https://issues.apache.org/jira/browse/PHOENIX-6441
> Project: Phoenix
> Issue Type: Bug
> Components: core, omid
>Affects Versions: 5.2.0
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
>
> OmidTransactionProvider references TSOMockModule which is a test dependency.
> It seems to be used only
[GitHub] [phoenix] chrajeshbabu commented on a change in pull request #1374: PHOENIX-6441 Remove TSOMockModule reference from OmidTransactionProvider
chrajeshbabu commented on a change in pull request #1374:
URL: https://github.com/apache/phoenix/pull/1374#discussion_r796136425
##
File path:
phoenix-core/src/main/java/org/apache/phoenix/transaction/OmidTransactionProvider.java
##
@@ -120,84 +101,20 @@ public void close() throws IOException {}
return commitTableClient;
}
-@Override
-public PhoenixTransactionService getTransactionService(Configuration
config, ConnectionInfo connectionInfo, int port) throws SQLException{
-TSOServerConfig tsoConfig = new TSOServerConfig();
-TSOServer tso;
-
-tsoConfig.setPort(port);
-tsoConfig.setConflictMapSize(config.getInt(OMID_TSO_CONFLICT_MAP_SIZE,
DEFAULT_OMID_TSO_CONFLICT_MAP_SIZE));
-tsoConfig.setTimestampType(config.get(OMID_TSO_TIMESTAMP_TYPE,
DEFAULT_OMID_TSO_TIMESTAMP_TYPE));
-tsoConfig.setWaitStrategy(WAIT_STRATEGY.LOW_CPU.toString());
-
-Injector injector = Guice.createInjector(new TSOMockModule(tsoConfig));
-tso = injector.getInstance(TSOServer.class);
-tso.startAsync();
-tso.awaitRunning();
-
-OmidClientConfiguration clientConfig = new OmidClientConfiguration();
-clientConfig.setConnectionString("localhost:" + port);
-
clientConfig.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-
-InMemoryCommitTable commitTable = (InMemoryCommitTable)
injector.getInstance(CommitTable.class);
-
-try {
-// Create the associated Handler
-TSOClient client = TSOClient.newInstance(clientConfig);
-
-HBaseOmidClientConfiguration clientConf = new
HBaseOmidClientConfiguration();
-clientConf.setConnectionString("localhost:" + port);
-
clientConf.setConflictAnalysisLevel(OmidClientConfiguration.ConflictDetectionLevel.ROW);
-clientConf.setHBaseConfiguration(config);
-commitTableClient = commitTable.getClient();
-commitTableWriter = commitTable.getWriter();
-transactionManager = HBaseTransactionManager.builder(clientConf)
-.commitTableClient(commitTableClient)
-.commitTableWriter(commitTableWriter)
-.tsoClient(client).build();
-} catch (IOException | InterruptedException e) {
-throw new SQLExceptionInfo.Builder(
-SQLExceptionCode.TRANSACTION_FAILED)
-.setMessage(e.getMessage()).setRootCause(e).build()
-.buildException();
-}
-
-return new OmidTransactionService(tso, transactionManager);
-}
-
-static class OmidTransactionService implements PhoenixTransactionService {
-private final HBaseTransactionManager transactionManager;
-private TSOServer tso;
-
-public OmidTransactionService(TSOServer tso, HBaseTransactionManager
transactionManager) {
-this.tso = tso;
-this.transactionManager = transactionManager;
-}
-
-public void start() {
-
-}
-
-@Override
-public void close() throws IOException {
-if (transactionManager != null) {
-transactionManager.close();
-}
-if (tso != null) {
-tso.stopAsync();
-tso.awaitTerminated();
-}
-}
+// For testing only
+public void injectTestService(HBaseTransactionManager transactionManager,
CommitTable.Client commitTableClient) {
+this.transactionManager = transactionManager;
+this.commitTableClient = commitTableClient;
}
@Override
-public Class getCoprocessor() {
-return OmidTransactionalProcessor.class;
Review comment:
Any reason to use static string to specify the coprocessor class name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (PHOENIX-6622) TransformMonitor should orchestrate transform and do retries
[
https://issues.apache.org/jira/browse/PHOENIX-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484963#comment-17484963
]
ASF GitHub Bot commented on PHOENIX-6622:
-
gokceni commented on a change in pull request #1373:
URL: https://github.com/apache/phoenix/pull/1373#discussion_r796101194
##
File path:
phoenix-core/src/it/java/org/apache/phoenix/end2end/transform/TransformMonitorIT.java
##
@@ -0,0 +1,670 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end.transform;
+
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.coprocessor.TaskRegionObserver;
+import org.apache.phoenix.coprocessor.tasks.TransformMonitorTask;
+import org.apache.phoenix.end2end.ParallelStatsDisabledIT;
+import org.apache.phoenix.end2end.index.SingleCellIndexIT;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.ConnectionQueryServices;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.query.QueryServicesOptions;
+import org.apache.phoenix.schema.PTable;
+import org.apache.phoenix.schema.task.SystemTaskParams;
+import org.apache.phoenix.schema.task.Task;
+import org.apache.phoenix.schema.transform.SystemTransformRecord;
+import org.apache.phoenix.schema.transform.Transform;
+import org.apache.phoenix.util.EnvironmentEdgeManager;
+import org.apache.phoenix.util.MetaDataUtil;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.SchemaUtil;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.List;
+import java.util.Properties;
+
+import static org.apache.phoenix.end2end.IndexRebuildTaskIT.waitForTaskState;
+import static
org.apache.phoenix.exception.SQLExceptionCode.CANNOT_CREATE_TENANT_SPECIFIC_TABLE;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.apache.phoenix.util.TestUtil.getRawRowCount;
+import static org.apache.phoenix.util.TestUtil.getRowCount;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+public class TransformMonitorIT extends ParallelStatsDisabledIT {
+private static RegionCoprocessorEnvironment TaskRegionEnvironment;
+
+private Properties testProps = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+
+public TransformMonitorIT() throws IOException, InterruptedException {
+testProps.put(QueryServices.DEFAULT_IMMUTABLE_STORAGE_SCHEME_ATTRIB,
"ONE_CELL_PER_COLUMN");
+testProps.put(QueryServices.DEFAULT_COLUMN_ENCODED_BYTES_ATRRIB, "0");
+testProps.put(QueryServices.PHOENIX_ACLS_ENABLED, "true");
+
+TaskRegionEnvironment = (RegionCoprocessorEnvironment) getUtility()
+.getRSForFirstRegionInTable(
+PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+
.getOnlineRegions(PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+.get(0).getCoprocessorHost()
+
.findCoprocessorEnvironment(TaskRegionObserver.class.getName());
+}
+
+@Before
+public void setupTest() throws Exception {
+try (Connection conn = DriverManager.getConnection(getUrl(),
testProps)) {
+conn.setAutoCommit(true);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TRANSFORM_NAME);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TASK_NAME);
+}
+}
+
+private void
[GitHub] [phoenix] gokceni commented on a change in pull request #1373: PHOENIX-6622 Transform monitor
gokceni commented on a change in pull request #1373:
URL: https://github.com/apache/phoenix/pull/1373#discussion_r796100863
##
File path:
phoenix-core/src/it/java/org/apache/phoenix/end2end/transform/TransformMonitorIT.java
##
@@ -0,0 +1,670 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end.transform;
+
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.coprocessor.TaskRegionObserver;
+import org.apache.phoenix.coprocessor.tasks.TransformMonitorTask;
+import org.apache.phoenix.end2end.ParallelStatsDisabledIT;
+import org.apache.phoenix.end2end.index.SingleCellIndexIT;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.ConnectionQueryServices;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.query.QueryServicesOptions;
+import org.apache.phoenix.schema.PTable;
+import org.apache.phoenix.schema.task.SystemTaskParams;
+import org.apache.phoenix.schema.task.Task;
+import org.apache.phoenix.schema.transform.SystemTransformRecord;
+import org.apache.phoenix.schema.transform.Transform;
+import org.apache.phoenix.util.EnvironmentEdgeManager;
+import org.apache.phoenix.util.MetaDataUtil;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.SchemaUtil;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.List;
+import java.util.Properties;
+
+import static org.apache.phoenix.end2end.IndexRebuildTaskIT.waitForTaskState;
+import static
org.apache.phoenix.exception.SQLExceptionCode.CANNOT_CREATE_TENANT_SPECIFIC_TABLE;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.apache.phoenix.util.TestUtil.getRawRowCount;
+import static org.apache.phoenix.util.TestUtil.getRowCount;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+public class TransformMonitorIT extends ParallelStatsDisabledIT {
+private static RegionCoprocessorEnvironment TaskRegionEnvironment;
+
+private Properties testProps = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+
+public TransformMonitorIT() throws IOException, InterruptedException {
+testProps.put(QueryServices.DEFAULT_IMMUTABLE_STORAGE_SCHEME_ATTRIB,
"ONE_CELL_PER_COLUMN");
+testProps.put(QueryServices.DEFAULT_COLUMN_ENCODED_BYTES_ATRRIB, "0");
+testProps.put(QueryServices.PHOENIX_ACLS_ENABLED, "true");
+
+TaskRegionEnvironment = (RegionCoprocessorEnvironment) getUtility()
+.getRSForFirstRegionInTable(
+PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+
.getOnlineRegions(PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+.get(0).getCoprocessorHost()
+
.findCoprocessorEnvironment(TaskRegionObserver.class.getName());
+}
+
+@Before
+public void setupTest() throws Exception {
+try (Connection conn = DriverManager.getConnection(getUrl(),
testProps)) {
+conn.setAutoCommit(true);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TRANSFORM_NAME);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TASK_NAME);
+}
+}
+
+private void testTransformTable(boolean createIndex, boolean createView,
boolean isImmutable) throws Exception {
+String schemaName = generateUniqueName();
+String dataTableName = "TBL_" + generateUniqueName();
+String dataTableFullName =
[GitHub] [phoenix] gokceni commented on a change in pull request #1373: PHOENIX-6622 Transform monitor
gokceni commented on a change in pull request #1373:
URL: https://github.com/apache/phoenix/pull/1373#discussion_r796101194
##
File path:
phoenix-core/src/it/java/org/apache/phoenix/end2end/transform/TransformMonitorIT.java
##
@@ -0,0 +1,670 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end.transform;
+
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.coprocessor.TaskRegionObserver;
+import org.apache.phoenix.coprocessor.tasks.TransformMonitorTask;
+import org.apache.phoenix.end2end.ParallelStatsDisabledIT;
+import org.apache.phoenix.end2end.index.SingleCellIndexIT;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.ConnectionQueryServices;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.query.QueryServicesOptions;
+import org.apache.phoenix.schema.PTable;
+import org.apache.phoenix.schema.task.SystemTaskParams;
+import org.apache.phoenix.schema.task.Task;
+import org.apache.phoenix.schema.transform.SystemTransformRecord;
+import org.apache.phoenix.schema.transform.Transform;
+import org.apache.phoenix.util.EnvironmentEdgeManager;
+import org.apache.phoenix.util.MetaDataUtil;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.SchemaUtil;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.List;
+import java.util.Properties;
+
+import static org.apache.phoenix.end2end.IndexRebuildTaskIT.waitForTaskState;
+import static
org.apache.phoenix.exception.SQLExceptionCode.CANNOT_CREATE_TENANT_SPECIFIC_TABLE;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.apache.phoenix.util.TestUtil.getRawRowCount;
+import static org.apache.phoenix.util.TestUtil.getRowCount;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+public class TransformMonitorIT extends ParallelStatsDisabledIT {
+private static RegionCoprocessorEnvironment TaskRegionEnvironment;
+
+private Properties testProps = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+
+public TransformMonitorIT() throws IOException, InterruptedException {
+testProps.put(QueryServices.DEFAULT_IMMUTABLE_STORAGE_SCHEME_ATTRIB,
"ONE_CELL_PER_COLUMN");
+testProps.put(QueryServices.DEFAULT_COLUMN_ENCODED_BYTES_ATRRIB, "0");
+testProps.put(QueryServices.PHOENIX_ACLS_ENABLED, "true");
+
+TaskRegionEnvironment = (RegionCoprocessorEnvironment) getUtility()
+.getRSForFirstRegionInTable(
+PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+
.getOnlineRegions(PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+.get(0).getCoprocessorHost()
+
.findCoprocessorEnvironment(TaskRegionObserver.class.getName());
+}
+
+@Before
+public void setupTest() throws Exception {
+try (Connection conn = DriverManager.getConnection(getUrl(),
testProps)) {
+conn.setAutoCommit(true);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TRANSFORM_NAME);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TASK_NAME);
+}
+}
+
+private void testTransformTable(boolean createIndex, boolean createView,
boolean isImmutable) throws Exception {
+String schemaName = generateUniqueName();
+String dataTableName = "TBL_" + generateUniqueName();
+String dataTableFullName =
[jira] [Commented] (PHOENIX-6622) TransformMonitor should orchestrate transform and do retries
[
https://issues.apache.org/jira/browse/PHOENIX-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484962#comment-17484962
]
ASF GitHub Bot commented on PHOENIX-6622:
-
gokceni commented on a change in pull request #1373:
URL: https://github.com/apache/phoenix/pull/1373#discussion_r796100863
##
File path:
phoenix-core/src/it/java/org/apache/phoenix/end2end/transform/TransformMonitorIT.java
##
@@ -0,0 +1,670 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end.transform;
+
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.coprocessor.TaskRegionObserver;
+import org.apache.phoenix.coprocessor.tasks.TransformMonitorTask;
+import org.apache.phoenix.end2end.ParallelStatsDisabledIT;
+import org.apache.phoenix.end2end.index.SingleCellIndexIT;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.ConnectionQueryServices;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.query.QueryServicesOptions;
+import org.apache.phoenix.schema.PTable;
+import org.apache.phoenix.schema.task.SystemTaskParams;
+import org.apache.phoenix.schema.task.Task;
+import org.apache.phoenix.schema.transform.SystemTransformRecord;
+import org.apache.phoenix.schema.transform.Transform;
+import org.apache.phoenix.util.EnvironmentEdgeManager;
+import org.apache.phoenix.util.MetaDataUtil;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.SchemaUtil;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.List;
+import java.util.Properties;
+
+import static org.apache.phoenix.end2end.IndexRebuildTaskIT.waitForTaskState;
+import static
org.apache.phoenix.exception.SQLExceptionCode.CANNOT_CREATE_TENANT_SPECIFIC_TABLE;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.apache.phoenix.util.TestUtil.getRawRowCount;
+import static org.apache.phoenix.util.TestUtil.getRowCount;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+public class TransformMonitorIT extends ParallelStatsDisabledIT {
+private static RegionCoprocessorEnvironment TaskRegionEnvironment;
+
+private Properties testProps = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+
+public TransformMonitorIT() throws IOException, InterruptedException {
+testProps.put(QueryServices.DEFAULT_IMMUTABLE_STORAGE_SCHEME_ATTRIB,
"ONE_CELL_PER_COLUMN");
+testProps.put(QueryServices.DEFAULT_COLUMN_ENCODED_BYTES_ATRRIB, "0");
+testProps.put(QueryServices.PHOENIX_ACLS_ENABLED, "true");
+
+TaskRegionEnvironment = (RegionCoprocessorEnvironment) getUtility()
+.getRSForFirstRegionInTable(
+PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+
.getOnlineRegions(PhoenixDatabaseMetaData.SYSTEM_TASK_HBASE_TABLE_NAME)
+.get(0).getCoprocessorHost()
+
.findCoprocessorEnvironment(TaskRegionObserver.class.getName());
+}
+
+@Before
+public void setupTest() throws Exception {
+try (Connection conn = DriverManager.getConnection(getUrl(),
testProps)) {
+conn.setAutoCommit(true);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TRANSFORM_NAME);
+conn.createStatement().execute("DELETE FROM " +
PhoenixDatabaseMetaData.SYSTEM_TASK_NAME);
+}
+}
+
+private void
[GitHub] [phoenix] yanxinyi merged pull request #1380: PhoenixResultSet#next() closes the result set if scanner returns null
yanxinyi merged pull request #1380: URL: https://github.com/apache/phoenix/pull/1380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (PHOENIX-6622) TransformMonitor should orchestrate transform and do retries
[
https://issues.apache.org/jira/browse/PHOENIX-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484961#comment-17484961
]
ASF GitHub Bot commented on PHOENIX-6622:
-
gokceni commented on a change in pull request #1373:
URL: https://github.com/apache/phoenix/pull/1373#discussion_r796096890
##
File path:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/tasks/TransformMonitorTask.java
##
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.coprocessor.tasks;
+
+import
org.apache.phoenix.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.mapreduce.Cluster;
+import org.apache.hadoop.mapreduce.Job;
+import org.apache.phoenix.coprocessor.TaskRegionObserver;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.mapreduce.transform.TransformTool;
+import org.apache.phoenix.schema.PTable;
+import org.apache.phoenix.schema.task.SystemTaskParams;
+import org.apache.phoenix.schema.task.Task;
+import org.apache.phoenix.schema.transform.SystemTransformRecord;
+import org.apache.phoenix.schema.transform.Transform;
+import org.apache.phoenix.util.QueryUtil;
+import org.apache.phoenix.util.SchemaUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.sql.Timestamp;
+
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.DEFAULT_TRANSFORM_MONITOR_ENABLED;
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.DEFAULT_TRANSFORM_RETRY_COUNT;
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.TRANSFORM_MONITOR_ENABLED;
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.TRANSFORM_RETRY_COUNT_VALUE;
+
+/**
+ * Task runs periodically to monitor and orchestrate ongoing transforms in
System.Transform table.
+ *
+ */
+public class TransformMonitorTask extends BaseTask {
+public static final String DEFAULT = "IndexName";
+
+public static final Logger LOGGER =
LoggerFactory.getLogger(TransformMonitorTask.class);
+
+private static boolean isDisabled = false;
+
+// Called from testong
+@VisibleForTesting
+public static void disableTransformMonitorTask(boolean disabled) {
+isDisabled = disabled;
+}
+
+@Override
+public TaskRegionObserver.TaskResult run(Task.TaskRecord taskRecord) {
+Configuration conf = HBaseConfiguration.create(env.getConfiguration());
+Configuration configuration =
HBaseConfiguration.addHbaseResources(conf);
+boolean transformMonitorEnabled =
configuration.getBoolean(TRANSFORM_MONITOR_ENABLED,
DEFAULT_TRANSFORM_MONITOR_ENABLED);
+if (!transformMonitorEnabled || isDisabled) {
+return new
TaskRegionObserver.TaskResult(TaskRegionObserver.TaskResultCode.FAIL,
"TransformMonitor is disabled");
+}
+
+try (PhoenixConnection conn =
QueryUtil.getConnectionOnServer(conf).unwrap(PhoenixConnection.class)){
+SystemTransformRecord systemTransformRecord =
Transform.getTransformRecord(taskRecord.getSchemaName(),
+taskRecord.getTableName(), null, taskRecord.getTenantId(),
conn);
+if (systemTransformRecord == null) {
+return new
TaskRegionObserver.TaskResult(TaskRegionObserver.TaskResultCode.FAIL,
+"No transform record is found");
+}
+String tableName =
SchemaUtil.getTableName(systemTransformRecord.getSchemaName(),
+systemTransformRecord.getLogicalTableName());
+
+if
(systemTransformRecord.getTransformStatus().equals(PTable.TransformStatus.CREATED.name()))
{
+LOGGER.info("Transform is created, starting the TransformTool
", tableName);
+// Kick a TransformTool run, it will already update transform
record status and job id
+TransformTool transformTool =
TransformTool.runTransformTool(systemTransformRecord, conf, false, null, null,
false,
[GitHub] [phoenix] gokceni commented on a change in pull request #1373: PHOENIX-6622 Transform monitor
gokceni commented on a change in pull request #1373:
URL: https://github.com/apache/phoenix/pull/1373#discussion_r796096890
##
File path:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/tasks/TransformMonitorTask.java
##
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.coprocessor.tasks;
+
+import
org.apache.phoenix.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.mapreduce.Cluster;
+import org.apache.hadoop.mapreduce.Job;
+import org.apache.phoenix.coprocessor.TaskRegionObserver;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.mapreduce.transform.TransformTool;
+import org.apache.phoenix.schema.PTable;
+import org.apache.phoenix.schema.task.SystemTaskParams;
+import org.apache.phoenix.schema.task.Task;
+import org.apache.phoenix.schema.transform.SystemTransformRecord;
+import org.apache.phoenix.schema.transform.Transform;
+import org.apache.phoenix.util.QueryUtil;
+import org.apache.phoenix.util.SchemaUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.sql.Timestamp;
+
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.DEFAULT_TRANSFORM_MONITOR_ENABLED;
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.DEFAULT_TRANSFORM_RETRY_COUNT;
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.TRANSFORM_MONITOR_ENABLED;
+import static
org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.TRANSFORM_RETRY_COUNT_VALUE;
+
+/**
+ * Task runs periodically to monitor and orchestrate ongoing transforms in
System.Transform table.
+ *
+ */
+public class TransformMonitorTask extends BaseTask {
+public static final String DEFAULT = "IndexName";
+
+public static final Logger LOGGER =
LoggerFactory.getLogger(TransformMonitorTask.class);
+
+private static boolean isDisabled = false;
+
+// Called from testong
+@VisibleForTesting
+public static void disableTransformMonitorTask(boolean disabled) {
+isDisabled = disabled;
+}
+
+@Override
+public TaskRegionObserver.TaskResult run(Task.TaskRecord taskRecord) {
+Configuration conf = HBaseConfiguration.create(env.getConfiguration());
+Configuration configuration =
HBaseConfiguration.addHbaseResources(conf);
+boolean transformMonitorEnabled =
configuration.getBoolean(TRANSFORM_MONITOR_ENABLED,
DEFAULT_TRANSFORM_MONITOR_ENABLED);
+if (!transformMonitorEnabled || isDisabled) {
+return new
TaskRegionObserver.TaskResult(TaskRegionObserver.TaskResultCode.FAIL,
"TransformMonitor is disabled");
+}
+
+try (PhoenixConnection conn =
QueryUtil.getConnectionOnServer(conf).unwrap(PhoenixConnection.class)){
+SystemTransformRecord systemTransformRecord =
Transform.getTransformRecord(taskRecord.getSchemaName(),
+taskRecord.getTableName(), null, taskRecord.getTenantId(),
conn);
+if (systemTransformRecord == null) {
+return new
TaskRegionObserver.TaskResult(TaskRegionObserver.TaskResultCode.FAIL,
+"No transform record is found");
+}
+String tableName =
SchemaUtil.getTableName(systemTransformRecord.getSchemaName(),
+systemTransformRecord.getLogicalTableName());
+
+if
(systemTransformRecord.getTransformStatus().equals(PTable.TransformStatus.CREATED.name()))
{
+LOGGER.info("Transform is created, starting the TransformTool
", tableName);
+// Kick a TransformTool run, it will already update transform
record status and job id
+TransformTool transformTool =
TransformTool.runTransformTool(systemTransformRecord, conf, false, null, null,
false, false);
+if (transformTool == null) {
+// This is not a map/reduce error. There must be some
unexpected issue. So, retrying will not solve the underlying issue.
+return new
[jira] [Commented] (PHOENIX-628) Support native JSON data type
[ https://issues.apache.org/jira/browse/PHOENIX-628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484947#comment-17484947 ] Gokcen Iskender commented on PHOENIX-628: - Here is a design doc about native Json support for Phoenix. [https://docs.google.com/document/d/1bpnADsXmvB-hlen2UWBFdrrKnW98A_dW0kJ3QIosMxo/edit?usp=sharing] I will attach a copy of this to the Jira as well. We would like to use BSON since our tests shows that to perform better. [~kozdemir] [~larsh] > Support native JSON data type > - > > Key: PHOENIX-628 > URL: https://issues.apache.org/jira/browse/PHOENIX-628 > Project: Phoenix > Issue Type: Task >Affects Versions: 4.4.0 >Reporter: James R. Taylor > Labels: JSON, Java, SQL > Fix For: 4.4.1 > > > MongoDB and PostGres do some interesting things with JSON. We should look at > adding similar support. For a detailed description, see JSONB support in > Postgres: > http://www.craigkerstiens.com/2014/03/24/Postgres-9.4-Looking-up > http://www.depesz.com/2014/03/25/waiting-for-9-4-introduce-jsonb-a-structured-format-for-storing-json/ > http://michael.otacoo.com/postgresql-2/manipulating-jsonb-data-with-key-unique/ -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (PHOENIX-6626) Make the without.tephra profile behaviour the default in 4.x and master, and remove the profile
[ https://issues.apache.org/jira/browse/PHOENIX-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484885#comment-17484885 ] ASF GitHub Bot commented on PHOENIX-6626: - stoty opened a new pull request #1382: URL: https://github.com/apache/phoenix/pull/1382 … 4.x and master, and remove the profile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Make the without.tephra profile behaviour the default in 4.x and master, and > remove the profile > --- > > Key: PHOENIX-6626 > URL: https://issues.apache.org/jira/browse/PHOENIX-6626 > Project: Phoenix > Issue Type: Sub-task > Components: core, tephra >Affects Versions: 4.17.0, 5.2.0 > Environment: Doing this is enough to get rid of all the problematic > dependencies that tephra includes and pulls in. >Reporter: Istvan Toth >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [phoenix] stoty opened a new pull request #1382: PHOENIX-6626 Make the without.tephra profile behaviour the default in…
stoty opened a new pull request #1382: URL: https://github.com/apache/phoenix/pull/1382 … 4.x and master, and remove the profile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (PHOENIX-6626) Make the without.tephra profile behaviour the default in 4.x and master, and remove the profile
[ https://issues.apache.org/jira/browse/PHOENIX-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484869#comment-17484869 ] ASF GitHub Bot commented on PHOENIX-6626: - stoty opened a new pull request #1381: URL: https://github.com/apache/phoenix/pull/1381 … 4.x and master, and remove the profile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Make the without.tephra profile behaviour the default in 4.x and master, and > remove the profile > --- > > Key: PHOENIX-6626 > URL: https://issues.apache.org/jira/browse/PHOENIX-6626 > Project: Phoenix > Issue Type: Sub-task > Components: core, tephra >Affects Versions: 4.17.0, 5.2.0 > Environment: Doing this is enough to get rid of all the problematic > dependencies that tephra includes and pulls in. >Reporter: Istvan Toth >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [phoenix] stoty opened a new pull request #1381: PHOENIX-6626 Make the without.tephra profile behaviour the default in…
stoty opened a new pull request #1381: URL: https://github.com/apache/phoenix/pull/1381 … 4.x and master, and remove the profile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (PHOENIX-6634) PhoenixResultSet.next method closes the result set if the scanner returns null
[
https://issues.apache.org/jira/browse/PHOENIX-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484859#comment-17484859
]
ASF GitHub Bot commented on PHOENIX-6634:
-
yanxinyi closed pull request #1377:
URL: https://github.com/apache/phoenix/pull/1377
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> PhoenixResultSet.next method closes the result set if the scanner returns null
> --
>
> Key: PHOENIX-6634
> URL: https://issues.apache.org/jira/browse/PHOENIX-6634
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.16.1
>Reporter: Xinyi Yan
>Assignee: Xinyi Yan
>Priority: Major
> Fix For: 4.17.0, 4.16.2
>
>
>
> {code:java}
> public void test() throws Exception {
> String query = "SELECT a_string FROM " + tableName + " WHERE
> organization_id=? LIMIT 1";
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> try(Connection conn = DriverManager.getConnection(getUrl(), props)) {
> PreparedStatement statement = conn.prepareStatement(query);
> statement.setString(1, tenantId);
> ResultSet rs = statement.executeQuery();
> while (rs.next()) {
> }
> assertFalse(rs.next()); // exception happens here
> }
> } {code}
>
>
> {code:java}
> java.sql.SQLException: ERROR 1101 (XCL01): ResultSet is closed. at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:615)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:217)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.checkOpen(PhoenixResultSet.java:288)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:847)
> at org.apache.phoenix.end2end.QueryIT.test(QueryIT.java:184)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method){code}
> Need to remove the unnecessary close inside the next method.
>
> https://github.com/apache/phoenix/blob/4.x/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSet.java#L855
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (PHOENIX-6634) PhoenixResultSet.next method closes the result set if the scanner returns null
[
https://issues.apache.org/jira/browse/PHOENIX-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484858#comment-17484858
]
ASF GitHub Bot commented on PHOENIX-6634:
-
yanxinyi commented on pull request #1377:
URL: https://github.com/apache/phoenix/pull/1377#issuecomment-1026095989
Thanks @stoty. I will close this one and merge the backport PR to the 4.x
branch https://github.com/apache/phoenix/pull/1380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> PhoenixResultSet.next method closes the result set if the scanner returns null
> --
>
> Key: PHOENIX-6634
> URL: https://issues.apache.org/jira/browse/PHOENIX-6634
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.16.1
>Reporter: Xinyi Yan
>Assignee: Xinyi Yan
>Priority: Major
> Fix For: 4.17.0, 4.16.2
>
>
>
> {code:java}
> public void test() throws Exception {
> String query = "SELECT a_string FROM " + tableName + " WHERE
> organization_id=? LIMIT 1";
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> try(Connection conn = DriverManager.getConnection(getUrl(), props)) {
> PreparedStatement statement = conn.prepareStatement(query);
> statement.setString(1, tenantId);
> ResultSet rs = statement.executeQuery();
> while (rs.next()) {
> }
> assertFalse(rs.next()); // exception happens here
> }
> } {code}
>
>
> {code:java}
> java.sql.SQLException: ERROR 1101 (XCL01): ResultSet is closed. at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:615)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:217)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.checkOpen(PhoenixResultSet.java:288)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:847)
> at org.apache.phoenix.end2end.QueryIT.test(QueryIT.java:184)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method){code}
> Need to remove the unnecessary close inside the next method.
>
> https://github.com/apache/phoenix/blob/4.x/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSet.java#L855
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [phoenix] yanxinyi closed pull request #1377: PHOENIX-6634 PhoenixResultSet.next method closes the result set if th…
yanxinyi closed pull request #1377: URL: https://github.com/apache/phoenix/pull/1377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on pull request #1377: PHOENIX-6634 PhoenixResultSet.next method closes the result set if th…
yanxinyi commented on pull request #1377: URL: https://github.com/apache/phoenix/pull/1377#issuecomment-1026095989 Thanks @stoty. I will close this one and merge the backport PR to the 4.x branch https://github.com/apache/phoenix/pull/1380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (PHOENIX-6591) Update OWASP plugin to latest
[ https://issues.apache.org/jira/browse/PHOENIX-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484586#comment-17484586 ] Istvan Toth commented on PHOENIX-6591: -- Committed to phoenix-connectors. > Update OWASP plugin to latest > - > > Key: PHOENIX-6591 > URL: https://issues.apache.org/jira/browse/PHOENIX-6591 > Project: Phoenix > Issue Type: Bug > Components: connectors, core, queryserver >Affects Versions: connectors-6.0.0, queryserver-6.0.0, 5.2.0 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: queryserver-6.0.1 > > > The URLs used by the OWASP plugin to download CVE information are no longer > live. > The latest version (6.4.1) switches to working URLs. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (PHOENIX-6591) Update OWASP plugin to latest
[ https://issues.apache.org/jira/browse/PHOENIX-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484585#comment-17484585 ] ASF GitHub Bot commented on PHOENIX-6591: - stoty closed pull request #70: URL: https://github.com/apache/phoenix-connectors/pull/70 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Update OWASP plugin to latest > - > > Key: PHOENIX-6591 > URL: https://issues.apache.org/jira/browse/PHOENIX-6591 > Project: Phoenix > Issue Type: Bug > Components: connectors, core, queryserver >Affects Versions: connectors-6.0.0, queryserver-6.0.0, 5.2.0 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: queryserver-6.0.1 > > > The URLs used by the OWASP plugin to download CVE information are no longer > live. > The latest version (6.4.1) switches to working URLs. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [phoenix-connectors] stoty closed pull request #70: PHOENIX-6591 Update OWASP plugin to latest
stoty closed pull request #70: URL: https://github.com/apache/phoenix-connectors/pull/70 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (PHOENIX-6591) Update OWASP plugin to latest
[ https://issues.apache.org/jira/browse/PHOENIX-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484584#comment-17484584 ] Istvan Toth commented on PHOENIX-6591: -- Committed to phoenix-queryserver. > Update OWASP plugin to latest > - > > Key: PHOENIX-6591 > URL: https://issues.apache.org/jira/browse/PHOENIX-6591 > Project: Phoenix > Issue Type: Bug > Components: connectors, core, queryserver >Affects Versions: connectors-6.0.0, queryserver-6.0.0, 5.2.0 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: queryserver-6.0.1 > > > The URLs used by the OWASP plugin to download CVE information are no longer > live. > The latest version (6.4.1) switches to working URLs. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [phoenix-queryserver] stoty closed pull request #82: PHOENIX-6591 Update OWASP plugin to latest
stoty closed pull request #82: URL: https://github.com/apache/phoenix-queryserver/pull/82 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
