[jira] [Updated] (PHOENIX-4900) Modify MAX_MUTATION_SIZE_EXCEEDED and MAX_MUTATION_SIZE_BYTES_EXCEEDED exception message to recommend turning autocommit on for deletes
[ https://issues.apache.org/jira/browse/PHOENIX-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-4900: Fix Version/s: 5.0.1 > Modify MAX_MUTATION_SIZE_EXCEEDED and MAX_MUTATION_SIZE_BYTES_EXCEEDED > exception message to recommend turning autocommit on for deletes > --- > > Key: PHOENIX-4900 > URL: https://issues.apache.org/jira/browse/PHOENIX-4900 > Project: Phoenix > Issue Type: Improvement >Reporter: Thomas D'Silva >Assignee: Xinyi Yan >Priority: Major > Labels: SFDC > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-4900-4.x-HBase-1.4.patch, PHOENIX-4900.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5069) Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache
[ https://issues.apache.org/jira/browse/PHOENIX-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5069: Fix Version/s: 5.0.1 > Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache > --- > > Key: PHOENIX-5069 > URL: https://issues.apache.org/jira/browse/PHOENIX-5069 > Project: Phoenix > Issue Type: Improvement >Reporter: Bin Shi >Assignee: Bin Shi >Priority: Major > Fix For: 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5069-4.14.1-hbase-1.3-phoenix-stats.001.patch, > PHOENIX-5069-4.14.1-hbase-1.3-phoenix-stats.002.patch, > PHOENIX-5069.4.x-HBase-1.3.001.patch, PHOENIX-5069.4.x-HBase-1.4.001.patch, > PHOENIX-5069.master.001.patch, PHOENIX-5069.master.002.patch, > PHOENIX-5069.master.003.patch, PHOENIX-5069.master.004.patch, > PHOENIX-5069.patch > > Time Spent: 6h 40m > Remaining Estimate: 0h > > The current Phoenix Stats Cache uses TTL based eviction policy. A cached > entry will expire after a given amount of time (900s by default) passed since > the entry's been created. This will lead to cache miss when > Compiler/Optimizer fetches stats from cache at the next time. As you can see > from the above graph, fetching stats from the cache is a blocking operation — > when there is cache miss, it has a round trip over the wire to scan the > SYSTEM.STATS Table and to get the latest stats info, rebuild the cache and > finally return the stats to the Compiler/Optimizer. Whenever there is a cache > miss, this blocking call causes significant performance penalty and see > periodic spikes. > *This Jira suggests to use asynchronous refresh mechanism to provide a > non-blocking cache. For details, please see the linked design document below.* > [~karanmehta93] [~twdsi...@gmail.com] [~dbwong] [~elserj] [~an...@apache.org] > [~sergey soldatov] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5131) Make spilling to disk for order/group by configurable
[ https://issues.apache.org/jira/browse/PHOENIX-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5131: Fix Version/s: 5.0.1 > Make spilling to disk for order/group by configurable > - > > Key: PHOENIX-5131 > URL: https://issues.apache.org/jira/browse/PHOENIX-5131 > Project: Phoenix > Issue Type: Improvement >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Major > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5131-4.x-HBase-1.2.patch, > PHOENIX-5131-4.x-HBase-1.3.patch, PHOENIX-5131-4.x-HBase-1.4.patch, > PHOENIX-5131-master-v2.patch, PHOENIX-5131-master-v2.patch, > PHOENIX-5131-master-v3.patch, PHOENIX-5131-master-v4.patch, > PHOENIX-5131-master.patch, PHOENIX-5131-master.patch > > > We've observed that large queries, doing order/group by leading to issues on > the regionserver (crashes/long gc pauses/file handler exhaustion etc.). We > should make spilling to disk configurable and in case its disabled, fail the > query once it hits the spilling limit on any of the region servers. Also make > spooling threshold server-side property only to prevent clients from > controlling memory allocation on the rs side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4907) IndexScrutinyTool should use empty catalog instead of null
[ https://issues.apache.org/jira/browse/PHOENIX-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-4907: Fix Version/s: 5.0.1 > IndexScrutinyTool should use empty catalog instead of null > -- > > Key: PHOENIX-4907 > URL: https://issues.apache.org/jira/browse/PHOENIX-4907 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 5.0.0, 4.15.0 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 4.15.0, 4.14.1, 5.1.0, 5.0.1 > > Attachments: PHOENIX-4907.patch > > > Before executing, the index scrutiny tool does a sanity check to make sure > that the given data table and index are valid and related to each other. This > check uses the JDBC metadata API, and passes in null for the catalog name. > Unfortunately, a null entry for catalog causes Phoenix to omit tenant_id from > the query against System.Catalog, causing a table scan, which can be lengthy > or time out if the server has too many views. > It should pass in the empty string for catalog, which will make Phoenix > filter on "WHERE tenant_id is NULL", which will avoid the table scan. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5048) Index Rebuilder does not handle INDEX_STATE timestamp check for all index
[
https://issues.apache.org/jira/browse/PHOENIX-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-5048:
Fix Version/s: 5.0.1
> Index Rebuilder does not handle INDEX_STATE timestamp check for all index
> -
>
> Key: PHOENIX-5048
> URL: https://issues.apache.org/jira/browse/PHOENIX-5048
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.14.0, 5.0.0, 4.14.1
>Reporter: Mihir Monani
>Assignee: Mihir Monani
>Priority: Major
> Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1
>
> Attachments: PHOENIX-5048.patch, PHOENIX-5048.v2.patch,
> PHOENIX-5048.v3.patch, PHOENIX-5048.v4.patch, PHOENIX-5048.v5.patch
>
>
> After rebuilder is finished for Partial Index Rebuild, It will check if Index
> state has been updated after Upper bound of the scan we use in partial index
> Rebuild. If that happens then it will fail Index Rebuild as Index write
> failure occured while we were rebuilding Index.
> {code:java}
> MetaDataEndpointImpl.java#updateIndexState()
> public void updateIndexState(RpcController controller,
> UpdateIndexStateRequest request,
> RpcCallback done) {
> ...
> // If the index status has been updated after the upper bound of the scan we
> use
> // to partially rebuild the index, then we need to fail the rebuild because an
> // index write failed before the rebuild was complete.
> if (actualTimestamp > expectedTimestamp) {
> builder.setReturnCode(MetaDataProtos.MutationCode.UNALLOWED_TABLE_MUTATION);
> builder.setMutationTime(EnvironmentEdgeManager.currentTimeMillis());
> done.run(builder.build());
> return;
> }
> ...
> }{code}
> After Introduction of TrackingParallelWriterIndexCommitter
> [PHOENIX-3815|https://issues.apache.org/jira/browse/PHOENIX-3815], we only
> disable Index which get failure . Before that , in
> ParallelWriterIndexCommitter we were disabling all index even if Index
> failure happens for one Index only.
> Suppose Data Table has 3 index and above condition becomes true for first
> index , then we won't even check for remain two Index.
> {code:java}
> MetaDataRegionObserver.java#BuildIndexScheduleTask.java#run()
> for (PTable indexPTable : indexesToPartiallyRebuild) {
> String indexTableFullName = SchemaUtil.getTableName(
> indexPTable.getSchemaName().getString(),
> indexPTable.getTableName().getString());
> if (scanEndTime == latestUpperBoundTimestamp) {
> IndexUtil.updateIndexState(conn, indexTableFullName, PIndexState.ACTIVE, 0L,
> latestUpperBoundTimestamp);
> batchExecutedPerTableMap.remove(dataPTable.getName());
> LOG.info("Making Index:" + indexPTable.getTableName() + " active after
> rebuilding");
> } else {
> // Increment timestamp so that client sees updated disable timestamp
> IndexUtil.updateIndexState(conn, indexTableFullName,
> indexPTable.getIndexState(), scanEndTime * signOfDisableTimeStamp,
> latestUpperBoundTimestamp);
> Long noOfBatches = batchExecutedPerTableMap.get(dataPTable.getName());
> if (noOfBatches == null) {
> noOfBatches = 0l;
> }
> batchExecutedPerTableMap.put(dataPTable.getName(), ++noOfBatches);
> LOG.info("During Round-robin build: Successfully updated index disabled
> timestamp for "
> + indexTableFullName + " to " + scanEndTime);
> }
> }
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5122) PHOENIX-4322 breaks client backward compatibility
[
https://issues.apache.org/jira/browse/PHOENIX-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-5122:
Fix Version/s: 5.0.1
> PHOENIX-4322 breaks client backward compatibility
> -
>
> Key: PHOENIX-5122
> URL: https://issues.apache.org/jira/browse/PHOENIX-5122
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
>Reporter: Jacob Isaac
>Assignee: Jacob Isaac
>Priority: Blocker
> Fix For: 4.13.0, 4.13.1, 4.15.0, 4.14.1, 5.1.0, 4.14.2, 5.0.1
>
> Attachments: PHOENIX-5122-4.x-HBase-1.3.patch, PHOENIX-5122.patch,
> Screen Shot 2019-03-04 at 6.17.42 PM.png, Screen Shot 2019-03-04 at 6.21.10
> PM.png
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Scenario :
> *4.13 client -> 4.14.1 server*
> {noformat}
> Connected to: Phoenix (version 4.13)
> Driver: PhoenixEmbeddedDriver (version 4.13)
> Autocommit status: true
> Transaction isolation: TRANSACTION_READ_COMMITTED
> Building list of tables and columns for tab-completion (set fastconnect to
> true to skip)...
> 135/135 (100%) Done
> Done
> sqlline version 1.1.9
> 0: jdbc:phoenix:localhost>
> 0: jdbc:phoenix:localhost>
> 0: jdbc:phoenix:localhost> CREATE table P_T02 (oid VARCHAR NOT NULL, code
> VARCHAR NOT NULL constraint pk primary key (oid DESC, code DESC));
> No rows affected (1.31 seconds)
> 0: jdbc:phoenix:localhost>
> 0: jdbc:phoenix:localhost> upsert into P_T02 (oid, code) values ('0001',
> 'v0001');
> 1 row affected (0.033 seconds)
> 0: jdbc:phoenix:localhost> upsert into P_T02 (oid, code) values ('0002',
> 'v0002');
> 1 row affected (0.004 seconds)
> 0: jdbc:phoenix:localhost>
> 0: jdbc:phoenix:localhost> select * from P_T02 where (oid, code) IN
> (('0001', 'v0001'), ('0002', 'v0002'));
> +--+--+
> | OID | CODE |
> +--+--+
> +--+--+
> {color:#FF}+*No rows selected (0.033 seconds)*+{color}
> 0: jdbc:phoenix:localhost> select * from P_T02 ;
> +--+--+
> | OID | CODE |
> +--+--+
> | 0002 | v0002 |
> | 0001 | v0001 |
> +--+--+
> 2 rows selected (0.016 seconds)
> 0: jdbc:phoenix:localhost>
> {noformat}
> *4.14.1 client -> 4.14.1 server*
> {noformat}
> Connected to: Phoenix (version 4.14)
> Driver: PhoenixEmbeddedDriver (version 4.14)
> Autocommit status: true
> Transaction isolation: TRANSACTION_READ_COMMITTED
> Building list of tables and columns for tab-completion (set fastconnect to
> true to skip)...
> 133/133 (100%) Done
> Done
> sqlline version 1.1.9
> 0: jdbc:phoenix:localhost>
> 0: jdbc:phoenix:localhost> CREATE table P_T01 (oid VARCHAR NOT NULL, code
> VARCHAR NOT NULL constraint pk primary key (oid DESC, code DESC));
> No rows affected (1.273 seconds)
> 0: jdbc:phoenix:localhost>
> 0: jdbc:phoenix:localhost> upsert into P_T01 (oid, code) values ('0001',
> 'v0001');
> 1 row affected (0.056 seconds)
> 0: jdbc:phoenix:localhost> upsert into P_T01 (oid, code) values ('0002',
> 'v0002');
> 1 row affected (0.004 seconds)
> 0: jdbc:phoenix:localhost>
> 0: jdbc:phoenix:localhost> select * from P_T01 where (oid, code) IN
> (('0001', 'v0001'), ('0002', 'v0002'));
> +--+--+
> | OID | CODE |
> +--+--+
> | 0002 | v0002 |
> | 0001 | v0001 |
> +--+--+
> 2 rows selected (0.051 seconds)
> 0: jdbc:phoenix:localhost> select * from P_T01 ;
> +--+--+
> | OID | CODE |
> +--+--+
> | 0002 | v0002 |
> | 0001 | v0001 |
> +--+--+
> 2 rows selected (0.017 seconds)
> 0: jdbc:phoenix:localhost>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5101) ScanningResultIterator getScanMetrics throws NPE
[
https://issues.apache.org/jira/browse/PHOENIX-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-5101:
Fix Version/s: 5.0.1
> ScanningResultIterator getScanMetrics throws NPE
>
>
> Key: PHOENIX-5101
> URL: https://issues.apache.org/jira/browse/PHOENIX-5101
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.1
>Reporter: Reid Chan
>Assignee: Karan Mehta
>Priority: Blocker
> Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1
>
> Attachments: PHOENIX-5101.414-HBase-1.4.001.patch, PHOENIX-5101.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {code}
> java.lang.NullPointerException
> at
> org.apache.phoenix.iterate.ScanningResultIterator.getScanMetrics(ScanningResultIterator.java:92)
> at
> org.apache.phoenix.iterate.ScanningResultIterator.close(ScanningResultIterator.java:79)
> at
> org.apache.phoenix.iterate.TableResultIterator.close(TableResultIterator.java:144)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator$1.close(LookAheadResultIterator.java:42)
> at
> org.apache.phoenix.iterate.BaseResultIterators.close(BaseResultIterators.java:1439)
> at
> org.apache.phoenix.iterate.MergeSortResultIterator.close(MergeSortResultIterator.java:44)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.close(PhoenixResultSet.java:176)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:807)
> at
> org.apache.calcite.avatica.jdbc.JdbcResultSet.frame(JdbcResultSet.java:148)
> at
> org.apache.calcite.avatica.jdbc.JdbcResultSet.create(JdbcResultSet.java:101)
> at
> org.apache.calcite.avatica.jdbc.JdbcResultSet.create(JdbcResultSet.java:81)
> at
> org.apache.calcite.avatica.jdbc.JdbcMeta.prepareAndExecute(JdbcMeta.java:759)
> at
> org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:206)
> at
> org.apache.calcite.avatica.remote.Service$PrepareAndExecuteRequest.accept(Service.java:927)
> at
> org.apache.calcite.avatica.remote.Service$PrepareAndExecuteRequest.accept(Service.java:879)
> at
> org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:94)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:46)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler$2.call(AvaticaProtobufHandler.java:123)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler$2.call(AvaticaProtobufHandler.java:121)
> at
> org.apache.phoenix.queryserver.server.QueryServer$PhoenixDoAsCallback$1.run(QueryServer.java:500)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.phoenix.queryserver.server.QueryServer$PhoenixDoAsCallback.doAsRemoteUser(QueryServer.java:497)
> at
> org.apache.calcite.avatica.server.HttpServer$Builder$1.doAsRemoteUser(HttpServer.java:884)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:120)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:542)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5213) Phoenix-client improvements: add more relocations, exclude log binding, add source jar
[
https://issues.apache.org/jira/browse/PHOENIX-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-5213:
Fix Version/s: 5.0.1
> Phoenix-client improvements: add more relocations, exclude log binding, add
> source jar
> ---
>
> Key: PHOENIX-5213
> URL: https://issues.apache.org/jira/browse/PHOENIX-5213
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Fix For: 4.15.0, 5.1.0, 5.0.1
>
> Attachments: PHOENIX-5213.4.x-HBase-1.4.v1.patch,
> PHOENIX-5213.4.x-HBase-1.4.v2.patch, PHOENIX-5213.4.x-HBase-1.4.v3.patch,
> PHOENIX-5213.4.x-HBase-1.4.v4.patch
>
>
> To make the existing phoenix-client, I'm proposing the following changes:
> 1) Add additional relocations of some packages
> Add a new "embedded" classifier to phoenix-client that does the following:
> 2) Exclude the slf4j-log4j12 binding. Apparently this isn't pulled in
> directly from phoenix-core itself, but transitively from other projects.
> It's generally considered best practice to not impose a log binding on
> downstream projects. The slf4j-log4j12 jar will still be in the phoenix
> tarball's /lib folder.
> 3) Create a source jar for phoenix-client embedded.
> 4) Create a dependency-reduced pom, so that the client can be used directly
> in downstream projects without having to exclude transitive artifacts.
> 5) rename the jar to match the final name in the repository:
> phoenix-client-{version}.jar There is now a symlink
> phoenix-{version}-client.jar to maintain backwards compatibility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5126) RegionScanner leak leading to store files not getting cleared
[ https://issues.apache.org/jira/browse/PHOENIX-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5126: Fix Version/s: 5.0.1 > RegionScanner leak leading to store files not getting cleared > - > > Key: PHOENIX-5126 > URL: https://issues.apache.org/jira/browse/PHOENIX-5126 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Major > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5126-master.patch > > > Having a regionScanner open indefinitely (due to any error condition before > the close) leads to the store files not getting cleared after compaction > since the already open scanner will have the store file referenced. Any > subsequent flushed files for the region also get opened by the scanner and > wont be cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5207) Create index if not exists fails incorrectly if table has 'maxIndexesPerTable' indexes already
[ https://issues.apache.org/jira/browse/PHOENIX-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5207: Fix Version/s: 5.0.1 > Create index if not exists fails incorrectly if table has > 'maxIndexesPerTable' indexes already > --- > > Key: PHOENIX-5207 > URL: https://issues.apache.org/jira/browse/PHOENIX-5207 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Major > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5207-4.14-HBase-1.4.patch, > PHOENIX-5207-master.patch > > > If a table has 'maxIndexesPerTable' indexes and we try to create another one > and if it already exists we should not throw 'ERROR 1047 (43A04): Too many > indexes have already been created' since weve put 'if not exists' already in > the statement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5184) HBase and Phoenix connection leaks in Indexing code path, OrphanViewTool and PhoenixConfigurationUtil
[ https://issues.apache.org/jira/browse/PHOENIX-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5184: Fix Version/s: 5.0.1 > HBase and Phoenix connection leaks in Indexing code path, OrphanViewTool and > PhoenixConfigurationUtil > - > > Key: PHOENIX-5184 > URL: https://issues.apache.org/jira/browse/PHOENIX-5184 > Project: Phoenix > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni >Priority: Major > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5184-4.x-HBase-1.3-v1.patch, > PHOENIX-5184-4.x-HBase-1.3.patch, PHOENIX-5184-v1.patch, PHOENIX-5184.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > I was debugging a connection leak issue and ran into a few areas where there > are connection leaks. I decided to take a broader look overall and see if > there were other places where we leak connections and found some candidates. > This is by no means an exhaustive search for connection leaks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5217) Incorrect result for COUNT DISTINCT limit
[
https://issues.apache.org/jira/browse/PHOENIX-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-5217:
Fix Version/s: 5.0.1
> Incorrect result for COUNT DISTINCT limit
> --
>
> Key: PHOENIX-5217
> URL: https://issues.apache.org/jira/browse/PHOENIX-5217
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.1
> Environment: 4.14.1: incorrect
> 4.6: correct.
>
>Reporter: Chen Feng
>Assignee: chenglei
>Priority: Critical
> Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1
>
> Attachments: PHOENIX-5217-4.14-HBase-1.4.patch,
> PHOENIX-5217_v1-4.x-HBase-1.4.patch, PHOENIX-5217_v2-master.patch
>
>
> For table t1(pk1, col1, CONSTRAINT(pk1))
> upsert into "t1" values (1, 1);
> upsert into "t1" values (2, 2);
> sql A: select count("pk1") from "t1" limit 1, return 2 [correct]
> sql B: select count(disctinct("pk1")) from "t1" limit 1, return 1 [incorrect]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5169) Query logger is still initialized for each query when the log level is off
[ https://issues.apache.org/jira/browse/PHOENIX-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5169: Fix Version/s: 5.0.1 > Query logger is still initialized for each query when the log level is off > -- > > Key: PHOENIX-5169 > URL: https://issues.apache.org/jira/browse/PHOENIX-5169 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0 >Reporter: jaanai >Assignee: jaanai >Priority: Major > Fix For: 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5169-master-v2.patch, > PHOENIX-5169-master-v3.patch, PHOENIX-5169-master-v4.patch, > PHOENIX-5169-master.patch, image-2019-02-28-10-05-00-518.png > > > we will still invoke createQueryLogger in PhoenixStatement for each query > when query logger level is OFF, which has significant throughput impacts > under multiple threads. > The below is jstack with the concurrent query: > !https://gw.alicdn.com/tfscom/TB1HC3bI4TpK1RjSZFMXXbG_VXa.png|width=500,height=400! > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5094) Index can transition from INACTIVE to ACTIVE via Phoenix Client
[ https://issues.apache.org/jira/browse/PHOENIX-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5094: Fix Version/s: 5.0.1 > Index can transition from INACTIVE to ACTIVE via Phoenix Client > --- > > Key: PHOENIX-5094 > URL: https://issues.apache.org/jira/browse/PHOENIX-5094 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.14.1 >Reporter: Mihir Monani >Assignee: Kiran Kumar Maturi >Priority: Major > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5094-4.14-HBase-1.3.01.patch, > PHOENIX-5094-4.14-HBase-1.3.02.patch, PHOENIX-5094-4.14-HBase-1.3.03.patch, > PHOENIX-5094-4.14-HBase-1.3.04.patch, PHOENIX-5094-4.14-HBase-1.3.05.patch, > PHOENIX-5094-master.01.patch, PHOENIX-5094-master.02.patch, > PHOENIX-5094-master.03.patch > > > Suppose Index is in INACTIVE state and Client load is running continuously. > With INACTIVE State, client will keep maintaining index. > Before Rebuilder could run and bring index back in sync with data table, If > some mutation for Index fails from client side, then client will transition > Index state (From INACTIVE--> PENDING_DISABLE). > If client succeeds in writing mutation in subsequent retries, it will > transition Index state again ( From PENDING_DISABLE --> ACTIVE) . > Above scenario will leave some part of Index out of sync with data table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5188) IndexedKeyValue should populate KeyValue fields
[ https://issues.apache.org/jira/browse/PHOENIX-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5188: Fix Version/s: 5.0.1 > IndexedKeyValue should populate KeyValue fields > --- > > Key: PHOENIX-5188 > URL: https://issues.apache.org/jira/browse/PHOENIX-5188 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.14.1 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5188-4.x-HBase-1.4..addendum.patch, > PHOENIX-5188-4.x-HBase-1.4.patch, PHOENIX-5188.patch > > > IndexedKeyValue subclasses the HBase KeyValue class, which has three primary > fields: bytes, offset, and length. These fields aren't populated by > IndexedKeyValue because it's concerned with index mutations, and has its own > fields that its own methods use. > However, KeyValue and its Cell interface have quite a few methods that assume > these fields are populated, and the HBase-level factory methods generally > ensure they're populated. Phoenix code should do the same, to maintain the > polymorphic contract. This is important in cases like custom > ReplicationEndpoints where HBase-level code may be iterating over WALEdits > that contain both KeyValues and IndexKeyValues and may need to interrogate > their contents. > Since the index mutation has a row key, this is straightforward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5018) Index mutations created by UPSERT SELECT will have wrong timestamps
[ https://issues.apache.org/jira/browse/PHOENIX-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5018: Fix Version/s: 5.0.1 > Index mutations created by UPSERT SELECT will have wrong timestamps > --- > > Key: PHOENIX-5018 > URL: https://issues.apache.org/jira/browse/PHOENIX-5018 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.0, 5.0.0 >Reporter: Geoffrey Jacoby >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5018.4.x-HBase-1.3.001.patch, > PHOENIX-5018.4.x-HBase-1.3.002.patch, PHOENIX-5018.4.x-HBase-1.4.001.patch, > PHOENIX-5018.4.x-HBase-1.4.002.patch, PHOENIX-5018.master.001.patch, > PHOENIX-5018.master.002.patch, PHOENIX-5018.master.003.patch, > PHOENIX-5018.master.004.patch > > Time Spent: 5h 40m > Remaining Estimate: 0h > > When doing a full rebuild (or initial async build) of a local or global index > using IndexTool and PhoenixIndexImportDirectMapper, or doing a synchronous > initial build of a global index using the index create DDL, we generate the > index mutations by using an UPSERT SELECT query from the base table to the > index. > The timestamps of the mutations use the default HBase behavior, which is to > take the current wall clock. However, the timestamp of an index KeyValue > should use the timestamp of the initial KeyValue in the base table. > Having base table and index timestamps out of sync can cause all sorts of > weird side effects, such as if the base table has data with an expired TTL > that isn't expired in the index yet. Also inserting old mutations with new > timestamps may overwrite the data that has been newly overwritten by the > regular data path during index build, which would lead to data loss and > inconsistency issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5226) The format of VIEW_MODIFIED_PROPERTY_BYTES is incorrect as a tag of the cell
[
https://issues.apache.org/jira/browse/PHOENIX-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-5226:
Fix Version/s: 5.0.1
> The format of VIEW_MODIFIED_PROPERTY_BYTES is incorrect as a tag of the cell
> -
>
> Key: PHOENIX-5226
> URL: https://issues.apache.org/jira/browse/PHOENIX-5226
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: jaanai
>Assignee: jaanai
>Priority: Critical
> Fix For: 4.15.0, 5.1.0, 5.0.1
>
> Attachments: PHOENIX-5226-master-v2.patch,
> PHOENIX-5226-master-v3.patch, PHOENIX-5226-master.patch, Screen Shot
> 2019-04-01 at 16.09.23.png, Screen Shot 2019-04-01 at 16.13.10.png
>
>
> We use a tag of cell to indicat that some properties should not be derived
> from the base table for view table. VIEW_MODIFIED_PROPERTY_BYTES is used a
> tag bytes, but the format is incorrect, the below is a reference from
> KeyValue interface:
> {quote}KeyValue can optionally contain Tags. When it contains tags, it is
> added in the byte array after
> * the value part. The format for this part is:
> .
> * tagslength maximum is Short.MAX_SIZE. The
> tagsbytes
> * contain one or more tags where as each tag is of the form
> * . tagtype is one
> byte
> * and taglength maximum is Short.MAX_SIZE and it
> includes 1 byte type
> * length and actual tag bytes length.{quote}
> The CATALOG table will be badly affected. Some errors will be caused when
> reads CATALOG table.
>
> {code:java}
> 0: jdbc:phoenix:thin:url=http://localhost> drop view "test_2"; Error: Error
> -1 (0) : Error while executing SQL "drop view "test_2"": Remote driver
> error: RuntimeException: org.apache.phoenix.exception.PhoenixIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException: test_2: 4 at
> org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:114) at
> org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropTable(MetaDataEndpointImpl.java:2729)
> at
> org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:17078)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:8210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:2475)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2457)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42010)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:418) at
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:136) at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324) at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4 at
> org.apache.hadoop.hbase.ArrayBackedTag.(ArrayBackedTag.java:97) at
> org.apache.hadoop.hbase.CellUtil$5.next(CellUtil.java:1107) at
> org.apache.hadoop.hbase.CellUtil$5.next(CellUtil.java:1094) at
> org.apache.hadoop.hbase.regionserver.querymatcher.ScanQueryMatcher.isCellTTLExpired(ScanQueryMatcher.java:153)
> at
> org.apache.hadoop.hbase.regionserver.querymatcher.ScanQueryMatcher.preCheck(ScanQueryMatcher.java:198)
> at
> org.apache.hadoop.hbase.regionserver.querymatcher.NormalUserScanQueryMatcher.match(NormalUserScanQueryMatcher.java:64)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:578)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5055) Split mutations batches probably affects correctness of index data
[
https://issues.apache.org/jira/browse/PHOENIX-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-5055:
Fix Version/s: 5.0.1
> Split mutations batches probably affects correctness of index data
> --
>
> Key: PHOENIX-5055
> URL: https://issues.apache.org/jira/browse/PHOENIX-5055
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.14.1
>Reporter: jaanai
>Assignee: jaanai
>Priority: Critical
> Fix For: 5.1.0, 4.14.2, 5.0.1
>
> Attachments: ConcurrentTest.java,
> PHOENIX-5055-4.x-HBase-1.4-v2.patch, PHOENIX-5055-4.x-HBase-1.4-v3.patch,
> PHOENIX-5055-4.x-HBase-1.4-v4.patch, PHOENIX-5055-v4.x-HBase-1.4.patch
>
>
> In order to get more performance, we split the list of mutations into
> multiple batches in MutationSate. For one upsert SQL with some null values
> that will produce two type KeyValues(Put and DeleteColumn), These KeyValues
> should have the same timestamp so that keep on an atomic operation for
> corresponding the row key.
> [^ConcurrentTest.java] produced some random upsert/delete SQL and
> concurrently executed, some SQL snippets as follows:
> {code:java}
> 1149:UPSERT INTO ConcurrentReadWritTest(A,C,E,F,G) VALUES
> ('3826','2563','3052','3170','3767');
> 1864:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,E,F,G) VALUES
> ('2563','4926','3526','678',null,null,'1617');
> 2332:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,E,F,G) VALUES
> ('1052','2563','1120','2314','1456',null,null);
> 2846:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,G) VALUES
> ('1922','146',null,'469','2563');
> 2847:DELETE FROM ConcurrentReadWritTest WHERE A = '2563’;
> {code}
> Found incorrect indexed data for the index tables by sqlline.
> !https://gw.alicdn.com/tfscom/TB1nSDqpxTpK1RjSZFGXXcHqFXa.png|width=665,height=400!
> Debugged the mutations of batches on the server side. the DeleteColumns and
> Puts were splitted into the different batches for the once upsert, the
> DeleteFaimly also was executed by another thread. due to DeleteColumns's
> timestamp is larger than DeleteFaimly under multiple threads.
> !https://gw.alicdn.com/tfscom/TB1frHmpCrqK1RjSZK9XXXyypXa.png|width=901,height=120!
>
> Running the following:
> {code:java}
> conn.createStatement().executeUpdate( "CREATE TABLE " + tableName + " (" + "A
> VARCHAR NOT NULL PRIMARY KEY," + "B VARCHAR," + "C VARCHAR," + "D VARCHAR)
> COLUMN_ENCODED_BYTES = 0");
> conn.createStatement().executeUpdate("CREATE INDEX " + indexName + " on " +
> tableName + " (C) INCLUDE(D)");
> conn.createStatement().executeUpdate("UPSERT INTO " + tableName + "(A,B,C,D)
> VALUES ('A2','B2','C2','D2')");
> conn.createStatement().executeUpdate("UPSERT INTO " + tableName + "(A,B,C,D)
> VALUES ('A3','B3', 'C3', null)");
> {code}
> dump IndexMemStore:
> {code:java}
> hbase.index.covered.data.IndexMemStore(117):
> Inserting:\x01A3/0:D/1542190446218/DeleteColumn/vlen=0/seqid=0/value=
> phoenix.hbase.index.covered.data.IndexMemStore(133): Current kv state:
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:B/1542190446167/Put/vlen=2/seqid=5/value=B3
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:C/1542190446167/Put/vlen=2/seqid=5/value=C3
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:D/1542190446218/DeleteColumn/vlen=0/seqid=0/value=
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:_0/1542190446167/Put/vlen=1/seqid=5/value=x
> phoenix.hbase.index.covered.data.IndexMemStore(137): == END MemStore
> Dump ==
> {code}
>
> The DeleteColumn's timestamp larger than other mutations.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5138) ViewIndexId sequences created after PHOENIX-5132 shouldn't collide with ones created before it
[ https://issues.apache.org/jira/browse/PHOENIX-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5138: Fix Version/s: 5.0.1 > ViewIndexId sequences created after PHOENIX-5132 shouldn't collide with ones > created before it > -- > > Key: PHOENIX-5138 > URL: https://issues.apache.org/jira/browse/PHOENIX-5138 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0, 5.1 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Blocker > Fix For: 4.15.0, 5.1.0, 5.0.1 > > Attachments: PHOENIX-5138-v2.patch, PHOENIX-5138-v3.patch, > PHOENIX-5138-v4.patch, PHOENIX-5138.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > PHOENIX-5132 changed the ViewIndexId generation logic to use one sequence per > physical view index table, whereas before it had been tenant + physical > table. This removed the possibility of a tenant view index and a global view > index having colliding ViewIndexIds. > However, existing Phoenix environments may have already created tenant-owned > view index ids using the old sequence, and under PHOENIX-5132 if they create > another, its ViewIndexId will got back to MIN_VALUE, which could cause a > collision with an existing view index id. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4759) During restart RS that hosts SYSTEM.CATALOG table may get stuck
[ https://issues.apache.org/jira/browse/PHOENIX-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-4759: Fix Version/s: 5.0.1 > During restart RS that hosts SYSTEM.CATALOG table may get stuck > --- > > Key: PHOENIX-4759 > URL: https://issues.apache.org/jira/browse/PHOENIX-4759 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.0, 5.0.0 >Reporter: Romil Choksi >Assignee: Sergey Soldatov >Priority: Blocker > Fix For: 4.14.0, 5.0.0, 5.0.1 > > Attachments: PHOENIX-4759-1.patch, PHOENIX-4759-2.master.patch > > > Sometimes when a cluster has restarted the regions that belong to > SYSTEM.CATALOG and other system tables on the same RS may be stuck in RiT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5266) Client can only write on Index Table and skip data table if failure happens because of region split/move etc
[ https://issues.apache.org/jira/browse/PHOENIX-5266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5266: Fix Version/s: 5.0.1 > Client can only write on Index Table and skip data table if failure happens > because of region split/move etc > > > Key: PHOENIX-5266 > URL: https://issues.apache.org/jira/browse/PHOENIX-5266 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0, 4.14.1, 5.1.0, 4.14.2 >Reporter: Mihir Monani >Assignee: Mihir Monani >Priority: Blocker > Fix For: 4.15.0, 5.1.0, 4.14.2, 5.0.1 > > Attachments: PHOENIX-5266-4.x-HBase-1.3.01.patch, > PHOENIX-5266-4.x-HBase-1.3.02.patch, PHOENIX-5266.01.patch, > PHOENIX-5266.patch, PHOENIX-5266.patch > > > With Phoenix 4.14.1 client, There is a scenario where client would skip data > table write but do successful index table write. In this case, we should > treat it as Data loss scenario. > > Relevant code path :- > [https://github.com/apache/phoenix/blob/4.x-HBase-1.3/phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java#L994-L1043] > [https://github.com/apache/phoenix/blob/4.x-HBase-1.3/phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java#L1089-L1109] > > Here is what happens :- > * Consider below assumptions for scenario :- > ** max no row in single batch = 100 > ** max size of batch = 2 MB > * When client faces SQLException Code 1121, it sets variable > shouldRetryIndexedMutation=true. > * In scenarios where client sends batch of 100 rows only as per > configuration, but batch size is >2 MB, MutationState.java#991 will split > this 100 row batch into multiple smaller batches which are <2MB. > ** MutationState.java#991 :- > [https://github.com/apache/phoenix/blob/4.x-HBase-1.3/phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java#L991] > * Suppose there are 5 batches of 20 rows but client faces 1121 > SQLExceptionCode on 2nd batch , then it will set > shouldRetryIndexedMutation=true and it will retry all 5 batches again with > only Index updates. This will results in rows missing from Data table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5132) View indexes with different owners but of the same base table can be assigned same ViewIndexId
[ https://issues.apache.org/jira/browse/PHOENIX-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaanai updated PHOENIX-5132: Fix Version/s: 5.0.1 > View indexes with different owners but of the same base table can be assigned > same ViewIndexId > -- > > Key: PHOENIX-5132 > URL: https://issues.apache.org/jira/browse/PHOENIX-5132 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.14.1 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Critical > Fix For: 4.15.0, 5.1.0, 5.0.1 > > Attachments: PHOENIX-5132-4.x-HBase-1.4.patch, > PHOENIX-5132-4.x-HBase-1.4.v2.patch, PHOENIX-5132-repro.patch > > > All indexes on views for a particular base table are stored in the same > physical HBase table. Phoenix distinguishes them by prepending each row key > with an encoded short or long integer called a ViewIndexId. > The ViewIndexId is generated by using a sequence to guarantee that each view > index id is unique. Unfortunately, the sequence used follows a convention of > [SaltByte, Tenant, Schema, BaseTable] for its key, which means that there's a > separate sequence for each tenant that owns an index in the view index table. > (See MetaDataUtil.getViewIndexSequenceKey) Since all the sequences start at > the same value, collisions are not only possible but likely. > I've written a test that confirms the ViewIndexId collision. This means it's > very likely that query results using one view index could mistakenly include > rows from another index, but I haven't confirmed this. > All view indexes for a base table, regardless of whether globally or > tenant-owned, should use the same sequence. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PHOENIX-5113) Reason through the correct plans for salted tables and local indexes
[ https://issues.apache.org/jira/browse/PHOENIX-5113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved PHOENIX-5113. Resolution: Invalid > Reason through the correct plans for salted tables and local indexes > > > Key: PHOENIX-5113 > URL: https://issues.apache.org/jira/browse/PHOENIX-5113 > Project: Phoenix > Issue Type: Task >Reporter: Lars Hofhansl >Priority: Major > > In PHOENIX-5109 I encountered various test failures in part due to salted > tables. > Often it is not clear what the best strategy is for retained the current > behavior in PHOENIX-5109. > This Jira is to reason through the issues and assure that Phoenix always > picks the right plans (heuristically). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5112) Simplify QueryPlan selection in Phoenix.
[
https://issues.apache.org/jira/browse/PHOENIX-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated PHOENIX-5112:
---
Attachment: 5112-master.txt
> Simplify QueryPlan selection in Phoenix.
>
>
> Key: PHOENIX-5112
> URL: https://issues.apache.org/jira/browse/PHOENIX-5112
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Priority: Major
> Attachments: 5112-master.txt, 5112-master.txt
>
>
> Brought to light in part in PHOENIX-5109, plan selection in Phoenix is too
> complicated with its logic spread over multiple areas. My recent changes let
> most index through the initial filters and next step is to put all the logic
> in {{QueryOptimizer.orderPlansBestToWorst}}.
> One exception is plans with global indexes for queries with uncovered
> queries, those should still be handled in {{addPlan}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5112) Simplify QueryPlan selection in Phoenix.
[
https://issues.apache.org/jira/browse/PHOENIX-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated PHOENIX-5112:
---
Attachment: (was: 5112-master.txt)
> Simplify QueryPlan selection in Phoenix.
>
>
> Key: PHOENIX-5112
> URL: https://issues.apache.org/jira/browse/PHOENIX-5112
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Priority: Major
> Attachments: 5112-master.txt
>
>
> Brought to light in part in PHOENIX-5109, plan selection in Phoenix is too
> complicated with its logic spread over multiple areas. My recent changes let
> most index through the initial filters and next step is to put all the logic
> in {{QueryOptimizer.orderPlansBestToWorst}}.
> One exception is plans with global indexes for queries with uncovered
> queries, those should still be handled in {{addPlan}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5112) Simplify QueryPlan selection in Phoenix.
[
https://issues.apache.org/jira/browse/PHOENIX-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated PHOENIX-5112:
---
Attachment: (was: 5112-master.txt)
> Simplify QueryPlan selection in Phoenix.
>
>
> Key: PHOENIX-5112
> URL: https://issues.apache.org/jira/browse/PHOENIX-5112
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Priority: Major
> Attachments: 5112-master.txt
>
>
> Brought to light in part in PHOENIX-5109, plan selection in Phoenix is too
> complicated with its logic spread over multiple areas. My recent changes let
> most index through the initial filters and next step is to put all the logic
> in {{QueryOptimizer.orderPlansBestToWorst}}.
> One exception is plans with global indexes for queries with uncovered
> queries, those should still be handled in {{addPlan}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (PHOENIX-5225) Update Omid to 1.0.1
[ https://issues.apache.org/jira/browse/PHOENIX-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl reassigned PHOENIX-5225: -- Assignee: Lars Hofhansl > Update Omid to 1.0.1 > > > Key: PHOENIX-5225 > URL: https://issues.apache.org/jira/browse/PHOENIX-5225 > Project: Phoenix > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Major > Fix For: 4.15.0, 5.1.0 > > Attachments: 5225-master.txt, 5225-v2.txt, 5225.txt, 5225_v3.txt > > > As soon as Omid 1.0.1 is officially released we should upgrade to that. > See PHOENIX-5082 for the various issues fixed in it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PHOENIX-5255) Create Orchestrator for PQS using PhoenixCanaryTool in phoenix-queryserver project
[ https://issues.apache.org/jira/browse/PHOENIX-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaroopa Kadam resolved PHOENIX-5255. - Resolution: Resolved Merged in phoenix-queryserver git repo. > Create Orchestrator for PQS using PhoenixCanaryTool in phoenix-queryserver > project > -- > > Key: PHOENIX-5255 > URL: https://issues.apache.org/jira/browse/PHOENIX-5255 > Project: Phoenix > Issue Type: Improvement >Reporter: Swaroopa Kadam >Assignee: Swaroopa Kadam >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > This orchestrator will execute PhoenixCanaryTool at every configured interval > and execute UPSERT/READ via PQS client. > This is mainly to demo/exemplify usability of PQS -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-5288) Add ITs to phoenix-queryserver repo for new canary-orchestrator
Swaroopa Kadam created PHOENIX-5288: --- Summary: Add ITs to phoenix-queryserver repo for new canary-orchestrator Key: PHOENIX-5288 URL: https://issues.apache.org/jira/browse/PHOENIX-5288 Project: Phoenix Issue Type: Improvement Reporter: Swaroopa Kadam Use the mini cluster and running queryserver to write ITs. Especially simulate an env where more than one host runs the orchestrator to strongly handle curator related edge cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Handling of HBase version specifics.
+1, I think this is a good idea. This would make it easier to contribute and commit since you would only have to create a single patch. The tests would take longer to run (1.3, 1.4, 1.5 and 2.x). We should make sure our precommit build will run the tests for all the modules. On Fri, May 17, 2019 at 11:23 AM la...@apache.org wrote: > Hi all, > historically we have a branch of each version of HBase we want to > support.As a result we have many branches, committing is a hassle and it is > easy to miss a change across branches. > Instead we could have a maven module per version of HBase we want to > support and move the version dependent code there.Take a look at what > Tephra is doing: https://github.com/apache/incubator-tephra > They have a compat module for each supported version of HBase, and version > dependent code is "simply" copied into those modules.There's still > duplicate code, but at least there's one branch to maintain. > It's somewhat of a bigger project now. > Thoughts? > -- Lars >
[VOTE] Release of Apache Phoenix 4.14.2 RC1
Hello Everyone, This is a call for a vote on Apache Phoenix 4.14.2 RC1. This is a patch release of Phoenix 4.14 and is compatible with Apache HBase 1.3 and 1.4. The release includes both a source-only release and a convenience binary release for each supported HBase version. This release has feature parity with supported HBase versions and includes several critical bug fixes. The source tarball, including signatures, digests, etc can be found at: https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.3-rc1/src/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.4-rc1/src/ The binary artifacts can be found at: https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.3-rc1/bin/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.4-rc1/bin/ For a complete list of changes, see: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12344379 Artifacts are signed with my "CODE SIGNING KEY": DFD86C02 KEYS file available here: https://dist.apache.org/repos/dist/dev/phoenix/KEYS The hash and tag to be voted upon: https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=e3505d91e46de1a1756a145d396f27a3c70e927f https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=4b717825fb284c1f9fbdeee3d0e5391f5baf13bb Vote will be open for at least 72 hours. Please vote: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, The Apache Phoenix Team
