[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462931#comment-16462931
]
Ankit Singhal commented on PHOENIX-3176:
Yes [~jamestaylor] , it is now fixed. Thanks for the ping.
Do you know from which version we started using latest timestamp? so that I
marked this Jira closed accordingly.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
>Priority: Major
> Labels: newbie
> Fix For: 4.15.0
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462782#comment-16462782
]
James Taylor commented on PHOENIX-3176:
---
Ping [~an...@apache.org]?
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
>Priority: Major
> Labels: newbie
> Fix For: 4.14.0
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259394#comment-16259394
]
James Taylor commented on PHOENIX-3176:
---
We use latest time stamp from the client now, so I assumed this was fixed too.
Did you try it, [~an...@apache.org]?
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.14.0
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259140#comment-16259140
]
Ankit Singhal commented on PHOENIX-3176:
[~giacomotaylor], this doesn't seem to be fixed. Commit in the last messages
seems unrelated to this Jira.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623519#comment-15623519
]
Hadoop QA commented on PHOENIX-3176:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12823239/PHOENIX-3176.patch
against master branch at commit 2d98be930234277291626b71cb6e7cf1004762f4.
ATTACHMENT ID: 12823239
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail .
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithRowTimestamp.java:[35,44]
cannot find symbol
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithRowTimestamp.java:[40,9]
cannot find symbol
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithRowTimestamp.java:[49,60]
cannot find symbol
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.0:testCompile
(default-testCompile) on project phoenix-core: Compilation failure: Compilation
failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithRowTimestamp.java:[35,44]
cannot find symbol
[ERROR] symbol: class BaseOwnClusterHBaseManagedTimeIT
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithRowTimestamp.java:[40,9]
cannot find symbol
[ERROR] symbol: method setUpTestDriver(org.apache.phoenix.util.ReadOnlyProps)
[ERROR] location: class org.apache.phoenix.end2end.QueryWithRowTimestamp
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithRowTimestamp.java:[49,60]
cannot find symbol
[ERROR] symbol: method getUrl()
[ERROR] location: class org.apache.phoenix.end2end.QueryWithRowTimestamp
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :phoenix-core
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/657//console
This message is automatically generated.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.10.0
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
>
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482547#comment-15482547
]
Hudson commented on PHOENIX-3176:
-
FAILURE: Integrated in Jenkins build Phoenix-master #1392 (See
[https://builds.apache.org/job/Phoenix-master/1392/])
PHOENIX-3176 Remove unnecessary PQS proxy user authorization (elserj: rev
c3cdb2afbe60a4959237a6ac264f42322a049d1d)
* (edit)
phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/Main.java
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475861#comment-15475861
]
Lars Hofhansl commented on PHOENIX-3176:
bq. This would be a pretty fundamental change, so we need to be careful with
it.
Let's not do that for 4.8.x then.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418116#comment-15418116
]
Samarth Jain commented on PHOENIX-3176:
---
Also, with Ankit's patch, the test
UpsertSelectAutoCommitIT#testUpsertSelectDoesntSeeUpsertedData passes.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418185#comment-15418185
]
James Taylor commented on PHOENIX-3176:
---
Looks like the RENEW_LEASE feature is impacting this. If that feature is
disabled, an infinite loop occurs as it did before when running
UpsertSelectAutoCommitIT#testUpsertSelectDoesntSeeUpsertedData. This statement
won't actually run on the server because of the usage of the sequence. I think
it's working now because we have a single scan running instead of the separate
scans per chunk as before. It'd likely still be a problem is a split occurs
while the scan is running, though. It might not cause an infinite loop, but if
the select starts seeing the rows from the upsert, the results will be wrong.
How about adapting that test to force a split while it's running,
[~an...@apache.org] or [~samarthjain]?
This would be a pretty fundamental change, so we need to be careful with it.
Also, if I change that test to not auto commit, but instead commit after the
upsert statements, it fails. That may just mean the count we're returning for
the UPSERT call is wrong, but that'd be a bug too. I filed PHOENIX-3178 for
that. Do you see anything wrong with that test, [~samarthjain]?
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418114#comment-15418114
]
Samarth Jain commented on PHOENIX-3176:
---
Wrote a test with ROW_TIMESTAMP column following
UpsertSelectAutoCommitIT#testUpsertSelectDoesntSeeUpsertedData. The test passes
with and without the patch. The time range on the scan doesn't take into
account the time at which the table was resolved.
Here is the test that I wrote:
@Test
public void testUpsertSelectDoesntSeeUpsertedDataWithRowTimestampColumn()
throws Exception {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
props.setProperty(QueryServices.MUTATE_BATCH_SIZE_ATTRIB,
Integer.toString(3));
props.setProperty(QueryServices.SCAN_CACHE_SIZE_ATTRIB,
Integer.toString(3));
props.setProperty(QueryServices.SCAN_RESULT_CHUNK_SIZE,
Integer.toString(3));
Connection conn = DriverManager.getConnection(getUrl(), props);
conn.setAutoCommit(true);
conn.createStatement().execute("CREATE SEQUENCE keys");
String tableName = generateRandomString();
conn.createStatement().execute(
"CREATE TABLE " + tableName + " (pk BIGINT PRIMARY KEY
ROW_TIMESTAMP, val INTEGER)");
conn.createStatement().execute(
"UPSERT INTO " + tableName + " VALUES (NEXT VALUE FOR keys,1)");
for (int i=0; i<6; i++) {
Statement stmt = conn.createStatement();
int upsertCount = stmt.executeUpdate(
"UPSERT INTO " + tableName + " SELECT NEXT VALUE FOR keys, val
FROM " + tableName);
assertEquals((int)Math.pow(2, i), upsertCount);
}
conn.close();
}
{code}
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417896#comment-15417896
]
James Taylor commented on PHOENIX-3176:
---
Those UpsertSelectIT test supply a CURRENT_SCN. We need tests that don't. Also,
this question:
bq. Does the time range on the scan already take into account the time at which
the table was resolved?
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417888#comment-15417888
]
Samarth Jain commented on PHOENIX-3176:
---
We have tests that do UPSERT SELECT with row_timestamp. UpsertSelectIT class
has testUpsertSelectWithRowtimeStampColumn among a few others.
[~an...@apache.org] - did all IT tests pass successfully for you?
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417879#comment-15417879
]
James Taylor commented on PHOENIX-3176:
---
We may need to advise against forward dating rows using the ROW_TIMESTAMP
feature given the correlation between the Cell time stamp and the way in which
Phoenix works (showing you the rows that exist as of when the query is
compiled). FWIW, a workaround for users is to connect with a CURRENT_SCN of
Long.MAX_VALUE.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417866#comment-15417866
]
James Taylor commented on PHOENIX-3176:
---
It's important to run the scan as of the time stamp from which the table was
resolved so that we have a consistent time across all our parallel scans. One
example where we depend on this is when an UPSERT SELECT is done on the same
table. We ensure that we don't see the new rows being inserted so that we don't
get into an infinite loop. See the
UpsertSelectAutoCommitIT.testUpsertSelectDoesntSeeUpsertedData(). Does the time
range on the scan already take into account the time at which the table was
resolved? It'd be interesting to have another similar test on a table using the
ROW_TIMESTAMP feature. Maybe we need a special case for this feature (which
would be a shame)? If so, we'd still need to handle the UPSERT SELECT case.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column
[
https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417815#comment-15417815
]
Samarth Jain commented on PHOENIX-3176:
---
Patch looks good to me, [~an...@apache.org]. [~jamestaylor], would be good to
get your keen eye on it too.
I would move the test that you have added to UpsertValuesIT as this is where
the other upsert tests for row_timestamp column were added.
> Rows will be skipped which are having future timestamp in row_timestamp column
> --
>
> Key: PHOENIX-3176
> URL: https://issues.apache.org/jira/browse/PHOENIX-3176
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
>Reporter: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts
> row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data
> values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +--+---+--+--+
> | ASSETID | METRICID |TS| VAL |
> +--+---+--+--+
> | 1| 2 | 2015-01-01 00:00:00.000 | 1.2 |
> +--+---+--+--+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +---+
> | COUNT(1) |
> +---+
> | 1 |
> +---+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA |
> | ROW TIMESTAMP FILTER [0, 1470901929982) |
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO SINGLE ROW |
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
