[jira] [Commented] (PHOENIX-914) Native HBase timestamp support to optimize date range queries in Phoenix
[
https://issues.apache.org/jira/browse/PHOENIX-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578387#comment-14578387
]
Vladimir Rodionov commented on PHOENIX-914:
---
Would it help if we could write custom meta-data to HFile on write
(flush/compaction) and read this meta-data on RegionScanner open? By installing
custom hook (observer) Phoenix could annotate every HFile and then use this
annotation later during open scanner operation. If we have query context (date
range) we can easily filter unneeded store files out.
> Native HBase timestamp support to optimize date range queries in Phoenix
> -
>
> Key: PHOENIX-914
> URL: https://issues.apache.org/jira/browse/PHOENIX-914
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Vladimir Rodionov
> Attachments: PHOENIX-914.patch
>
>
> For many applications one of the column of a table can be (and must be)
> naturally mapped
> to HBase timestamp. What it gives us is the optimization on StoreScanner
> where HFiles with timestamps out of range of
> a Scan operator will be omitted. Let us say that we have time-series type of
> data (EVENTS) and custom compaction, where we create
> series of HFiles with continuous non-overlapping timestamp ranges.
> CREATE TABLE IF NOT EXISTS ODS.EVENTS (
> METRICID VARCHAR NOT NULL,
> METRICNAME VARCHAR,
> SERVICENAME VARCHAR NOT NULL,
> ORIGIN VARCHAR NOT NULL,
> APPID VARCHAR,
> IPID VARCHAR,
> NVALUE DOUBLE,
> TIME TIMESTAMP NOT NULL /+ TIMESTAMP +/,
> DATA VARCHAR,
> SVALUE VARCHAR
> CONSTRAINT PK PRIMARY KEY (METRICID, SERVICENAME, ORIGIN, APPID, IPID,
> TIME)
> ) SALT_BUCKETS=40, IMMUTABLE_ROWS=true,VERSIONS=1,DATA_BLOCK_ENCODING='NONE';
> Make note on TIME TIMESTAMP NOT NULL /+ TIMESTAMP +/ - this is the Hint to
> Phoenix that the column
> TIME must be mapped to HBase timestamp.
> The Query:
> Select all events of type 'X' for last 7 days
> SELECT * from EVENTS WHERE METRICID = 'X' and TIME < NOW() and TIME > NOW() -
> 7*24*360; (this may be not correct SQL syntax of course)
> These types of queries will be efficiently optimized if:
> 1. Phoenix maps TIME column to HBase timestamp
> 2. Phoenix smart enough to map WHERE clause on TIME attribute to Scan
> timerange
> Although this :
> Properties props = new Properties();
> props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts));
> Connection conn = DriverManager.connect(myUrl, props);
> conn.createStatement().execute("UPSERT INTO myTable VALUES ('a')");
> conn.commit();
> will work in my case- it may not be efficient from performance point of view
> because for every INSERT/UPSERT
> new Connection object and new Statement is created, beside this we still need
> the optimization 2. (see above).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Reminder: 1.2 branching
Hi folks! If you happen to have any open issues with HBase that require a minor version bump, the 1.2 branch is planned to be made in a week. This is a general heads-up; I'll be going through open tickets targeting the release this week and checking on specific statuses. -- Sean On Jun 8, 2015 7:55 PM, "Nick Dimiduk" wrote: > You may want to give a heads-up on user@ and phoenix lists as well so that > folks have a chance to push their open tickets. > > On Mon, Jun 8, 2015 at 5:32 PM, Sean Busbey wrote: > > > Hi all! > > > > Just a quick reminder that the plan is to branch 1.2 on or about Monday > > June 15th, which is a week from today. > > > > Any one have concerns? If we considered the branch feature frozen, would > > that work for everyone? > > > > -- > > Sean > > >
[jira] [Commented] (PHOENIX-1987) SIGN built-in function should be order preserving
[
https://issues.apache.org/jira/browse/PHOENIX-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578129#comment-14578129
]
Shuxiong Ye commented on PHOENIX-1987:
--
Hi [~jamestaylor],
Patch [1] adds tests in QueryCompilerTest to check optimization about ORDER BY.
Please help me to review it.
I have a question in my previous comment. Please give me some hints about it. :
)
[1]
0001-PHOENIX-1987-fixing-getKeyFormationTraversalIndex-in-SIGN-function.patch
> SIGN built-in function should be order preserving
> -
>
> Key: PHOENIX-1987
> URL: https://issues.apache.org/jira/browse/PHOENIX-1987
> Project: Phoenix
> Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Shuxiong Ye
> Fix For: 5.0.0, 4.5.0, 4.4.1
>
> Attachments:
> 0001-PHOENIX-1987-SIGN-built-in-function-should-be-order_v2.patch,
> 0001-PHOENIX-1987-fixing-getKeyFormationTraversalIndex-in-SIGN-function.patch
>
>
> The SIGN built-in function preserves the order of its input. This needs to be
> explicitly implemented in SignFunction by implementing the following method:
> {code}
> public OrderPreserving preservesOrder() {
> return OrderPreserving.YES;
> }
> {code}
> This will allow certain optimizations reducing the amount of memory and/or
> buffering that is required if SIGN is used in a GROUP BY or ORDER BY clause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-2029) Queries are making two rpc calls for getTable
[ https://issues.apache.org/jira/browse/PHOENIX-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-2029: Attachment: PHOENIX-2029.patch Test that repros the issue. > Queries are making two rpc calls for getTable > -- > > Key: PHOENIX-2029 > URL: https://issues.apache.org/jira/browse/PHOENIX-2029 > Project: Phoenix > Issue Type: Bug >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva > Attachments: PHOENIX-2029.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2029) Queries are making two rpc calls for getTable
Thomas D'Silva created PHOENIX-2029: --- Summary: Queries are making two rpc calls for getTable Key: PHOENIX-2029 URL: https://issues.apache.org/jira/browse/PHOENIX-2029 Project: Phoenix Issue Type: Bug Reporter: Thomas D'Silva Assignee: Thomas D'Silva -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2027) Subqueries with no data are raising IllegalStateException
[ https://issues.apache.org/jira/browse/PHOENIX-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577107#comment-14577107 ] Rajeshbabu Chintaguntla commented on PHOENIX-2027: -- +1. Will commit soon. > Subqueries with no data are raising IllegalStateException > - > > Key: PHOENIX-2027 > URL: https://issues.apache.org/jira/browse/PHOENIX-2027 > Project: Phoenix > Issue Type: Bug >Reporter: Alicia Ying Shu >Assignee: Alicia Ying Shu > Fix For: 5.0.0, 4.5.0, 4.4.1 > > Attachments: Phoenix-2027-v1.patch, Phoenix-2027.patch > > > SELECT /*+ USE_SORT_MERGE_JOIN */ * FROM (SELECT ID, BIG_ID, DATE_ID FROM > LARGE_TABLE AS A WHERE (A.ID % 5) = 0) AS A INNER JOIN (SELECT SEC_ID, > SEC_TINY_ID, SEC_UNSIG_FLOAT_ID FROM SECONDARY_LARGE_TABLE AS B WHERE > (B.SEC_ID % 5) = 0) AS B ON A.ID=B.SEC_ID WHERE A.DATE_ID > ALL (SELECT > SEC_DATE_ID FROM SECONDARY_LARGE_TABLE LIMIT 100) AND > B.SEC_UNSIG_FLOAT_ID = ANY (SELECT sec_unsig_float_id > FROM SECONDARY_LARGE_TABLE > WHERE SEC_ID > ALL (SELECT MIN (ID) FROM GRAMMAR_TABLE WHERE UNSIG_ID IS > NULL) AND SEC_UNSIG_ID < ANY > (SELECT DISTINCT(UNSIG_ID) FROM LARGE_TABLE WHERE UNSIG_ID<2500) LIMIT 1000) > AND A.ID < 1; > java.lang.IllegalStateException: Uninitialiized. Null constructor called w/o > accompaying readFields invocation > at > org.apache.hadoop.hbase.io.ImmutableBytesWritable.get(ImmutableBytesWritable.java:96) > at > org.apache.phoenix.execute.SortMergeJoinPlan$JoinKey.set(SortMergeJoinPlan.java:503) > at > org.apache.phoenix.execute.SortMergeJoinPlan$BasicJoinIterator.advance(SortMergeJoinPlan.java:337) > at > org.apache.phoenix.execute.SortMergeJoinPlan$BasicJoinIterator.init(SortMergeJoinPlan.java:325) > at > org.apache.phoenix.execute.SortMergeJoinPlan$BasicJoinIterator.next(SortMergeJoinPlan.java:256) > at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:757) > at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62) > at sqlline.AbstractOutputFormat.print(AbstractOutputFormat.java:30) > at sqlline.SqlLine.print(SqlLine.java:1653) > at sqlline.Commands.execute(Commands.java:833) > at sqlline.Commands.sql(Commands.java:732) > at sqlline.SqlLine.dispatch(SqlLine.java:808) > at sqlline.SqlLine.runCommands(SqlLine.java:1711) > at sqlline.Commands.run(Commands.java:1285) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36) > at sqlline.SqlLine.dispatch(SqlLine.java:804) > at sqlline.SqlLine.initArgs(SqlLine.java:613) > at sqlline.SqlLine.begin(SqlLine.java:656) > at sqlline.SqlLine.start(SqlLine.java:398) > at sqlline.SqlLine.main(SqlLine.java:292) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information
Hi All, Basically what type of use cases are you expecting or performing at the moment with regard to tracing? For example these are the use cases I'm planing. 1. Searching by parent id / trace id / description (regx search) 2. Grouping and ordering the tracing information by time period. 3. Counting the trace count per day / hour. 4. Comparing and distinguishing two sets of tracing. Thanks. On Mon, Jun 8, 2015 at 4:00 PM, Nishani (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ] > > Nishani updated PHOENIX-1118: > -- > Attachment: Screenshot of dependency tree.png > > Attaching the dependency tree on tracing. > Pull request can be found here. > https://github.com/AyolaJayamaha/TracingWebApp/pull/1 > > > Provide a tool for visualizing Phoenix tracing information > > -- > > > > Key: PHOENIX-1118 > > URL: https://issues.apache.org/jira/browse/PHOENIX-1118 > > Project: Phoenix > > Issue Type: Sub-task > >Reporter: James Taylor > >Assignee: Nishani > > Labels: Java, SQL, Visualization, gsoc2015, mentor > > Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, > MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of > dependency tree.png, screenshot of tracing web app.png > > > > > > Currently there's no means of visualizing the trace information provided > by Phoenix. We should provide some simple charting over our metrics tables. > Take a look at the following JIRA for sample queries: > https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151 > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) > -- Best Regards, Nishani Jayamaha http://ayolajayamaha.blogspot.com/
[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information
[ https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishani updated PHOENIX-1118: -- Attachment: Screenshot of dependency tree.png Attaching the dependency tree on tracing. Pull request can be found here. https://github.com/AyolaJayamaha/TracingWebApp/pull/1 > Provide a tool for visualizing Phoenix tracing information > -- > > Key: PHOENIX-1118 > URL: https://issues.apache.org/jira/browse/PHOENIX-1118 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Nishani > Labels: Java, SQL, Visualization, gsoc2015, mentor > Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, > MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency > tree.png, screenshot of tracing web app.png > > > Currently there's no means of visualizing the trace information provided by > Phoenix. We should provide some simple charting over our metrics tables. Take > a look at the following JIRA for sample queries: > https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
