[jira] [Commented] (PHOENIX-3224) Observations from large scale testing.
[ https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15450118#comment-15450118 ] Lars Hofhansl commented on PHOENIX-3224: [~giacomotaylor], [~mujtabachohan], FYI. > Observations from large scale testing. > -- > > Key: PHOENIX-3224 > URL: https://issues.apache.org/jira/browse/PHOENIX-3224 > Project: Phoenix > Issue Type: Task >Reporter: Lars Hofhansl > > We have a >1000 node physical cluster at our disposal for a short time, > before it'll be handed off to its intended use. > Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of > queries. Most tables are between 100G and 500G (uncompressed) and between > 600m and 2bn rows. > The good news is that many things just worked. We sorted > 400G is < 5s with > HBase and Phoenix. Scans work. Joins work (as long as one side is kept under > 1m rows or so). > For the issues we observers I'll file sub jiras under this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3224) Observations from large scale testing.
[ https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15450454#comment-15450454 ] James Taylor commented on PHOENIX-3224: --- Would be good to get more info on bigger joins. Using the USE_SORT_MERGE_JOIN hint, you should be able to do larger joins and perf would be equivalent to doing a sort on each of the tables being joined. Can we try some smaller tests first with this hint to make sure there's not something else going on? Please let us know if we're missing anything, [~maryannxue]. > Observations from large scale testing. > -- > > Key: PHOENIX-3224 > URL: https://issues.apache.org/jira/browse/PHOENIX-3224 > Project: Phoenix > Issue Type: Task >Reporter: Lars Hofhansl > > We have a >1000 node physical cluster at our disposal for a short time, > before it'll be handed off to its intended use. > Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of > queries. Most tables are between 100G and 500G (uncompressed) and between > 600m and 2bn rows. > The good news is that many things just worked. We sorted > 400G is < 5s with > HBase and Phoenix. Scans work. Joins work (as long as one side is kept under > 1m rows or so). > For the issues we observers I'll file sub jiras under this. > I'm going to write a lob post about this and attach a link here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3224) Observations from large scale testing.
[ https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452966#comment-15452966 ] Lars Hofhansl commented on PHOENIX-3224: The issue with that is that both sides need to be brought back to the client for the merge, right? In this case this was close 400GB each 800GB in total, which in this case would take over 10 minutes even with full 10ge (or over 1.5h with 1ge). The query was a COUNT over a self-join on a large table. > Observations from large scale testing. > -- > > Key: PHOENIX-3224 > URL: https://issues.apache.org/jira/browse/PHOENIX-3224 > Project: Phoenix > Issue Type: Task >Reporter: Lars Hofhansl > > We have a >1000 node physical cluster at our disposal for a short time, > before it'll be handed off to its intended use. > Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of > queries. Most tables are between 100G and 500G (uncompressed) and between > 600m and 2bn rows. > The good news is that many things just worked. We sorted > 400G is < 5s with > HBase and Phoenix. Scans work. Joins work (as long as one side is kept under > 1m rows or so). > For the issues we observers I'll file sub jiras under this. > I'm going to write a lob post about this and attach a link here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3224) Observations from large scale testing.
[ https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453323#comment-15453323 ] Maryann Xue commented on PHOENIX-3224: -- The sort-merge join takes inputs from both sides of the join in a streaming fashion, so the performance should be close to sorting both tables. There are chances though that sort merge join would have to cache a lot of data on the client side and perform really badly. It is when there are a big amount of rows from both sides that have the same join keys. In that case there will be caching and backtracking to cross join all those rows. > Observations from large scale testing. > -- > > Key: PHOENIX-3224 > URL: https://issues.apache.org/jira/browse/PHOENIX-3224 > Project: Phoenix > Issue Type: Task >Reporter: Lars Hofhansl > > We have a >1000 node physical cluster at our disposal for a short time, > before it'll be handed off to its intended use. > Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of > queries. Most tables are between 100G and 500G (uncompressed) and between > 600m and 2bn rows. > The good news is that many things just worked. We sorted > 400G is < 5s with > HBase and Phoenix. Scans work. Joins work (as long as one side is kept under > 1m rows or so). > For the issues we observers I'll file sub jiras under this. > I'm going to write a lob post about this and attach a link here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3224) Observations from large scale testing.
[ https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453854#comment-15453854 ] James Taylor commented on PHOENIX-3224: --- So in theory, a big join is similar to an ORDER BY, in that there'd be a pause while sorting is being done on both sides, but then you should start getting results flowing back to the client after that. If you do an aggregation, though, like a count( * ) on the joined results, then you wouldn't get an answer back until the client has seen all of the rows (which will take a long time as you mentioned, Lars). I think Phoenix would need a kind of shuffle step in it's processing model to handle this better. > Observations from large scale testing. > -- > > Key: PHOENIX-3224 > URL: https://issues.apache.org/jira/browse/PHOENIX-3224 > Project: Phoenix > Issue Type: Task >Reporter: Lars Hofhansl > > We have a >1000 node physical cluster at our disposal for a short time, > before it'll be handed off to its intended use. > Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of > queries. Most tables are between 100G and 500G (uncompressed) and between > 600m and 2bn rows. > The good news is that many things just worked. We sorted > 400G is < 5s with > HBase and Phoenix. Scans work. Joins work (as long as one side is kept under > 1m rows or so). > For the issues we observers I'll file sub jiras under this. > I'm going to write a lob post about this and attach a link here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
