[
https://issues.apache.org/jira/browse/PHOENIX-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16020010#comment-16020010
]
James Taylor commented on PHOENIX-3853:
---------------------------------------
[~lhofhansl] - see the very nice graph that Mujtaba put together here:
https://issues.apache.org/jira/browse/PHOENIX-3853?focusedCommentId=16014642&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16014642
I suspected this was due to the parallelization we do for writes for non local
indexes (as local index writes would all occur in the caller thread with
PHOENIX-3827), but it appears this is not the cause (as I gave Mujtaba a patch
that turns off parallelization for the non local case and he's still seeing the
same results as with that graph). Do you see anything in HBASE-15600 that would
account for this degradation? Do we care, since our batch sizes will be in the
1000 range?
> Local Index - Writes to local index are twice as slow as global and get
> exponentially slower with PHOENIX-3827_v2 patch
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3853
> URL: https://issues.apache.org/jira/browse/PHOENIX-3853
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.10.0
> Environment: HBase 1.3.1 4GB heap in local mode
> Reporter: Mujtaba Chohan
> Fix For: 4.11.0
>
> Attachments: after_3858.png, batch_mutable.png, batch_time.png,
> PHOENIX-3853_test_only.patch
>
>
> HBase 1.3.1 with head of Phoenix 4.x with/without PHOENIX-3827 v2 patch
> applied. This is with immutable non-covered local/global index on a single
> column with varying batch size when writing data to base table plus index.
> !batch_time.png!
> | Batch Size | Local Index with PHOENIX-3827_v2 patch (sec)| Local Index
> without PHOENIX-3827_v2.patch (sec)| Global (sec)|
> | 100 | 0.02 | 0.03 | 0.013 |
> | 1000 | 0.3 | 0.3 | 0.13 |
> | 10000 | 4.3 | 2.6 | 1.3 |
> | 12500 | 8.1 | 3 | 1.6 |
> | 15000 | 13.3 | 3.1 | 1.9 |
> Schema and index
> {noformat}
> CREATE TABLE IF NOT EXISTS T (OID CHAR(15) NOT NULL, PKP CHAR(3) NOT NULL,
> PIH CHAR(15) NOT NULL, FD DATE NOT NULL, SB CHAR(15) NOT NULL, BJ CHAR(15),
> JR VARCHAR, FIELD VARCHAR, YM VARCHAR, WN VARCHAR, LG VARCHAR, XHJ VARCHAR,
> HF VARCHAR, GA VARCHAR, MX VARCHAR, NZ DECIMAL, JV DECIMAL, AG DATE, KV DATE,
> JK VARCHAR, DK VARCHAR, EU DATE, OE VARCHAR, DV INTEGER, IK VARCHAR
> CONSTRAINT PK PRIMARY KEY ( OID, PKP, PIH, FD DESC, SB ))
> VERSIONS=1,IMMUTABLE_ROWS=true
> CREATE INDEX IF NOT EXISTS IDXT ON T (JV)
> {noformat}
> Data CSV
> https://expirebox.com/download/1cea73af1831b5193f0539d6e3442292.html
> [~rajeshbabu], [~lhofhansl], [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
