[
https://issues.apache.org/jira/browse/PHOENIX-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabriel Reid resolved PHOENIX-570.
----------------------------------
Resolution: Fixed
Bulk resolve of closed issues imported from GitHub. This status was reached by
first re-opening all closed imported issues and then resolving them in bulk.
> Parallelizing writes to the index tables.
> -----------------------------------------
>
> Key: PHOENIX-570
> URL: https://issues.apache.org/jira/browse/PHOENIX-570
> Project: Phoenix
> Issue Type: Task
> Reporter: Jason Yates
>
> This is the last, likely, major source of 'slowness' in indexing. It actually
> slows the existing tests down a bit in the default implementation as we start
> with no threads and have to spin some up just for each index batch, creating
> more overhead. However, for more sustained write-load (especially to a
> consistent set of index tables), we should see dramatic speedup (and we can
> roll it back/fix it, if not :)).
> We basically add a thread pool to which we submit write tasks for each index
> table
> in the batch. Each batch operates independently and either returns success or
> throws
> an exception. If any write throws an exception, we will need to replay that
> entire batch
> (at least in the current implementation), so we attempt to stop all pending
> index writes
> as soon as possible.
> Also added a CachingHTableFactory. Its a very simple caching algorithm - it
> just uses
> the JVM GC to release HTables when they are not used. This lets us save the
> effort of
> setting up and tearing down the table for each batch of index writes,
> potentially a huge gain
> over time.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
