[
https://issues.apache.org/jira/browse/PHOENIX-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349982#comment-14349982
]
James Taylor commented on PHOENIX-1674:
---------------------------------------
Thanks, [~ghelmling] - agree, this is a good discussion.
bq. Ultimately, I think it's best to ensure that HBase has the feature we need
so that we don't have to implement our own delete logic. I think this "only"
requires two issues
I think those should be implemented first. Going down the path of a custom
delete marker is going to be really ugly. Plus, then we have to support it
going forward (at least until we know that any major compaction ran, I guess
:-) ). Do you think it'd be feasible to attempt that? Maybe you and
[~lhofhansl] could collaborate?
bq. How do you think this would be expressed in Phoenix? Would you add support
for START TRANSACTION / BEGIN / COMMIT statements? Absent those, each statement
runs in its own transaction?
In Phoenix, we wouldn't need to introduce any new APIs yet - we'd just rely on
standard JDBC semantics. The start of a transaction is implicit: it's started
on a connection when the first statement is run (or the first statement after a
commit/rollback). The end is explicit: it ends when conn.commit() or
conn.rollback() is called.
If auto commit is ON, then each statement runs in it's own transaction (as
there's an implicit commit at the end of a statement).
bq. I think TransactionVisibilityFilter still needs to come first so that no
data from invalid transactions is visible and that delete handling works
correctly. ... So I don't think we would lose any efficiency here.
Functionally I think it's the same to have the SkipScanFilter and
TransactionVisibilityFilter in either order, but I think we'd take a perf hit
under some cases if SkipScanFilter isn't first. Phoenix doesn't use the HBase
Get for point lookups, but uses the SkipScanFilter instead. If we're seeking
over a billion rows and only returning a few rows (which would be a few seeks
from the SkipScanFilter), running the TransactionVisibilityFilter over every
row is going to be more expensive than running it over only the rows that pass
through the SkipScanFilter.
bq. Could the query to the SYSTEM.CATALOG table just be done in the same
transaction so it's using the same snapshot for the schema state?
We have a client-side cache of table metadata: its columns/types and its
secondary indexes with a kind of object representation of the metadata (PTable,
PColumn) that we use during the compilation and optimization process. We update
the cache as the first step of validating the query (i.e. making sure all
referenced columns exist, types match up, etc.). This is the point at which we
ping the server to find out if the table metadata is up to date (we go through
an endpoint coprocessor with caching on the server side for the PTable as
well). We piggyback on this by sending back the current time from the server
side. This time becomes the upper bound of the time range for the parallel
scans we run (so that they see the same snapshot). Also, when we run an UPSERT
SELECT call (http://phoenix.apache.org/language/index.html#upsert_select), the
same timestamp becomes the timestamp we use for the Puts. This is important in
particular when you SELECT from the same table that you're doing the UPSERT
into, as otherwise you can get in an infinite loop by seeing what you're
upserting.
That's great about the ability to read your own writes. Another operation we'd
like to support is proper INSERT and UPDATE semantics (rather than just
UPSERT). With transactions, we can support this by doing checkAndPut operations
(say for a batch of rows being inserted in the same txn). In this case, we'd
rollback the transaction if any of the checkAndPuts failed. Do you think it's
feasible to support checkAndPut in Tephra?
> Snapshot isolation transaction support through Tephra
> -----------------------------------------------------
>
> Key: PHOENIX-1674
> URL: https://issues.apache.org/jira/browse/PHOENIX-1674
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: James Taylor
>
> Tephra (http://tephra.io/ and https://github.com/caskdata/tephra) is one
> option for getting transaction support in Phoenix. Let's use this JIRA to
> discuss the way in which this could be integrated along with the pros and
> cons.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
