[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522221#comment-15522221
]
Kevin Liew edited comment on PHOENIX-476 at 9/26/16 6:43 AM:
-------------------------------------------------------------
Thanks for the feedback guys. I investigated some more and I think I understand
the issue more clearly now.
We’ve set (in MetadataClient) tables with default value columns to store nulls
in HBase and we can detect whether a cell is null or default in
`PRowImpl.setValue`. Next, we’d need to communicate that from the coprocessor
to the driver but the rows serialized on the coprocessors contain only the
non-null values followed by a bitmask indicating set/null values. Currently
both nulls and default values are indicated as nulls in the bitmask.
We can’t change the value bitmask to indicate that default values are set and
not null, because that would affect the size of the expected bytes and the
offsets used for deserialization at the driver.
I think that given the constraints that we have now, we might need to serialize
another bitmask indicating default values.
Does that sound like a good solution?
I think we’d run into the same constraints after migrating to Calcite because
the root of the issue lies in the communication between the coprocessor and the
driver.
Regarding `isConstant`: I think we can borrow the term ‘pure’ from functional
programmers :). The current implementation for `isConstant` describes the
criteria for a pure expression.
Thanks for pointing out the parens [~julianhyde]. I’ll remove them so that the
tests will pass after migrating to Calcite.
was (Author: kliew):
Thanks for the feedback guys. I investigated some more and I think I understand
the issue more clearly now.
We’ve set (in MetadataClient) tables with default value columns to store nulls
in HBase and we can detect whether a cell is null or default in
`PRowImpl.setValue`. Next, we’d need to communicate that from the coprocessor
to the driver but the rows serialized on the coprocessors contain only the
non-null values followed by a bitmask indicating set/null values. Currently
both nulls and default values are indicated as nulls in the bitmask.
We can’t change the value bitmask to indicate that default values are set and
not null, because that would affect the size of the expected bytes and the
offsets used for deserialization at the driver.
I think that given the constraints that we have now, we might need to serialize
another bitmask indicating default values.
Does that sound like a good solution?
I think we’d run into the same constraints after migrating to Calcite because
the root of the issue lies in the communication between the coprocessor and the
driver.
Regarding `isConstant`: I think we can borrow the term ‘pure’ from functional
programmers :). The current implementation for `isConstant` describes the
criteria for a pure expression.
Thanks for pointing out the parens [~julianhyde]. I’ll removed them so that the
tests will pass after migrating to Calcite.
> Support declaration of DEFAULT in CREATE statement
> --------------------------------------------------
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
> Affects Versions: 3.0-Release
> Reporter: James Taylor
> Assignee: Kevin Liew
> Labels: enhancement
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
