[jira] [Updated] (PHOENIX-5478) IndexTool mapper task should not timeout
[ https://issues.apache.org/jira/browse/PHOENIX-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5478: --- Attachment: PHOENIX-5478.master.addendum2.patch > IndexTool mapper task should not timeout > - > > Key: PHOENIX-5478 > URL: https://issues.apache.org/jira/browse/PHOENIX-5478 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.15.0 >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Attachments: PHOENIX-5478.master.001.patch, > PHOENIX-5478.master.002.patch, PHOENIX-5478.master.003.patch, > PHOENIX-5478.master.addendum.patch, PHOENIX-5478.master.addendum2.patch > > Time Spent: 2.5h > Remaining Estimate: 0h > > In the old design, the IndexTool MR job mapper first scanned the data table > rows one by one using a Phoenix client code, then constructed the index rows > and finally sent these row mutations to region servers to update the rows on > the index table regions. In the new design, this entire process is done on > the server side (within a coprocessor). So, the mapper just issues one RPC > call to instruct the coprocessor to build the entire table region. This RPC > call can timeout if the table region is large. The temporary solution that is > currently used is to set very large timeout values. We should break up single > table region rebuild into smaller operations and eliminate setting large > timeout values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly loading IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Attachment: PHOENIX-5556-4.x-HBase-1.4.patch
> Avoid repeatedly loading IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
> Attachments: PHOENIX-5556-4.x-HBase-1.4.patch
>
>
> Loading {{IndexMetaData}} is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {{IndexRegionObserver.preBatchMutateWithExceptions}} method,
> {{IndexMetaData}} is loaded already for line 613, but in
> {{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
> 639, {{IndexMetaData}} is loaded again, which is unnecessary.
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly loading IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Attachment: (was: PHOENIX-5556_v1-4.x-HBase-1.4.patch)
> Avoid repeatedly loading IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
>
> Loading {{IndexMetaData}} is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {{IndexRegionObserver.preBatchMutateWithExceptions}} method,
> {{IndexMetaData}} is loaded already for line 613, but in
> {{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
> 639, {{IndexMetaData}} is loaded again, which is unnecessary.
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly loading IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Attachment: PHOENIX-5556_v1-4.x-HBase-1.4.patch
> Avoid repeatedly loading IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
> Attachments: PHOENIX-5556_v1-4.x-HBase-1.4.patch
>
>
> Loading {{IndexMetaData}} is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {{IndexRegionObserver.preBatchMutateWithExceptions}} method,
> {{IndexMetaData}} is loaded already for line 613, but in
> {{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
> 639, {{IndexMetaData}} is loaded again, which is unnecessary.
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly loading IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Description:
Loading {{IndexMetaData}} is a somewhat expensive operation because it may
deserialize from scan attribute. In following
{{IndexRegionObserver.preBatchMutateWithExceptions}} method, {{IndexMetaData}}
is loaded already for line 613, but in
{{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
639, {{IndexMetaData}} is loaded again, which is unnecessary.
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611MiniBatchOperationInProgress miniBatchOp)
throws Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
614setBatchMutateContext(c, context);
615 Mutation firstMutation = miniBatchOp.getOperation(0);
639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
640
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
was:
Loading {{IndexMetaData}} is a somewhat expensive operation because it may
deserialize from scan attribute. In following
{{IndexRegionObserver.preBatchMutateWithExceptions}} method,loading
{{IndexMetaData}} for line 613, but in
{{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
639, {{IndexMetaData}} is loaded again, which is unnecessary.
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611MiniBatchOperationInProgress miniBatchOp)
throws Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
614setBatchMutateContext(c, context);
615 Mutation firstMutation = miniBatchOp.getOperation(0);
639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
640
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
> Avoid repeatedly loading IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
>
> Loading {{IndexMetaData}} is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {{IndexRegionObserver.preBatchMutateWithExceptions}} method,
> {{IndexMetaData}} is loaded already for line 613, but in
> {{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
> 639, {{IndexMetaData}} is loaded again, which is unnecessary.
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly loading IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Description:
Loading {{IndexMetaData}} is a somewhat expensive operation because it may
deserialize from scan attribute. In following
{{IndexRegionObserver.preBatchMutateWithExceptions}} method,loading
{{IndexMetaData}} for line 613, but in
{{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
639, {{IndexMetaData}} is loaded again, which is unnecessary.
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611MiniBatchOperationInProgress miniBatchOp)
throws Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
614setBatchMutateContext(c, context);
615 Mutation firstMutation = miniBatchOp.getOperation(0);
639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
640
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
was:
Loading {{IndexMetaData}} is a somewhat expensive operation because it may
deserialize from scan attribute. In following
{{IndexRegionObserver.preBatchMutateWithExceptions}} method,loading
{{IndexMetaData}} for line 613, but in
{{IndexRegionObserver.prepareIndexMutations}} which is invoked by line 639,
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611MiniBatchOperationInProgress miniBatchOp)
throws Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
614setBatchMutateContext(c, context);
615 Mutation firstMutation = miniBatchOp.getOperation(0);
639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
640
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
> Avoid repeatedly loading IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
>
> Loading {{IndexMetaData}} is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {{IndexRegionObserver.preBatchMutateWithExceptions}} method,loading
> {{IndexMetaData}} for line 613, but in
> {{IndexRegionObserver.prepareIndexMutations}} method which is invoked by line
> 639, {{IndexMetaData}} is loaded again, which is unnecessary.
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly load IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Description:
Loading {{IndexMetaData}} is a somewhat expensive operation because it may
deserialize from scan attribute. In following
{{IndexRegionObserver.preBatchMutateWithExceptions}} method,loading
{{IndexMetaData}} for line 613, but in
{{IndexRegionObserver.prepareIndexMutations}} which is invoked by line 639,
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611MiniBatchOperationInProgress miniBatchOp)
throws Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
614setBatchMutateContext(c, context);
615 Mutation firstMutation = miniBatchOp.getOperation(0);
639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
640
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
was:
Load IndexMetaData is a somewhat expensive operation because it may deserialize
from scan attribute. In following preBatchMutateWithExceptions
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611MiniBatchOperationInProgress miniBatchOp)
throws Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
614setBatchMutateContext(c, context);
615 Mutation firstMutation = miniBatchOp.getOperation(0);
639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
640
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
> Avoid repeatedly load IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
>
> Loading {{IndexMetaData}} is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {{IndexRegionObserver.preBatchMutateWithExceptions}} method,loading
> {{IndexMetaData}} for line 613, but in
> {{IndexRegionObserver.prepareIndexMutations}} which is invoked by line 639,
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly loading IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Summary: Avoid repeatedly loading IndexMetaData For IndexRegionObserver
(was: Avoid repeatedly load IndexMetaData For IndexRegionObserver)
> Avoid repeatedly loading IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
>
> Loading {{IndexMetaData}} is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {{IndexRegionObserver.preBatchMutateWithExceptions}} method,loading
> {{IndexMetaData}} for line 613, but in
> {{IndexRegionObserver.prepareIndexMutations}} which is invoked by line 639,
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly load IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Description:
Load IndexMetaData is a somewhat expensive operation because it may deserialize
from scan attribute. In following preBatchMutateWithExceptions
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611MiniBatchOperationInProgress miniBatchOp)
throws Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
614setBatchMutateContext(c, context);
615 Mutation firstMutation = miniBatchOp.getOperation(0);
639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
640
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
was:
Load IndexMetaData is a somewhat expensive operation because it may deserialize
from scan attribute. In following
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611 MiniBatchOperationInProgress miniBatchOp) throws
Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
setBatchMutateContext(c, context);
Mutation firstMutation = miniBatchOp.getOperation(0);
ReplayWrite replayWrite = this.builder.getReplayWrite(firstMutation);
/*
* Exclusively lock all rows so we get a consistent read
* while determining the index updates
*/
if (replayWrite == null) {
populateRowsToLock(miniBatchOp, context);
lockRows(context);
}
long now = EnvironmentEdgeManager.currentTimeMillis();
// Add the table rows in the mini batch to the collection of pending
rows. This will be used to detect
// concurrent updates
if (replayWrite == null) {
populatePendingRows(context, now);
}
// First group all the updates for a single row into a single update to
be processed
Collection mutations = groupMutations(miniBatchOp,
now, replayWrite);
// early exit if it turns out we don't have any edits
if (mutations == null) {
return;
}
long start = EnvironmentEdgeManager.currentTimeMillis();
prepareIndexMutations(c, miniBatchOp, context, mutations, now);
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
> Avoid repeatedly load IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
>
> Load IndexMetaData is a somewhat expensive operation because it may
> deserialize from scan attribute. In following preBatchMutateWithExceptions
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611MiniBatchOperationInProgress miniBatchOp)
> throws Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> 614setBatchMutateContext(c, context);
> 615 Mutation firstMutation = miniBatchOp.getOperation(0);
>
> 639prepareIndexMutations(c, miniBatchOp, context, mutations, now);
> 640
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly load IndexMetaData For IndexRegionObserver
[
https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5556:
--
Description:
Load IndexMetaData is a somewhat expensive operation because it may deserialize
from scan attribute. In following
{code:java}
610 public void
preBatchMutateWithExceptions(ObserverContext c,
611 MiniBatchOperationInProgress miniBatchOp) throws
Throwable {
612 ignoreAtomicOperations(miniBatchOp);
613 BatchMutateContext context = new
BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
setBatchMutateContext(c, context);
Mutation firstMutation = miniBatchOp.getOperation(0);
ReplayWrite replayWrite = this.builder.getReplayWrite(firstMutation);
/*
* Exclusively lock all rows so we get a consistent read
* while determining the index updates
*/
if (replayWrite == null) {
populateRowsToLock(miniBatchOp, context);
lockRows(context);
}
long now = EnvironmentEdgeManager.currentTimeMillis();
// Add the table rows in the mini batch to the collection of pending
rows. This will be used to detect
// concurrent updates
if (replayWrite == null) {
populatePendingRows(context, now);
}
// First group all the updates for a single row into a single update to
be processed
Collection mutations = groupMutations(miniBatchOp,
now, replayWrite);
// early exit if it turns out we don't have any edits
if (mutations == null) {
return;
}
long start = EnvironmentEdgeManager.currentTimeMillis();
prepareIndexMutations(c, miniBatchOp, context, mutations, now);
metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
- start);
{code}
> Avoid repeatedly load IndexMetaData For IndexRegionObserver
> ---
>
> Key: PHOENIX-5556
> URL: https://issues.apache.org/jira/browse/PHOENIX-5556
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0, 5.1.0
>Reporter: Chenglei
>Assignee: chenglei
>Priority: Major
>
> Load IndexMetaData is a somewhat expensive operation because it may
> deserialize from scan attribute. In following
> {code:java}
> 610 public void
> preBatchMutateWithExceptions(ObserverContext c,
> 611 MiniBatchOperationInProgress miniBatchOp) throws
> Throwable {
> 612 ignoreAtomicOperations(miniBatchOp);
> 613 BatchMutateContext context = new
> BatchMutateContext(this.builder.getIndexMetaData(miniBatchOp).getClientVersion());
> setBatchMutateContext(c, context);
> Mutation firstMutation = miniBatchOp.getOperation(0);
> ReplayWrite replayWrite = this.builder.getReplayWrite(firstMutation);
> /*
>* Exclusively lock all rows so we get a consistent read
>* while determining the index updates
>*/
> if (replayWrite == null) {
> populateRowsToLock(miniBatchOp, context);
> lockRows(context);
> }
> long now = EnvironmentEdgeManager.currentTimeMillis();
> // Add the table rows in the mini batch to the collection of pending
> rows. This will be used to detect
> // concurrent updates
> if (replayWrite == null) {
> populatePendingRows(context, now);
> }
> // First group all the updates for a single row into a single update to
> be processed
> Collection mutations = groupMutations(miniBatchOp,
> now, replayWrite);
> // early exit if it turns out we don't have any edits
> if (mutations == null) {
> return;
> }
> long start = EnvironmentEdgeManager.currentTimeMillis();
> prepareIndexMutations(c, miniBatchOp, context, mutations, now);
>
> metricSource.updateIndexPrepareTime(EnvironmentEdgeManager.currentTimeMillis()
> - start);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5556) Avoid repeatedly load IndexMetaData For IndexRegionObserver
[ https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenglei updated PHOENIX-5556: -- Summary: Avoid repeatedly load IndexMetaData For IndexRegionObserver (was: Avoid repeatedly load IndexMetaData) > Avoid repeatedly load IndexMetaData For IndexRegionObserver > --- > > Key: PHOENIX-5556 > URL: https://issues.apache.org/jira/browse/PHOENIX-5556 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0, 5.1.0 >Reporter: Chenglei >Assignee: chenglei >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1 At 2019-10-31 07:57:32, "rajeshb...@apache.org" wrote: >+1 > >On Wed, Oct 30, 2019, 11:55 PM Ankit Singhal >wrote: > >> +1 >> >> On Wed, Oct 30, 2019 at 10:44 AM Andrew Purtell >> wrote: >> >> > +0 (binding) >> > >> > I support this but it isn't my time I'd be committing to maintaining the >> > new repos with a +1... >> > >> > On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: >> > >> > > Hi, >> > > >> > > As was previously discussed[1][2], there is a motion to "adopt" the >> > > Tephra and Omid podlings as sub-projects to Apache Phoenix. A >> > > sub-project is a distinct software project from some top-level project >> > > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) >> > > >> > > Per the Incubator guidelines[3], we need to have a formal vote to adopt >> > > them. While we still need details from these two podlings, I believe we >> > > can have a VOTE now to keep things moving. >> > > >> > > Actions: >> > > * Phoenix will incorporate Omid and Tephra as sub-projects and they >> will >> > > continue to function under the Apache Phoenix PMC guidance. >> > > * Any current podling member may request to be added as a Phoenix >> > > committer. Podling members would be expected to demonstrate the normal >> > > level of commitment to grow from a committer to a PMC member. >> > > >> > > Stating what I see as an implicit decision (but to alleviate any >> > > possible confusion): all new community members will be expected to >> > > function in the way that Phoenix currently does today (e.g >> > > review-then-commit). Future Omid and Tephra development would happen in >> > > the same way that Phoenix development happens today. >> > > >> > > Please vote: >> > > >> > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any >> > > PPMC to become Phoenix committers. >> > > >> > > -1/-0/+0: No and why.. >> > > >> > > Here is my +1 (binding) >> > > >> > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). >> > > >> > > >> > > [1] >> > > >> > > >> > >> https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E >> > > [2] >> > > >> > > >> > >> https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E >> > > [3] >> > > >> > > >> > >> https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject >> > > >> > >> > >> > -- >> > Best regards, >> > Andrew >> > >> > Words like orphans lost among the crosstalk, meaning torn from truth's >> > decrepit hands >> >- A23, Crosstalk >> > >>
[jira] [Assigned] (PHOENIX-5432) Refactor LiteralExpression to use the builder pattern
[ https://issues.apache.org/jira/browse/PHOENIX-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Feng reassigned PHOENIX-5432: --- Assignee: Christine Feng > Refactor LiteralExpression to use the builder pattern > - > > Key: PHOENIX-5432 > URL: https://issues.apache.org/jira/browse/PHOENIX-5432 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.15.0, 5.1.0 >Reporter: Chinmay Kulkarni >Assignee: Christine Feng >Priority: Major > Attachments: PHOENIX-5432.patch > > Time Spent: 20m > Remaining Estimate: 0h > > LiteralExpression is a mess. While it provides newConstant() APIs to build > the object, it also provides two public constructors. There are 10 overloaded > newConstant() methods and it is unclear which API to use in which case. > This should be refactored to use the builder pattern and final member > variables. Ideally, getters such as getMaxLength() should be simple member > variable accessors and other ad-hoc logic surrounding those variables should > be handled correctly when setting their respective values. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5432) Refactor LiteralExpression to use the builder pattern
[ https://issues.apache.org/jira/browse/PHOENIX-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Feng updated PHOENIX-5432: Attachment: PHOENIX-5432.patch > Refactor LiteralExpression to use the builder pattern > - > > Key: PHOENIX-5432 > URL: https://issues.apache.org/jira/browse/PHOENIX-5432 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.15.0, 5.1.0 >Reporter: Chinmay Kulkarni >Priority: Major > Attachments: PHOENIX-5432.patch > > Time Spent: 20m > Remaining Estimate: 0h > > LiteralExpression is a mess. While it provides newConstant() APIs to build > the object, it also provides two public constructors. There are 10 overloaded > newConstant() methods and it is unclear which API to use in which case. > This should be refactored to use the builder pattern and final member > variables. Ideally, getters such as getMaxLength() should be simple member > variable accessors and other ad-hoc logic surrounding those variables should > be handled correctly when setting their respective values. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1 On Wed, Oct 30, 2019, 11:55 PM Ankit Singhal wrote: > +1 > > On Wed, Oct 30, 2019 at 10:44 AM Andrew Purtell > wrote: > > > +0 (binding) > > > > I support this but it isn't my time I'd be committing to maintaining the > > new repos with a +1... > > > > On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: > > > > > Hi, > > > > > > As was previously discussed[1][2], there is a motion to "adopt" the > > > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > > > sub-project is a distinct software project from some top-level project > > > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > > > > > Per the Incubator guidelines[3], we need to have a formal vote to adopt > > > them. While we still need details from these two podlings, I believe we > > > can have a VOTE now to keep things moving. > > > > > > Actions: > > > * Phoenix will incorporate Omid and Tephra as sub-projects and they > will > > > continue to function under the Apache Phoenix PMC guidance. > > > * Any current podling member may request to be added as a Phoenix > > > committer. Podling members would be expected to demonstrate the normal > > > level of commitment to grow from a committer to a PMC member. > > > > > > Stating what I see as an implicit decision (but to alleviate any > > > possible confusion): all new community members will be expected to > > > function in the way that Phoenix currently does today (e.g > > > review-then-commit). Future Omid and Tephra development would happen in > > > the same way that Phoenix development happens today. > > > > > > Please vote: > > > > > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any > > > PPMC to become Phoenix committers. > > > > > > -1/-0/+0: No and why.. > > > > > > Here is my +1 (binding) > > > > > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > > > > > > > [1] > > > > > > > > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > > > [2] > > > > > > > > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > > > [3] > > > > > > > > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject > > > > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > >
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1 On Wed, Oct 30, 2019 at 12:43 PM James Taylor wrote: > +1 > > On Wed, Oct 30, 2019 at 12:32 PM Geoffrey Jacoby > wrote: > > > +1 > > > > On Wed, Oct 30, 2019 at 1:25 PM Ankit Singhal > > wrote: > > > > > +1 > > > > > > On Wed, Oct 30, 2019 at 10:44 AM Andrew Purtell > > > wrote: > > > > > > > +0 (binding) > > > > > > > > I support this but it isn't my time I'd be committing to maintaining > > the > > > > new repos with a +1... > > > > > > > > On Wed, Oct 30, 2019 at 8:27 AM Josh Elser > wrote: > > > > > > > > > Hi, > > > > > > > > > > As was previously discussed[1][2], there is a motion to "adopt" the > > > > > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > > > > > sub-project is a distinct software project from some top-level > > project > > > > > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > > > > > > > > > Per the Incubator guidelines[3], we need to have a formal vote to > > adopt > > > > > them. While we still need details from these two podlings, I > believe > > we > > > > > can have a VOTE now to keep things moving. > > > > > > > > > > Actions: > > > > > * Phoenix will incorporate Omid and Tephra as sub-projects and they > > > will > > > > > continue to function under the Apache Phoenix PMC guidance. > > > > > * Any current podling member may request to be added as a Phoenix > > > > > committer. Podling members would be expected to demonstrate the > > normal > > > > > level of commitment to grow from a committer to a PMC member. > > > > > > > > > > Stating what I see as an implicit decision (but to alleviate any > > > > > possible confusion): all new community members will be expected to > > > > > function in the way that Phoenix currently does today (e.g > > > > > review-then-commit). Future Omid and Tephra development would > happen > > in > > > > > the same way that Phoenix development happens today. > > > > > > > > > > Please vote: > > > > > > > > > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow > > any > > > > > PPMC to become Phoenix committers. > > > > > > > > > > -1/-0/+0: No and why.. > > > > > > > > > > Here is my +1 (binding) > > > > > > > > > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > > > > > [2] > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > > > > > [3] > > > > > > > > > > > > > > > > > > > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Andrew > > > > > > > > Words like orphans lost among the crosstalk, meaning torn from > truth's > > > > decrepit hands > > > >- A23, Crosstalk > > > > > > > > > >
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1 On Wed, Oct 30, 2019 at 12:32 PM Geoffrey Jacoby wrote: > +1 > > On Wed, Oct 30, 2019 at 1:25 PM Ankit Singhal > wrote: > > > +1 > > > > On Wed, Oct 30, 2019 at 10:44 AM Andrew Purtell > > wrote: > > > > > +0 (binding) > > > > > > I support this but it isn't my time I'd be committing to maintaining > the > > > new repos with a +1... > > > > > > On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: > > > > > > > Hi, > > > > > > > > As was previously discussed[1][2], there is a motion to "adopt" the > > > > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > > > > sub-project is a distinct software project from some top-level > project > > > > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > > > > > > > Per the Incubator guidelines[3], we need to have a formal vote to > adopt > > > > them. While we still need details from these two podlings, I believe > we > > > > can have a VOTE now to keep things moving. > > > > > > > > Actions: > > > > * Phoenix will incorporate Omid and Tephra as sub-projects and they > > will > > > > continue to function under the Apache Phoenix PMC guidance. > > > > * Any current podling member may request to be added as a Phoenix > > > > committer. Podling members would be expected to demonstrate the > normal > > > > level of commitment to grow from a committer to a PMC member. > > > > > > > > Stating what I see as an implicit decision (but to alleviate any > > > > possible confusion): all new community members will be expected to > > > > function in the way that Phoenix currently does today (e.g > > > > review-then-commit). Future Omid and Tephra development would happen > in > > > > the same way that Phoenix development happens today. > > > > > > > > Please vote: > > > > > > > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow > any > > > > PPMC to become Phoenix committers. > > > > > > > > -1/-0/+0: No and why.. > > > > > > > > Here is my +1 (binding) > > > > > > > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > > > > > > > > > > [1] > > > > > > > > > > > > > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > > > > [2] > > > > > > > > > > > > > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > > > > [3] > > > > > > > > > > > > > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject > > > > > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > >- A23, Crosstalk > > > > > >
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1 On Wed, Oct 30, 2019 at 1:25 PM Ankit Singhal wrote: > +1 > > On Wed, Oct 30, 2019 at 10:44 AM Andrew Purtell > wrote: > > > +0 (binding) > > > > I support this but it isn't my time I'd be committing to maintaining the > > new repos with a +1... > > > > On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: > > > > > Hi, > > > > > > As was previously discussed[1][2], there is a motion to "adopt" the > > > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > > > sub-project is a distinct software project from some top-level project > > > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > > > > > Per the Incubator guidelines[3], we need to have a formal vote to adopt > > > them. While we still need details from these two podlings, I believe we > > > can have a VOTE now to keep things moving. > > > > > > Actions: > > > * Phoenix will incorporate Omid and Tephra as sub-projects and they > will > > > continue to function under the Apache Phoenix PMC guidance. > > > * Any current podling member may request to be added as a Phoenix > > > committer. Podling members would be expected to demonstrate the normal > > > level of commitment to grow from a committer to a PMC member. > > > > > > Stating what I see as an implicit decision (but to alleviate any > > > possible confusion): all new community members will be expected to > > > function in the way that Phoenix currently does today (e.g > > > review-then-commit). Future Omid and Tephra development would happen in > > > the same way that Phoenix development happens today. > > > > > > Please vote: > > > > > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any > > > PPMC to become Phoenix committers. > > > > > > -1/-0/+0: No and why.. > > > > > > Here is my +1 (binding) > > > > > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > > > > > > > [1] > > > > > > > > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > > > [2] > > > > > > > > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > > > [3] > > > > > > > > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject > > > > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > >
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1 On Wed, Oct 30, 2019 at 10:44 AM Andrew Purtell wrote: > +0 (binding) > > I support this but it isn't my time I'd be committing to maintaining the > new repos with a +1... > > On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: > > > Hi, > > > > As was previously discussed[1][2], there is a motion to "adopt" the > > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > > sub-project is a distinct software project from some top-level project > > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > > > Per the Incubator guidelines[3], we need to have a formal vote to adopt > > them. While we still need details from these two podlings, I believe we > > can have a VOTE now to keep things moving. > > > > Actions: > > * Phoenix will incorporate Omid and Tephra as sub-projects and they will > > continue to function under the Apache Phoenix PMC guidance. > > * Any current podling member may request to be added as a Phoenix > > committer. Podling members would be expected to demonstrate the normal > > level of commitment to grow from a committer to a PMC member. > > > > Stating what I see as an implicit decision (but to alleviate any > > possible confusion): all new community members will be expected to > > function in the way that Phoenix currently does today (e.g > > review-then-commit). Future Omid and Tephra development would happen in > > the same way that Phoenix development happens today. > > > > Please vote: > > > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any > > PPMC to become Phoenix committers. > > > > -1/-0/+0: No and why.. > > > > Here is my +1 (binding) > > > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > > > > [1] > > > > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > > [2] > > > > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > > [3] > > > > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+0 (binding) I support this but it isn't my time I'd be committing to maintaining the new repos with a +1... On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: > Hi, > > As was previously discussed[1][2], there is a motion to "adopt" the > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > sub-project is a distinct software project from some top-level project > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > Per the Incubator guidelines[3], we need to have a formal vote to adopt > them. While we still need details from these two podlings, I believe we > can have a VOTE now to keep things moving. > > Actions: > * Phoenix will incorporate Omid and Tephra as sub-projects and they will > continue to function under the Apache Phoenix PMC guidance. > * Any current podling member may request to be added as a Phoenix > committer. Podling members would be expected to demonstrate the normal > level of commitment to grow from a committer to a PMC member. > > Stating what I see as an implicit decision (but to alleviate any > possible confusion): all new community members will be expected to > function in the way that Phoenix currently does today (e.g > review-then-commit). Future Omid and Tephra development would happen in > the same way that Phoenix development happens today. > > Please vote: > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any > PPMC to become Phoenix committers. > > -1/-0/+0: No and why.. > > Here is my +1 (binding) > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > [1] > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > [2] > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > [3] > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1 On Wed, Oct 30, 2019 at 8:57 AM William Shen wrote: > +1! > > On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: > > > Hi, > > > > As was previously discussed[1][2], there is a motion to "adopt" the > > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > > sub-project is a distinct software project from some top-level project > > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > > > Per the Incubator guidelines[3], we need to have a formal vote to adopt > > them. While we still need details from these two podlings, I believe we > > can have a VOTE now to keep things moving. > > > > Actions: > > * Phoenix will incorporate Omid and Tephra as sub-projects and they will > > continue to function under the Apache Phoenix PMC guidance. > > * Any current podling member may request to be added as a Phoenix > > committer. Podling members would be expected to demonstrate the normal > > level of commitment to grow from a committer to a PMC member. > > > > Stating what I see as an implicit decision (but to alleviate any > > possible confusion): all new community members will be expected to > > function in the way that Phoenix currently does today (e.g > > review-then-commit). Future Omid and Tephra development would happen in > > the same way that Phoenix development happens today. > > > > Please vote: > > > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any > > PPMC to become Phoenix committers. > > > > -1/-0/+0: No and why.. > > > > Here is my +1 (binding) > > > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > > > > [1] > > > > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > > [2] > > > > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > > [3] > > > > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject > > >
Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
+1! On Wed, Oct 30, 2019 at 8:27 AM Josh Elser wrote: > Hi, > > As was previously discussed[1][2], there is a motion to "adopt" the > Tephra and Omid podlings as sub-projects to Apache Phoenix. A > sub-project is a distinct software project from some top-level project > (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) > > Per the Incubator guidelines[3], we need to have a formal vote to adopt > them. While we still need details from these two podlings, I believe we > can have a VOTE now to keep things moving. > > Actions: > * Phoenix will incorporate Omid and Tephra as sub-projects and they will > continue to function under the Apache Phoenix PMC guidance. > * Any current podling member may request to be added as a Phoenix > committer. Podling members would be expected to demonstrate the normal > level of commitment to grow from a committer to a PMC member. > > Stating what I see as an implicit decision (but to alleviate any > possible confusion): all new community members will be expected to > function in the way that Phoenix currently does today (e.g > review-then-commit). Future Omid and Tephra development would happen in > the same way that Phoenix development happens today. > > Please vote: > > +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any > PPMC to become Phoenix committers. > > -1/-0/+0: No and why.. > > Here is my +1 (binding) > > This vote will be open for at least 72hrs (2019/11/02 1600 UTC). > > > [1] > > https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E > [2] > > https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E > [3] > > https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject >
[VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects
Hi, As was previously discussed[1][2], there is a motion to "adopt" the Tephra and Omid podlings as sub-projects to Apache Phoenix. A sub-project is a distinct software project from some top-level project (TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC) Per the Incubator guidelines[3], we need to have a formal vote to adopt them. While we still need details from these two podlings, I believe we can have a VOTE now to keep things moving. Actions: * Phoenix will incorporate Omid and Tephra as sub-projects and they will continue to function under the Apache Phoenix PMC guidance. * Any current podling member may request to be added as a Phoenix committer. Podling members would be expected to demonstrate the normal level of commitment to grow from a committer to a PMC member. Stating what I see as an implicit decision (but to alleviate any possible confusion): all new community members will be expected to function in the way that Phoenix currently does today (e.g review-then-commit). Future Omid and Tephra development would happen in the same way that Phoenix development happens today. Please vote: +1: to accept Omid and Tephra as Phoenix sub-projects and to allow any PPMC to become Phoenix committers. -1/-0/+0: No and why.. Here is my +1 (binding) This vote will be open for at least 72hrs (2019/11/02 1600 UTC). [1] https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E [2] https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E [3] https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject
Re: [DISCUSS] Omid and Tephra podlings
Just noticed that the links Alan sent out do explicitly require a formal VOTE. Incoming... On 10/30/19 10:06 AM, Josh Elser wrote: I am OK with Tephra and Omid committers becoming Phoenix committers. We can send them a welcome package to remind them that Phoenix is a "review-then-commit" community which assuages any fears I have. Agree on not giving PMC membership immediately. We can ask infra to just rename each of their repositories out of "incubator" and into "phoenix (e.g. incubator-omid becomes phoenix-omid) to match the infra conventions, and ask Infra to switch the LDAP permissions to point at Phoenix group instead of the podlings/incubator groups. (with Phoenix VP hat on) I think you're good to go, Alan. The only objection we've heard was a weak one about not wasting Phoenix committers times (and the above plan isn't a time sink on anyone). Our discussions meet my bar for lazy consensus. If anyone wants (or needs) to see an official VOTE, please holler and I'll send a thread out. - Josh On 10/28/19 7:01 PM, Alan Gates wrote: It sounds like the Phoenix community is on board, so the next step will be to have votes in Tephra and Omid. Before I start votes there I need some indication of how Phoenix will integrate the people in Tephra and Omid. My suggestion is that you allow any committer on Omid or Tephra to request to be a committer on Phoenix. This way anyone who wants to continue to develop the technology can, while those who have already moved away from the project won't be added to Phoenix as inactive committers. I wouldn't put Tephra or Omid PPMC members on the Phoenix PMC. They will be committers if they choose, and from there they can become PMC members in time. This is just a suggestion, feel free to modify or ignore it. Once you build consensus around a proposal I'll start votes in Tephra and Omid. Once those communities have ratified it the Phoenix PMC and then the Incubator PMC will need to ratify it as well. Alan. On 2019/10/28 20:08:36, Geoffrey Jacoby wrote: +1. I agree with Vincent and James. So long as we support transactions in Phoenix, we're bound to keep at least one transaction manager working, and we'd need to contribute at least compatibility and bug fixes. That's probably more easily done under our umbrella if the Omid and Tephra communities have become inactive. If we were sure that most of our users would use one over the other, we could potentially just adopt one, but since they have different strengths and drawbacks, that might never be the case. (If I remember right, Lars Hofhansl did some perf testing awhile back that showed Tephra better for fewer, longer transactions and Omid better for more frequent short ones.) At least for now, probably better to take on both. Geoffrey On Fri, Oct 25, 2019 at 6:14 PM Vincent Poon wrote: +1 I don't think pulling these under Phoenix would change their volume of activity in the near term. However, when/if usage of transactions with Phoenix increases, we'd want a path forward to be able to make modifications where necessary to these projects. This seems the best way to ensure there's an expedient way to do that. On Fri, Oct 25, 2019 at 10:36 AM James Taylor wrote: +1 to pulling these projects into Phoenix. We’ve already done some work in those projects and so have some familiarity with them. We also have some overlap in the committers with Omid. At a minimum we’d need to create new compat modules when necessary for future HBase releases. The alternative would be to rip out the transaction support which would be a mistake IMHO. On Fri, Oct 25, 2019 at 8:08 AM Josh Elser wrote: I share your same hesitation. I'm worried about us adopting code that no one intends to actually maintain. My immediate concern is whether adoption of these codebases would impact the PMC's ability to develop on Phoenix -- I think there's a path forward to avoid that. Is that sufficient to say we "should" adopt them? I dunno. On 10/24/19 5:37 PM, Andrew Purtell wrote: Tephra and Omid are interesting in that they serve a similar function - a transaction oracle - but scale differently per different choices in design, so one is more appropriate for some kinds of transactional workloads vs the other. It's akin to secondary indexing, there is not an index type that fits all use cases. If you are going to consider one, you should consider both. That said my guess is you will find eventually one is the 'winner' per second order measures like contributions or user issues. This should be fine. As separate repositories they can move at their own speed and only consume bandwidth as usage and uptake actually demands. For what it's worth I'd vote as PMC +0 on accepting these code bases. '+' because Phoenix transactional indexes are a promising feature, and could be compelling, and they need one of these transaction oracles. '0' because it would be unfair to
Re: [DISCUSS] Omid and Tephra podlings
I am OK with Tephra and Omid committers becoming Phoenix committers. We can send them a welcome package to remind them that Phoenix is a "review-then-commit" community which assuages any fears I have. Agree on not giving PMC membership immediately. We can ask infra to just rename each of their repositories out of "incubator" and into "phoenix (e.g. incubator-omid becomes phoenix-omid) to match the infra conventions, and ask Infra to switch the LDAP permissions to point at Phoenix group instead of the podlings/incubator groups. (with Phoenix VP hat on) I think you're good to go, Alan. The only objection we've heard was a weak one about not wasting Phoenix committers times (and the above plan isn't a time sink on anyone). Our discussions meet my bar for lazy consensus. If anyone wants (or needs) to see an official VOTE, please holler and I'll send a thread out. - Josh On 10/28/19 7:01 PM, Alan Gates wrote: It sounds like the Phoenix community is on board, so the next step will be to have votes in Tephra and Omid. Before I start votes there I need some indication of how Phoenix will integrate the people in Tephra and Omid. My suggestion is that you allow any committer on Omid or Tephra to request to be a committer on Phoenix. This way anyone who wants to continue to develop the technology can, while those who have already moved away from the project won't be added to Phoenix as inactive committers. I wouldn't put Tephra or Omid PPMC members on the Phoenix PMC. They will be committers if they choose, and from there they can become PMC members in time. This is just a suggestion, feel free to modify or ignore it. Once you build consensus around a proposal I'll start votes in Tephra and Omid. Once those communities have ratified it the Phoenix PMC and then the Incubator PMC will need to ratify it as well. Alan. On 2019/10/28 20:08:36, Geoffrey Jacoby wrote: +1. I agree with Vincent and James. So long as we support transactions in Phoenix, we're bound to keep at least one transaction manager working, and we'd need to contribute at least compatibility and bug fixes. That's probably more easily done under our umbrella if the Omid and Tephra communities have become inactive. If we were sure that most of our users would use one over the other, we could potentially just adopt one, but since they have different strengths and drawbacks, that might never be the case. (If I remember right, Lars Hofhansl did some perf testing awhile back that showed Tephra better for fewer, longer transactions and Omid better for more frequent short ones.) At least for now, probably better to take on both. Geoffrey On Fri, Oct 25, 2019 at 6:14 PM Vincent Poon wrote: +1 I don't think pulling these under Phoenix would change their volume of activity in the near term. However, when/if usage of transactions with Phoenix increases, we'd want a path forward to be able to make modifications where necessary to these projects. This seems the best way to ensure there's an expedient way to do that. On Fri, Oct 25, 2019 at 10:36 AM James Taylor wrote: +1 to pulling these projects into Phoenix. We’ve already done some work in those projects and so have some familiarity with them. We also have some overlap in the committers with Omid. At a minimum we’d need to create new compat modules when necessary for future HBase releases. The alternative would be to rip out the transaction support which would be a mistake IMHO. On Fri, Oct 25, 2019 at 8:08 AM Josh Elser wrote: I share your same hesitation. I'm worried about us adopting code that no one intends to actually maintain. My immediate concern is whether adoption of these codebases would impact the PMC's ability to develop on Phoenix -- I think there's a path forward to avoid that. Is that sufficient to say we "should" adopt them? I dunno. On 10/24/19 5:37 PM, Andrew Purtell wrote: Tephra and Omid are interesting in that they serve a similar function - a transaction oracle - but scale differently per different choices in design, so one is more appropriate for some kinds of transactional workloads vs the other. It's akin to secondary indexing, there is not an index type that fits all use cases. If you are going to consider one, you should consider both. That said my guess is you will find eventually one is the 'winner' per second order measures like contributions or user issues. This should be fine. As separate repositories they can move at their own speed and only consume bandwidth as usage and uptake actually demands. For what it's worth I'd vote as PMC +0 on accepting these code bases. '+' because Phoenix transactional indexes are a promising feature, and could be compelling, and they need one of these transaction oracles. '0' because it would be unfair to commit someone else's time. I'm not around here much... but may be around more going forward. On Thu, Oct 24, 2019 at 10:14 AM Josh Elser wrote:
[jira] [Assigned] (PHOENIX-5556) Avoid repeatedly load IndexMetaData
[ https://issues.apache.org/jira/browse/PHOENIX-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenglei reassigned PHOENIX-5556: - Assignee: chenglei > Avoid repeatedly load IndexMetaData > --- > > Key: PHOENIX-5556 > URL: https://issues.apache.org/jira/browse/PHOENIX-5556 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0, 5.1.0 >Reporter: Chenglei >Assignee: chenglei >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (PHOENIX-5555) Remove duplicate semicolons for import in IndexCommitter
[
https://issues.apache.org/jira/browse/PHOENIX-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenglei resolved PHOENIX-.
---
Fix Version/s: 5.1.0
4.15.0
Resolution: Fixed
> Remove duplicate semicolons for import in IndexCommitter
>
>
> Key: PHOENIX-
> URL: https://issues.apache.org/jira/browse/PHOENIX-
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Chenglei
>Priority: Minor
> Fix For: 4.15.0, 5.1.0
>
>
> Remove duplicate semicolons for import in {{IndexCommitter}}, which may
> cause compiler error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (PHOENIX-5556) Avoid repeatedly load IndexMetaData
Chenglei created PHOENIX-5556: - Summary: Avoid repeatedly load IndexMetaData Key: PHOENIX-5556 URL: https://issues.apache.org/jira/browse/PHOENIX-5556 Project: Phoenix Issue Type: Bug Affects Versions: 4.15.0, 5.1.0 Reporter: Chenglei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5555) Remove duplicate semicolons for import in IndexCommitter
[
https://issues.apache.org/jira/browse/PHOENIX-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenglei updated PHOENIX-:
--
Summary: Remove duplicate semicolons for import in IndexCommitter (was:
Remove duplicate semicolons for import)
> Remove duplicate semicolons for import in IndexCommitter
>
>
> Key: PHOENIX-
> URL: https://issues.apache.org/jira/browse/PHOENIX-
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Chenglei
>Priority: Minor
>
> Remove duplicate semicolons for import in {{IndexCommitter}}, which may
> cause compiler error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5555) Remove duplicate semicolons for import
[
https://issues.apache.org/jira/browse/PHOENIX-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenglei updated PHOENIX-:
--
Description: Remove duplicate semicolons for import in {{IndexCommitter}},
which may cause compiler error.
> Remove duplicate semicolons for import
> --
>
> Key: PHOENIX-
> URL: https://issues.apache.org/jira/browse/PHOENIX-
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Chenglei
>Priority: Minor
>
> Remove duplicate semicolons for import in {{IndexCommitter}}, which may
> cause compiler error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (PHOENIX-5555) Remove duplicate semicolons for import
Chenglei created PHOENIX-: - Summary: Remove duplicate semicolons for import Key: PHOENIX- URL: https://issues.apache.org/jira/browse/PHOENIX- Project: Phoenix Issue Type: Bug Affects Versions: 4.15.0 Reporter: Chenglei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (PHOENIX-5554) Synchronize @Parameters, @BeforeClass and @AfterClass methods
István Tóth created PHOENIX-5554: Summary: Synchronize @Parameters, @BeforeClass and @AfterClass methods Key: PHOENIX-5554 URL: https://issues.apache.org/jira/browse/PHOENIX-5554 Project: Phoenix Issue Type: Bug Reporter: István Tóth Assignee: István Tóth According to the failsafe docs, all methods that are annotated with @Parameters, @BeforeClass and @AfterClass should be synchronized. This is not just a theoretical problem, it causes random test failures, like [https://builds.apache.org/job/PreCommit-PHOENIX-Build/3078//testReport/org.apache.phoenix.end2end/DescOrderKeysIT/org_apache_phoenix_end2end_DescOrderKeysIT/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
