Re: [Discuss] Releasing Phoenix 4.16

2021-01-07 Thread Istvan Toth
On the release binaries:

The current solution (which the default profile change has broken) was
based on Lars's idea at
https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
However, I agree that providing separate assemblies for each HBase profile
is better for our users, as they won't have to rebuild Phoenix to take
advantage of any new features, and to get the general  improvements in
later HBase releases.
I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to track
this.

On the 5.1 release:

Yes, I do want to release 5.1 shortly. In fact $dayjob desperately needs it
ASAP.
I have a few older PRs waiting for review that fell between the cracks, but
as soon as those are merged, I want to cut the first RC.
It would be nice to have 4.16 and 5.1 as close as possible, and
PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
I will start an official 5.1 release thread and volunteer to be the release
manager soon. (unless you want to take that up too, Xinyi).



On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan  wrote:

> If we can modify the dev/create-release scripts and make them work for the
> 4.16 release with this hbase.profile option, it would make our life much
> easier to release multiple HBase profiles from the single branch in the
> future too(the master branch will have a release shortly right?). Geoffrey
> and Istvan, what do you think?
>
> Thanks,
> Xinyi
>
> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby 
> wrote:
>
> > Thanks for bringing up the default branch issue, Istvan, I've been
> meaning
> > to start a conversation about it on this list.
> >
> > As part of PHOENIX-5435, I changed the default 4.x HBase release to 1.5
> and
> > the default 5.x HBase to 2.3 because the WAL annotation feature
> introduced
> > by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc hook
> > introduced in HBASE-22623). That means that all tests of that feature
> must
> > no-op when run against an earlier HBase, which means that it would never
> be
> > tested in our CI pipelines if the default was 1.3 or 2.1.
> >
> > This has come up quite a few times recently. In particular, the new
> > indexing framework runs in a degraded state against HBase 2.1 and 2.2
> > (still better than the old indexing framework though!), because we lack
> > support in 2.1 for Filters on raw scans and can't use the "max lookback
> > age" feature with HBase 2.1 or 2.2, which keeps major compactions from
> > messing with index rebuilds. That's why lots of indexing tests no-op or
> > have to make different assertions when run against 2.1 or 2.2, so we only
> > properly test the indexing framework if CI and local developer
> > tests run against HBase 2.3 or up.
> >
> > As for the release, don't we usually release separate binaries that are
> > suitable for all the HBase versions we support? Back before we had a
> > unified 4.x branch, I believe we used to release from 3 different
> > 4.x-HBase-* branches each time. I've been assuming that part of the
> release
> > process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
> > 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
> >
> > Geoffrey
> >
> > On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth  wrote:
> >
> > > Two more issues came to my mind:
> > >
> > > In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> > > This causes two conflicts:
> > > - Our documentation says that the default profile is the lowest
> > supported,
> > > which is no longer true
> > > - Unless we change our binary release process, the published maven
> > > artifacts and binaries will also be built with HBase 1.5, thus they
> will
> > be
> > > incompatible with Hbase 1.3 and 1.4
> > >
> > > This should be addressed before release.
> > > One possible solution:
> > > * Add an "oldest" hbase.profile option which always chooses the oldest
> > > supported version
> > > * Update the release scripts to specify this profile
> > > * Update the docs.
> > >
> > > I have also adopted the hbase release scripts to Phoenix, they are
> > present
> > > in dev/create-release in the master branch, but
> > > should be able to perform 4.16 release as well. I have used this for
> the
> > > phoenix-thirdparty, omid, and tephra releases, but no live phoenix
> > releases
> > > have been done with them yet, obviously.
> > >
> > > regards
> > > Istvan
> > >
> > > On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth  wrote:
> > >
> > > > Hi!
> > > >
> > > > A question of process: Should we backport fixes to the 4.16 branch at
> > our
> > > > discretion, or is backporting those handled by the release manager
> > > (Xinyi) ?
> > > >
> > > > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth 
> wrote:
> > > >
> > > >> Hi!
> > > >>
> > > >> Thanks for everyone's effort on fixing the flakey tests.
> > > >> However, our ASF Jenkins runs are still very unstable, and I am
> afraid
> > > >> that 

[jira] [Created] (PHOENIX-6307) Build and release official binary distributions with each HBase profile

2021-01-07 Thread Istvan Toth (Jira)
Istvan Toth created PHOENIX-6307:


 Summary: Build and release official binary distributions with each 
HBase profile
 Key: PHOENIX-6307
 URL: https://issues.apache.org/jira/browse/PHOENIX-6307
 Project: Phoenix
  Issue Type: Improvement
  Components: core
Affects Versions: 5.1.0, 4.16.0
Reporter: Istvan Toth
Assignee: Istvan Toth


The release scripts curently assume that we release a single binary artifact 
for Phoenix.

Change the release scripts, so that binaries are built ans released for each 
HBase profile.

Also revert the change that omits the phoenix-compatibility classes from 
phoenix-server, as it not needed, and a potentical source of problems with this 
approach.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [Discuss] Releasing Phoenix 4.16

2021-01-07 Thread Xinyi Yan
If we can modify the dev/create-release scripts and make them work for the
4.16 release with this hbase.profile option, it would make our life much
easier to release multiple HBase profiles from the single branch in the
future too(the master branch will have a release shortly right?). Geoffrey
and Istvan, what do you think?

Thanks,
Xinyi

On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby  wrote:

> Thanks for bringing up the default branch issue, Istvan, I've been meaning
> to start a conversation about it on this list.
>
> As part of PHOENIX-5435, I changed the default 4.x HBase release to 1.5 and
> the default 5.x HBase to 2.3 because the WAL annotation feature introduced
> by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc hook
> introduced in HBASE-22623). That means that all tests of that feature must
> no-op when run against an earlier HBase, which means that it would never be
> tested in our CI pipelines if the default was 1.3 or 2.1.
>
> This has come up quite a few times recently. In particular, the new
> indexing framework runs in a degraded state against HBase 2.1 and 2.2
> (still better than the old indexing framework though!), because we lack
> support in 2.1 for Filters on raw scans and can't use the "max lookback
> age" feature with HBase 2.1 or 2.2, which keeps major compactions from
> messing with index rebuilds. That's why lots of indexing tests no-op or
> have to make different assertions when run against 2.1 or 2.2, so we only
> properly test the indexing framework if CI and local developer
> tests run against HBase 2.3 or up.
>
> As for the release, don't we usually release separate binaries that are
> suitable for all the HBase versions we support? Back before we had a
> unified 4.x branch, I believe we used to release from 3 different
> 4.x-HBase-* branches each time. I've been assuming that part of the release
> process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
> 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
>
> Geoffrey
>
> On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth  wrote:
>
> > Two more issues came to my mind:
> >
> > In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> > This causes two conflicts:
> > - Our documentation says that the default profile is the lowest
> supported,
> > which is no longer true
> > - Unless we change our binary release process, the published maven
> > artifacts and binaries will also be built with HBase 1.5, thus they will
> be
> > incompatible with Hbase 1.3 and 1.4
> >
> > This should be addressed before release.
> > One possible solution:
> > * Add an "oldest" hbase.profile option which always chooses the oldest
> > supported version
> > * Update the release scripts to specify this profile
> > * Update the docs.
> >
> > I have also adopted the hbase release scripts to Phoenix, they are
> present
> > in dev/create-release in the master branch, but
> > should be able to perform 4.16 release as well. I have used this for the
> > phoenix-thirdparty, omid, and tephra releases, but no live phoenix
> releases
> > have been done with them yet, obviously.
> >
> > regards
> > Istvan
> >
> > On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth  wrote:
> >
> > > Hi!
> > >
> > > A question of process: Should we backport fixes to the 4.16 branch at
> our
> > > discretion, or is backporting those handled by the release manager
> > (Xinyi) ?
> > >
> > > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth  wrote:
> > >
> > >> Hi!
> > >>
> > >> Thanks for everyone's effort on fixing the flakey tests.
> > >> However, our ASF Jenkins runs are still very unstable, and I am afraid
> > >> that the single clean 4.x run was down more to luck than to fixing the
> > >> underlying problem.
> > >> While I do not consider this a blocker, fixing this would make the
> pre-
> > >> and post commit tests far more useful, and let us start with a clean
> > slate
> > >> for the next release.
> > >> I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to
> > >> track the generic cluster setup issue, but my attempts to fix this, or
> > at
> > >> least figure out the root cause have been unsuccessful so far.
> > >>
> > >> I ask for your help in fixing this issue. I have added some
> inconclusive
> > >> analysis to the ticket, and the full failsafe output directory is
> > >> downloadable as artifacts from the failed multibranch runs (stored
> for a
> > >> few days), which should hold the answer to this riddle.
> > >>
> > >> regards,
> > >> Istvan
> > >>
> > >> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan  wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> We finally have a stable 4.x branch build after many flapper test
> > fixing
> > >>> contributions from the community. I'm going to fork a new
> branch(4.16)
> > >>> from
> > >>> the current 4.x branch for the 4.16 release. As a cutoff, I will not
> > >>> include any new features other than PHOENIX-6211 and bug fix. Please
> > let
> > >>> me
> > >>> know if you have any questions or 

[jira] [Updated] (PHOENIX-6211) Paged scan filters

2021-01-07 Thread Kadir OZDEMIR (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kadir OZDEMIR updated PHOENIX-6211:
---
Attachment: PHOENIX-6211.master.010.patch

> Paged scan filters
> --
>
> Key: PHOENIX-6211
> URL: https://issues.apache.org/jira/browse/PHOENIX-6211
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0, 4.14.3
>Reporter: Kadir OZDEMIR
>Assignee: Kadir OZDEMIR
>Priority: Critical
> Fix For: 4.16.0
>
> Attachments: PHOENIX-6211.4.x.001.patch, PHOENIX-6211.4.x.002.patch, 
> PHOENIX-6211.4.x.004.patch, PHOENIX-6211.4.x.005.patch, 
> PHOENIX-6211.master.001.patch, PHOENIX-6211.master.002.patch, 
> PHOENIX-6211.master.003.patch, PHOENIX-6211.master.004.patch, 
> PHOENIX-6211.master.005.patch, PHOENIX-6211.master.006.patch, 
> PHOENIX-6211.master.007.patch, PHOENIX-6211.master.008.patch, 
> PHOENIX-6211.master.009.patch, PHOENIX-6211.master.010.patch
>
>
> Phoenix performs two main operations on the server side: aggregation and 
> filtering. However, currently there is no internal Phoenix paging capability, 
> and thus server side operations can take long enough to lead to HBase client 
> timeouts. PHOENIX-5998 and PHOENIX-6207 are for providing the paging 
> capability for ungrouped and grouped aggregate operations. This improvement 
> Jira is for adding the paging capability for scan filters. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-6211) Paged scan filters

2021-01-07 Thread Kadir OZDEMIR (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kadir OZDEMIR updated PHOENIX-6211:
---
Attachment: PHOENIX-6211.4.x.005.patch

> Paged scan filters
> --
>
> Key: PHOENIX-6211
> URL: https://issues.apache.org/jira/browse/PHOENIX-6211
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0, 4.14.3
>Reporter: Kadir OZDEMIR
>Assignee: Kadir OZDEMIR
>Priority: Critical
> Fix For: 4.16.0
>
> Attachments: PHOENIX-6211.4.x.001.patch, PHOENIX-6211.4.x.002.patch, 
> PHOENIX-6211.4.x.004.patch, PHOENIX-6211.4.x.005.patch, 
> PHOENIX-6211.master.001.patch, PHOENIX-6211.master.002.patch, 
> PHOENIX-6211.master.003.patch, PHOENIX-6211.master.004.patch, 
> PHOENIX-6211.master.005.patch, PHOENIX-6211.master.006.patch, 
> PHOENIX-6211.master.007.patch, PHOENIX-6211.master.008.patch, 
> PHOENIX-6211.master.009.patch
>
>
> Phoenix performs two main operations on the server side: aggregation and 
> filtering. However, currently there is no internal Phoenix paging capability, 
> and thus server side operations can take long enough to lead to HBase client 
> timeouts. PHOENIX-5998 and PHOENIX-6207 are for providing the paging 
> capability for ungrouped and grouped aggregate operations. This improvement 
> Jira is for adding the paging capability for scan filters. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-6211) Paged scan filters

2021-01-07 Thread Kadir OZDEMIR (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kadir OZDEMIR updated PHOENIX-6211:
---
Attachment: PHOENIX-6211.4.x.004.patch

> Paged scan filters
> --
>
> Key: PHOENIX-6211
> URL: https://issues.apache.org/jira/browse/PHOENIX-6211
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0, 4.14.3
>Reporter: Kadir OZDEMIR
>Assignee: Kadir OZDEMIR
>Priority: Critical
> Fix For: 4.16.0
>
> Attachments: PHOENIX-6211.4.x.001.patch, PHOENIX-6211.4.x.002.patch, 
> PHOENIX-6211.4.x.004.patch, PHOENIX-6211.master.001.patch, 
> PHOENIX-6211.master.002.patch, PHOENIX-6211.master.003.patch, 
> PHOENIX-6211.master.004.patch, PHOENIX-6211.master.005.patch, 
> PHOENIX-6211.master.006.patch, PHOENIX-6211.master.007.patch, 
> PHOENIX-6211.master.008.patch, PHOENIX-6211.master.009.patch
>
>
> Phoenix performs two main operations on the server side: aggregation and 
> filtering. However, currently there is no internal Phoenix paging capability, 
> and thus server side operations can take long enough to lead to HBase client 
> timeouts. PHOENIX-5998 and PHOENIX-6207 are for providing the paging 
> capability for ungrouped and grouped aggregate operations. This improvement 
> Jira is for adding the paging capability for scan filters. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [Discuss] Releasing Phoenix 4.16

2021-01-07 Thread Geoffrey Jacoby
Thanks for bringing up the default branch issue, Istvan, I've been meaning
to start a conversation about it on this list.

As part of PHOENIX-5435, I changed the default 4.x HBase release to 1.5 and
the default 5.x HBase to 2.3 because the WAL annotation feature introduced
by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc hook
introduced in HBASE-22623). That means that all tests of that feature must
no-op when run against an earlier HBase, which means that it would never be
tested in our CI pipelines if the default was 1.3 or 2.1.

This has come up quite a few times recently. In particular, the new
indexing framework runs in a degraded state against HBase 2.1 and 2.2
(still better than the old indexing framework though!), because we lack
support in 2.1 for Filters on raw scans and can't use the "max lookback
age" feature with HBase 2.1 or 2.2, which keeps major compactions from
messing with index rebuilds. That's why lots of indexing tests no-op or
have to make different assertions when run against 2.1 or 2.2, so we only
properly test the indexing framework if CI and local developer
tests run against HBase 2.3 or up.

As for the release, don't we usually release separate binaries that are
suitable for all the HBase versions we support? Back before we had a
unified 4.x branch, I believe we used to release from 3 different
4.x-HBase-* branches each time. I've been assuming that part of the release
process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).

Geoffrey

On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth  wrote:

> Two more issues came to my mind:
>
> In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> This causes two conflicts:
> - Our documentation says that the default profile is the lowest supported,
> which is no longer true
> - Unless we change our binary release process, the published maven
> artifacts and binaries will also be built with HBase 1.5, thus they will be
> incompatible with Hbase 1.3 and 1.4
>
> This should be addressed before release.
> One possible solution:
> * Add an "oldest" hbase.profile option which always chooses the oldest
> supported version
> * Update the release scripts to specify this profile
> * Update the docs.
>
> I have also adopted the hbase release scripts to Phoenix, they are present
> in dev/create-release in the master branch, but
> should be able to perform 4.16 release as well. I have used this for the
> phoenix-thirdparty, omid, and tephra releases, but no live phoenix releases
> have been done with them yet, obviously.
>
> regards
> Istvan
>
> On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth  wrote:
>
> > Hi!
> >
> > A question of process: Should we backport fixes to the 4.16 branch at our
> > discretion, or is backporting those handled by the release manager
> (Xinyi) ?
> >
> > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth  wrote:
> >
> >> Hi!
> >>
> >> Thanks for everyone's effort on fixing the flakey tests.
> >> However, our ASF Jenkins runs are still very unstable, and I am afraid
> >> that the single clean 4.x run was down more to luck than to fixing the
> >> underlying problem.
> >> While I do not consider this a blocker, fixing this would make the pre-
> >> and post commit tests far more useful, and let us start with a clean
> slate
> >> for the next release.
> >> I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to
> >> track the generic cluster setup issue, but my attempts to fix this, or
> at
> >> least figure out the root cause have been unsuccessful so far.
> >>
> >> I ask for your help in fixing this issue. I have added some inconclusive
> >> analysis to the ticket, and the full failsafe output directory is
> >> downloadable as artifacts from the failed multibranch runs (stored for a
> >> few days), which should hold the answer to this riddle.
> >>
> >> regards,
> >> Istvan
> >>
> >> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan  wrote:
> >>
> >>> Hi,
> >>>
> >>> We finally have a stable 4.x branch build after many flapper test
> fixing
> >>> contributions from the community. I'm going to fork a new branch(4.16)
> >>> from
> >>> the current 4.x branch for the 4.16 release. As a cutoff, I will not
> >>> include any new features other than PHOENIX-6211 and bug fix. Please
> let
> >>> me
> >>> know if you have any questions or concerns.
> >>>
> >>> Thanks,
> >>> Xinyi
> >>>
> >>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani 
> wrote:
> >>>
> >>> > An update on test stability:
> >>> >
> >>> > As per recent 4.x build results, we are left with very few flappers
> and
> >>> > specifically
> >>> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR
> >>> > precommit results)
> >>> > results without any test failure.
> >>> >
> >>> >
> >>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan 
> wrote:
> >>> >
> >>> > > Yes, we are currently working on fixing the 4.x branch test
> flappers
> >>> > while
> >>> > > waiting for the 

[jira] [Updated] (PHOENIX-6148) [SchemaExtractionTool]DDL parsing exception in Phoenix in view name

2021-01-07 Thread Swaroopa Kadam (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam updated PHOENIX-6148:

Fix Version/s: (was: 4.17.0)
   (was: 4.16.1)
   4.16.0

> [SchemaExtractionTool]DDL parsing exception in Phoenix in view name
> ---
>
> Key: PHOENIX-6148
> URL: https://issues.apache.org/jira/browse/PHOENIX-6148
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Swaroopa Kadam
>Assignee: Swaroopa Kadam
>Priority: Major
> Fix For: 4.16.0
>
> Attachments: PHOENIX-6148.4.x.patch, PHOENIX-6148.patch
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> 584 org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): 
> Syntax error. Encountered ".0" at line 1, column 28.
> 585 at 
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> 586 at 
> org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> Seeing this when parsing create view statement generated from 
> SchemaExtractionTool as below: 
> CREAT VIEW TEST.04KA AS SELECT * FROM TEST.TABLE_NAME;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-6120) Change IndexMaintainer for SINGLE_CELL_ARRAY_WITH_OFFSETS indexes. Currently it assumes data and index table having the same storage and encoding format.

2021-01-07 Thread Gokcen Iskender (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gokcen Iskender updated PHOENIX-6120:
-
Attachment: PHOENIX-6120-addendum.4.x.001.patch

> Change IndexMaintainer for SINGLE_CELL_ARRAY_WITH_OFFSETS indexes. Currently 
> it assumes data and index table having the same storage and encoding format.
> -
>
> Key: PHOENIX-6120
> URL: https://issues.apache.org/jira/browse/PHOENIX-6120
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Gokcen Iskender
>Priority: Major
> Attachments: PHOENIX-6120-addendum.4.x.001.patch, 
> PHOENIX-6120.4.x.001.patch
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This Jira is a followup Jira of PHOENIX-5923 where data and index might have 
> different storage and encoding formats. IndexMaintainer and protobuf needs to 
> change to handle this. Right now, index maintainer assumes both index and 
> data tables have the same format. The protobuf needs to be changed so that 
> IndexMaintainer knows about the data table storage and encoding formats
> Repro:
>  # Create a table with ONE_CELL_PER_COLUMN storage format
>  # Create an index that has covered columns and the IMMUTABLE_STORAGE_SCHEME 
> is SINGLE_CELL_ARRAY_WITH_OFFSETS
>  # Upsert rows to data table
>  # Query the covered column using index --> Null value is returned even 
> though there is a value. 
> IndexMaintainer has a 4 year old code that doesn't return a value for covered 
> columns,



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [Discuss] Releasing Phoenix 4.16

2021-01-07 Thread Istvan Toth
Two more issues came to my mind:

In PHOENIX-5435 the default profile was changed to HBase 1.5.0
This causes two conflicts:
- Our documentation says that the default profile is the lowest supported,
which is no longer true
- Unless we change our binary release process, the published maven
artifacts and binaries will also be built with HBase 1.5, thus they will be
incompatible with Hbase 1.3 and 1.4

This should be addressed before release.
One possible solution:
* Add an "oldest" hbase.profile option which always chooses the oldest
supported version
* Update the release scripts to specify this profile
* Update the docs.

I have also adopted the hbase release scripts to Phoenix, they are present
in dev/create-release in the master branch, but
should be able to perform 4.16 release as well. I have used this for the
phoenix-thirdparty, omid, and tephra releases, but no live phoenix releases
have been done with them yet, obviously.

regards
Istvan

On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth  wrote:

> Hi!
>
> A question of process: Should we backport fixes to the 4.16 branch at our
> discretion, or is backporting those handled by the release manager (Xinyi) ?
>
> On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth  wrote:
>
>> Hi!
>>
>> Thanks for everyone's effort on fixing the flakey tests.
>> However, our ASF Jenkins runs are still very unstable, and I am afraid
>> that the single clean 4.x run was down more to luck than to fixing the
>> underlying problem.
>> While I do not consider this a blocker, fixing this would make the pre-
>> and post commit tests far more useful, and let us start with a clean slate
>> for the next release.
>> I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to
>> track the generic cluster setup issue, but my attempts to fix this, or at
>> least figure out the root cause have been unsuccessful so far.
>>
>> I ask for your help in fixing this issue. I have added some inconclusive
>> analysis to the ticket, and the full failsafe output directory is
>> downloadable as artifacts from the failed multibranch runs (stored for a
>> few days), which should hold the answer to this riddle.
>>
>> regards,
>> Istvan
>>
>> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan  wrote:
>>
>>> Hi,
>>>
>>> We finally have a stable 4.x branch build after many flapper test fixing
>>> contributions from the community. I'm going to fork a new branch(4.16)
>>> from
>>> the current 4.x branch for the 4.16 release. As a cutoff, I will not
>>> include any new features other than PHOENIX-6211 and bug fix. Please let
>>> me
>>> know if you have any questions or concerns.
>>>
>>> Thanks,
>>> Xinyi
>>>
>>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani  wrote:
>>>
>>> > An update on test stability:
>>> >
>>> > As per recent 4.x build results, we are left with very few flappers and
>>> > specifically
>>> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR
>>> > precommit results)
>>> > results without any test failure.
>>> >
>>> >
>>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan  wrote:
>>> >
>>> > > Yes, we are currently working on fixing the 4.x branch test flappers
>>> > while
>>> > > waiting for the PHOENIX-5435. After that, I will try to have an RC
>>> ASAP.
>>> > >
>>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
>>> ankitsingha...@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > I see that both the blockers listed here PHOENIX-5712 and
>>> PHOENIX-6241
>>> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA query
>>> > there
>>> > > > is no Jira marked as a blocker for 4.16 except the one related to
>>> > > > documentation
>>> > > > for "splittable catalog table".
>>> > > >
>>> > > > Xinyi, so are we good to start the release process now?
>>> > > >
>>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan 
>>> wrote:
>>> > > >
>>> > > > > Thanks for replying and providing suggestions. I looked at the
>>> wrong
>>> > > > result
>>> > > > > Jira list that Daniel provided and did some local testing, and
>>> here
>>> > is
>>> > > > the
>>> > > > > result:
>>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot
>>> > > reproduce
>>> > > > > it.
>>> > > > > [More information is required] PHOENIX-4504 cannot reproduce it
>>> but
>>> > > > someone
>>> > > > > claimed that he had a similar issue.
>>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
>>> > > > >
>>> > > > > Based on my finding, I think it's better to have more frequent
>>> > > > housekeeping
>>> > > > > and resolve unreproducible bugs especially since many of them are
>>> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6).
>>> Since I
>>> > > still
>>> > > > > need time to work on the blocker Jira(PHOENIX-5712,
>>> PHOENIX-6241) and
>>> > > fix
>>> > > > > test flappers, if you want to fix "unusual query" bugs, feel
>>> free to
>>> > do
>>> > > > so.
>>> > > > >
>>> > > > >
>>> > > > > Sincerely,
>>> > > > > Xinyi
>>> > > > >
>>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal 
>>> > 

[jira] [Resolved] (PHOENIX-6306) slf4j exclusion from phoenix-server doesn't work

2021-01-07 Thread Istvan Toth (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Istvan Toth resolved PHOENIX-6306.
--
Fix Version/s: 5.1.0
   Resolution: Fixed

Committed to master.
Does not apply to 4.x
Thanks for the review [~vjasani]

> slf4j exclusion from phoenix-server doesn't work
> 
>
> Key: PHOENIX-6306
> URL: https://issues.apache.org/jira/browse/PHOENIX-6306
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.0
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 5.1.0
>
>
> We are excluding the non-existing org.slf4j:slf4j-log4j artifact instead of 
> the org.slf4j:slf4j-log4j12 artifact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (PHOENIX-6306) slf4j exclusion from phoenix-server doesn't work

2021-01-07 Thread Istvan Toth (Jira)
Istvan Toth created PHOENIX-6306:


 Summary: slf4j exclusion from phoenix-server doesn't work
 Key: PHOENIX-6306
 URL: https://issues.apache.org/jira/browse/PHOENIX-6306
 Project: Phoenix
  Issue Type: Bug
  Components: core
Affects Versions: 5.1.0
Reporter: Istvan Toth
Assignee: Istvan Toth


We are excluding the non-existing org.slf4j:slf4j-log4j artifact instead of the 
org.slf4j:slf4j-log4j12 artifact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (PHOENIX-6305) Throttling decision does not take offheap memstore size into account

2021-01-07 Thread Istvan Toth (Jira)
Istvan Toth created PHOENIX-6305:


 Summary: Throttling decision does not take offheap memstore size 
into account
 Key: PHOENIX-6305
 URL: https://issues.apache.org/jira/browse/PHOENIX-6305
 Project: Phoenix
  Issue Type: Improvement
  Components: core
Affects Versions: 5.0.0
Reporter: Ankit Singhal
Assignee: Istvan Toth






--
This message was sent by Atlassian Jira
(v8.3.4#803005)